Enable flaky tests reporter bot in PRs #35029
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kevin940726
added
the
[Type] Automated Testing
kevin940726
requested review from
ajitbohra,
nerrad and
ntwb
as code owners
kevin940726
requested review from
tellthemachines,
noisysocks,
talldan,
adamziel,
draganescu and
gziolo
|
Size Change: +4.54 kB (0%) Total Size: 1.07 MB
ℹ️ View Unchanged
|
|
One concern I have is that a PR can be significantly behind Maybe it'd be worth investing time into making sure tests always run on the latest code in |
noisysocks
approved these changes
Sep 23, 2021
Good question! I don't really know either, but I'd imagine it's the other way around. I'll do some research. |
|
Okay I did some research. The
So I think it's the other way around. GH will always run on the latest code on trunk. |
|
I also added a note in the testing documentation. Please help review it if you think it needs more clarity :). |
|
Thanks for checking that. I say let's try it |
I'm not completely confident about that, because regularly I've seen the advice to 'rebase a PR' when a test has been fixed in |
It just happened to one of my PR and I think I know why. It's because rebasing a PR is the easiest way to re-trigger the workflow. If we only re-run the workflow, the commit will still be the outdated merged commit that never pass, we have to re-trigger the I guess we could try creating the merge commit on our end using |
Right, I see. So technically if an empty commit is pushed, that will make the workflow use the latest merge ref? |
I think so, but I haven't tested it 😅. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Continued from #34432. The bot has been enabled for a week now and is pretty stable on trunk. I think it makes sense to enable it on all PRs to unblock contributors from flaky e2e tests.
You can view all the issues the bot has created.
How has this been tested?
No way to test it for now. We just have to be sure that the bot won't run arbitrary code from all PRs opened in this repo for security reasons.
Types of changes
New feature
Checklist:
*.native.jsfiles for terms that need renaming or removal).