Harvest the "without changes" runs from Taskcluster #865
Conversation
This change extends the set of valid task names to accept the "*-results-without-patch" tasks that are being added in web-platform-tests/wpt#14382 . The change also adds a "without_patch" label to these special runs to prepare for the next step: using these runs to calculate PR regressions (not implemented in this change).
|
Staging instance deployed by Travis CI! |
|
Staging instance deployed by Travis CI! |
shared/util.go
Outdated
| @@ -33,6 +33,10 @@ const BetaLabel = "beta" | |||
| // i.e. run from the master branch. | |||
| const MasterLabel = "master" | |||
|
|
|||
| // WithoutPatchLabel is the label for running just the affected tests on a PR | |||
| // but without the patch. | |||
| const WithoutPatchLabel = "without_patch" | |||
There was a problem hiding this comment.
So... I did say that eventually master might stop implying a full run, and this is that day.
I'm wondering if labelling the runs that aren't just the affected tests as full is a better route for our default homepage.
Alternatively (or even in conjunction with full) we can pick a more succinct name for this label. How about we go the science route, and label them as control? Or baseline? normal? prior?
... I think baseline might be best, since we use that term with the same semantic meaning in other contexts.
There was a problem hiding this comment.
@foolip suggested base (without_patch) and head (with patch) offline. They work IMHO, WDYT?
There was a problem hiding this comment.
Then I threw pr_base and pr_head into the mix. Either is fine, only possible misreading is that head could mean HEAD, i.e. master.
There was a problem hiding this comment.
Back to your first comment, Luke. I understand master eventually won't be full runs in the future, but this PR doesn't change the current status that master runs are always full runs, right?
I like pr_base and pr_head, for the overuse of the term "head" itself.
We settled down on the names of Taskcluster tasks (already landed in WPT) and labels on wpt.fyi.
|
@lukebjerring PTAL again |
Hexcles
changed the title
This change extends the set of valid task names to accept the
"*-results-without-changes" tasks that are being added in
web-platform-tests/wpt#14382 . The change also
adds a "pr_base" label to these special runs, and "pr_head" to the PR
runs with the changes to prepare for the next step: using these runs
to calculate PR regressions (not implemented in this change).
Related to #708 .