Update SuperPMI CI automation #60375
Conversation
A number of changes: 1. Move the SuperPMI replay pipeline to the public instance, so it can be triggered by GitHub PRs. 2. Rename SuperPMI collection scripts to contain "collect" in their names, to distinguish them from the "replay" scripts. 3. Remove a lot of unused copy/paste cruft 4. Create a new azdo_pipelines_util.py script for all the CI scripts to depend on, so they don't import the superpmi.py script, or each other. 5. Some changes to simplify the Python imports and be more consistent in imported API usage.
dotnet-issue-labeler
bot
added
the
area-CodeGen-coreclr
|
Tagging subscribers to this area: @JulieLeeMSFT Issue DetailsA number of changes:
|
Don't overwrite the existing collection
|
@kunalspathak This change looks good to me. Can you check the Antigen results to tell me if anything looks wrong? I'm not sure if it's supposed to be clean, or if there are new failures here due to my change. https://dev.azure.com/dnceng/public/_build/results?buildId=1419755&view=results |
|
I pushed a change to not overwrite the superpmi collection. I'd like to:
|
|
Wonder why it doesn't show superpmi-replay runs in the checks - https://dev.azure.com/dnceng/public/_build/results?buildId=1419680&view=results @BruceForstall - I think one of the only thing remaining from #59559 is having |
| @@ -30,10 +30,19 @@ | |||
| <SuperpmiLogsLocation>%HELIX_WORKITEM_UPLOAD_ROOT%</SuperpmiLogsLocation> | |||
| <!-- Workaround until https://github.com/dotnet/arcade/pull/6179 is not available --> | |||
| <HelixResultsDestinationDir>$(BUILD_SOURCESDIRECTORY)\artifacts\helixresults</HelixResultsDestinationDir> | |||
| <WorkItemCommand>$(Python) $(ProductDirectory)\superpmi-replay.py -jit_directory $(ProductDirectory)</WorkItemCommand> | |||
| <WorkItemCommand>$(Python) $(ProductDirectory)\superpmi_replay.py -jit_directory $(ProductDirectory)</WorkItemCommand> | |||
| <WorkItemTimeout>5:00</WorkItemTimeout> | |||
There was a problem hiding this comment.
I am actually wondering if we should reduce this to 2:00 or something.
There was a problem hiding this comment.
Yeah, 5:00 might be too long.
There was a problem hiding this comment.
Looks like average times for the runs are 2:15 to 2:30, with one hitting 2:52. So maybe 3:15 would be reasonable.
| print(define_variable_format.format(name, value)) # set variable | ||
|
|
||
|
|
||
| class TempDir: |
There was a problem hiding this comment.
There is also a copy of this class in jitrollingbuild.py , can we delete that as well?
There was a problem hiding this comment.
Did you also check if there is any reusable functionality in superpmi.py that should be moved to this file or some functions from this file that can be used in superpmi.py?
There was a problem hiding this comment.
We can update jitrollingbuild.py/superpmi.py later, if needed.
Actually, I specifically did not want to have superpmi.py depend on a "...pipelines....py" file, so I decided to duplicate the code. We could add a more general "util.py" file. However, I was a little torn because there already is a "utilities.py" file for the "other" set of py scripts, and I don't really want to mix with those.
ghost
locked as resolved and limited conversation to collaborators
A number of changes:
triggered by GitHub PRs.
to distinguish them from the "replay" scripts.
on, so they don't import the superpmi.py script, or each other.
API usage.
Fixes: #59559