Remove CoreCLR TargetSpecific test build jobs #35645
Conversation
Refactor targetSpecific test build Remove targetSpecific build-test-jobs.yml jobs Build targetSpecific tests in run-test-job.yml jobs
sdmaclea
force-pushed
the
runBuildsTests
branch
from
1b1d8d1 to
f5e1a72
Compare
| # Build targetSpecific managed test components | ||
| - script: $(coreClrRepoRootDir)build-test$(scriptExt) targetSpecific skipnative skipgeneratelayout skiptestwrappers $(buildConfig) $(archType) $(crossArg) $(priorityArg) ci $(librariesOverrideArg) | ||
| displayName: Build managed test components | ||
|
|
There was a problem hiding this comment.
The earliest draft of this was also adding an archive of the tests.
I was fiddling with two variants.
- Archive all tests
- Archive target specific tests built here.
I thought that if performance was really critical, no artifact would be fastest (and most reliable).
However having the test artifact would help with debugging. Having all tests in the archive is easier and probably less risky as the script ordering can stay as is.
Trying to archive only the target specific tests is harder and less performant.
- Basic target specific test archive approach:
- Alter the build order so that the target specific tests are built and archived before downloading generic tests.
- Risks -- In itself that is safe, but it does run the risk that a generic test overwrites a specific test.
- Move copy target specific test archive approach:
- Download generic tests to a
test staging directory - Build the specific tests in a
test build directory - Archive the specific tests
- Copy the specific tests onto the generic tests in the
test staging directory - Run the helix jobs from the
test staging directory - Risks -- More moving pieces
- Download generic tests to a
|
One potential other problem with this approach is when we run multiple jobs on the same tests. R2R & !R2R & JitStress* &&&&.... Every job would be building the target specific tests. |
I guess the target specific build was generating this too... Fixed Now using the locally generated one. |
|
This seems to be working now. |
|
The other disadvantage here is retrying a test run will cause a rebuild too. Not sure if this matters too much. |
While that is true, I think this will bring faster builds, less legs meaning less prone to infra errors, and less times hitting the network to download artifacts, etc. So as long as the tests are reliable enough, we shouldn't be specializing for retry. |
|
There are 5 readytorun tests which are not happy with this change. I am assuming it is because the tests were I assume therefore, these tests need to be marked |
These tests are failing in PR runs when built on OSX and run on specific platforms
|
I marked the 5 tests failing in p0 target specific.. Running an outerloop p1 run here https://dev.azure.com/dnceng/public/_build/results?buildId=627124&view=results I am expecting to have to mark some p1 tests too. |
|
Closing in favor of #35783 |
ghost
locked as resolved and limited conversation to collaborators
Refactor targetSpecific test build
Remove targetSpecific build-test-jobs.yml jobs
Build targetSpecific tests in run-test-job.yml jobs