Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pri-1 builds all failing #12173

Closed
BruceForstall opened this issue Mar 4, 2019 · 2 comments
Closed

Pri-1 builds all failing #12173

BruceForstall opened this issue Mar 4, 2019 · 2 comments
Assignees
Labels
area-Infrastructure-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' bug
Milestone

Comments

@BruceForstall
Copy link
Member

It appears dotnet/coreclr#22885 has broken all pri-1 builds, with errors like:

error : Unexpected test count. Expected > 9000, found 3145.

It appears pri-1 builds are only doing pri-0 test builds.

(note, this error was introduced specifically to find problems like this)

e.g.,
https://ci.dot.net/job/dotnet_coreclr/job/master/job/arm64_cross_checked_windows_nt/6682/
https://ci.dot.net/job/dotnet_coreclr/job/master/view/x64/job/checked_windows_nt/11480/

@echesakov
Copy link
Contributor

This doesn't block official builds and AzDO CI so removing the label - only Jenkins Pri1 tests are affected.

I think it's related to how we build the tests in Jenkins - instead of calling build.cmd then build-test.cmd we have only one call to build.cmd so suspect the priority hasn't been passed properly

@echesakov
Copy link
Contributor

echesakov commented Mar 4, 2019

The issue is introduced in dotnet/coreclr#21608 in line #3968 of build.cmd.

All the Windows Pri1 Jenkins jobs has been running Pri0 tests were ignoring test partitioning but still running Pri1 tests since that change got merged. For example, https://ci.dot.net/job/dotnet_coreclr/job/master/job/arm64_cross_checked_windows_nt/6611/consoleFull from February 14.

Actually, the change in dotnet/coreclr#22885 helped to expose that original issue.

The problem explanation:
Before dotnet/coreclr#21608 build.cmd was calling build-test.cmd with -priority=n argument. After that change this call has been transformed to

BUILD: call D:\j\workspace\arm64_cross_c---0aaef436\build-test.cmd arm64 Checked  /p:CLRTestPriorityToBuild=1

effectively ignoring test priority passed to build.cmd.

As a remedy I suggest:

  1. change Jenkins netci.groovy to be running build.cmd then build-test.cmd (the same way we run in AzDO and official build)
  2. figure out the original problem in build.cmd

Eventually, we should get rid of build-test.cmd logic inside of build.cmd.

@msftgits msftgits transferred this issue from dotnet/coreclr Jan 31, 2020
@msftgits msftgits added this to the 3.0 milestone Jan 31, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 14, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-Infrastructure-coreclr blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' bug
Projects
None yet
Development

No branches or pull requests

3 participants