Enable crossgen tests

[billwert]

This turns on benchmarking for a few binaries to track crossgen.

[billwert]

/azp run dotnet-runtime-perf

[azure-pipelines]

Azure Pipelines could not run because the pipeline triggers exclude this branch/path.

[billwert]

/azp run

[azure-pipelines]

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

[billwert]

/azp help

[azure-pipelines]

Supported commands

• help:
• Get descriptions, examples and documentation about supported commands
• Example: help "command_name"
• list:
• List all pipelines for this repository using a comment.
• Example: "list"
• run:
• Run all pipelines or specific pipelines for this repository using a comment. Use this command by itself to trigger all related pipelines, or specify specific pipelines to run.
• Example: "run" or "run pipeline_name, pipeline_name, pipeline_name"
• where:
• Report back the Azure DevOps orgs that are related to this repository and org
• Example: "where"

See additional documentation.

[billwert]

/azp list

[azure-pipelines]

CI/CD Pipelines for this repository:

• coreclr-gc-longrunning
• coreclr-gc-simulator
• runtime-coreclr outerloop
• runtime-coreclr crossgen2
• runtime-libraries outerloop
• runtime-libraries outerloop-windows
• runtime-libraries outerloop-linux
• runtime-libraries outerloop-osx
• runtime
• runtime-libraries enterprise-linux
• runtime-libraries stress-http
• runtime-libraries stress-ssl
• runtime-live-build
• runtime-coreclr crossgen2 outerloop
• coreclr-release-outerloop-nightly
• sync-runtime-to-mono
• dotnet-runtime-perf

[billwert]

/azp run runtime

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[billwert]

/azp run dotnet-runtime-perf

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[billwert]

This PR does some support work that is necessary when the PR from arcade comes in which will enable the tests, so it has to go first. I'd like to get this merged ASAP and then put the PR in arcade which in theory will flow in tomorrow.

[BruceForstall]

I'm not familiar with this code, so I can't comment much on it. Hopefully you can test if it works before merging? Or do you need to wait for arcade changes to come in first?

[billwert]

I'm not familiar with this code, so I can't comment much on it. Hopefully you can test if it works before merging? Or do you need to wait for arcade changes to come in first?

Good question. Before I force pushed and cleaned it up I had the change that's going into arcade in this PR. Here's output from the last CI run showing it worked:

C:\h\w\A4080960\w\B66A0983\e>py -3 C:\h\w\A4080960\p\performance\src\scenarios\crossgen\test.py crossgen --test-name Microsoft.CodeAnalysis.CSharp.dll --core-root C:\h\w\A4080960\p\Core_Root 
[2020/02/26 18:22:21][INFO] ----------------------------------------------
[2020/02/26 18:22:21][INFO] Initializing logger 2020-02-26 18:22:21.123407
[2020/02/26 18:22:21][INFO] ----------------------------------------------
[2020/02/26 18:22:21][INFO] $ C:\h\w\A4080960\w\B66A0983\u\Startup\Startup.exe --app-exe C:\h\w\A4080960\p\Core_Root\crossgen.exe --metric-type ProcessTime --trace-file-name "Crossgen Throughput - Microsoft.CodeAnalysis.CSharp.dll_startup.etl" --process-will-exit true --iterations 1 --timeout 50 --warmup true --gui-app false --working-dir C:\h\w\A4080960\p\Core_Root --report-json-path traces\perf-lab-report.json --trace-directory traces --scenario-name "Crossgen Throughput - Microsoft.CodeAnalysis.CSharp.dll" --app-args "/nologo /p C:\h\w\A4080960\p\Core_Root C:\h\w\A4080960\p\Core_Root\Microsoft.CodeAnalysis.CSharp.dll"
[2020/02/26 18:22:24][INFO] Running C:\h\w\A4080960\p\Core_Root\crossgen.exe (args: "/nologo /p C:\h\w\A4080960\p\Core_Root C:\h\w\A4080960\p\Core_Root\Microsoft.CodeAnalysis.CSharp.dll")
[2020/02/26 18:22:24][INFO] Working Directory: C:\h\w\A4080960\p\Core_Root
[2020/02/26 18:22:24][INFO] Iteration set up:  (args: )
[2020/02/26 18:22:24][INFO] Iteration clean up:  (args: )
[2020/02/26 18:22:24][INFO] =============== Warm up ================
[2020/02/26 18:22:24][INFO] ***Test***
[2020/02/26 18:22:30][INFO] Native image C:\h\w\A4080960\p\Core_Root\Microsoft.CodeAnalysis.CSharp.ni.dll generated successfully.
[2020/02/26 18:22:30][INFO] 
[2020/02/26 18:22:30][INFO] 
[2020/02/26 18:22:30][INFO] 
[2020/02/26 18:22:30][INFO] 
[2020/02/26 18:22:30][INFO] =============== Iteration 0 ================
[2020/02/26 18:22:30][INFO] ***Test***
[2020/02/26 18:22:34][INFO] Native image C:\h\w\A4080960\p\Core_Root\Microsoft.CodeAnalysis.CSharp.ni.dll generated successfully.
[2020/02/26 18:22:34][INFO] 
[2020/02/26 18:22:34][INFO] 
[2020/02/26 18:22:34][INFO] 
[2020/02/26 18:22:34][INFO] 
[2020/02/26 18:22:34][INFO] Parsing..
[2020/02/26 18:22:34][INFO] Metric         |Average        |Min            |Max
[2020/02/26 18:22:34][INFO] ---------------|---------------|---------------|---------------
[2020/02/26 18:22:34][INFO] Process Time   |3905.544 ms    |3905.544 ms    |3905.544 ms
[2020/02/26 18:22:34][INFO] Time on Thread |3680.664 ms    |3680.664 ms    |3680.664 ms
[2020/02/26 18:22:34][INFO] =============== Profile Iteration ================
[2020/02/26 18:22:35][INFO] ***Test***
[2020/02/26 18:22:39][INFO] Native image C:\h\w\A4080960\p\Core_Root\Microsoft.CodeAnalysis.CSharp.ni.dll generated successfully.
[2020/02/26 18:22:39][INFO] 
[2020/02/26 18:22:39][INFO] 
[2020/02/26 18:22:39][INFO] 
[2020/02/26 18:22:39][INFO] 
[2020/02/26 18:22:40][INFO] Merging profile..

(It only ran one iteration beacuse it wasn't set to upload to the lab. It will run 5 normally.)

[BruceForstall]

cc @briansull

[billwert]

Merging as the perf legs are passed and that's the only thing impacted by this change.