[Perf] Linux/arm64: 7 Regressions on 8/16/2023 6:13:11 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1eb702ce1efe3581488a37704cee47603a291069
Compare	2ed506c010b41c41c01997c81df977831f0fa217
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
CheckObjIsInterfaceYes - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	34.60 μs	66.73 μs	1.93	0.01	True
CheckArrayIsInterfaceNo - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	42.23 μs	66.74 μs	1.58	0.29	False
CheckIsInstAnyIsInterfaceYes - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	34.40 μs	66.73 μs	1.94	0.04	True
CheckObjIsInterfaceNo - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	36.60 μs	66.73 μs	1.82	0.02	True
CheckIsInstAnyIsInterfaceNo - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	36.30 μs	66.74 μs	1.84	0.02	True
CheckArrayIsInterfaceYes - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	36.68 μs	66.74 μs	1.82	0.21	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

PerfLabTests.CastingPerf.CheckObjIsInterfaceYes

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73232568389058 > 36.2238440422299.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -94.91188368627567 (T) = (0 -66734.08829810777) / Math.Sqrt((905714.6123888374 / (8)) + (5.706073941345815 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.9177178443006162 = (34798.70018232292 - 66734.08829810777) / 34798.70018232292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.7387235460993 > 44.32910540625.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/22/2023 10:35:01 AM, 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -251.68138676820354 (T) = (0 -66819.43071454899) / Math.Sqrt((24519.946352112434 / (8)) + (78709.24978590856 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.5860725520725225 = (42128.86139870208 - 66819.43071454899) / 42128.86139870208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.7337411824877 > 36.11401637740386.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -334.39463039735574 (T) = (0 -66735.3113627234) / Math.Sqrt((74105.26446431954 / (8)) + (60.82104586749099 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.9319715275966103 = (34542.59569018738 - 66735.3113627234) / 34542.59569018738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckObjIsInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73152224544073 > 38.3692867041266.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -272.9926031056872 (T) = (0 -66734.28745182567) / Math.Sqrt((99242.9998696021 / (8)) + (2.448275140323358 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.8369790052924982 = (36328.279887553595 - 66734.28745182567) / 36328.279887553595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73695978343467 > 38.38318182327975.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -202.23312856744124 (T) = (0 -66734.15043460824) / Math.Sqrt((180713.35554949474 / (8)) + (4.545200605981834 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.8364396016799459 = (36338.87571013002 - 66734.15043460824) / 36338.87571013002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.73957407801417 > 38.74307212392471.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -189.78439300211238 (T) = (0 -66739.49726234317) / Math.Sqrt((195571.75354173727 / (8)) + (37.52510022052004 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.8006500261359132 = (37064.11367763791 - 66739.49726234317) / 37064.11367763791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	ubuntu 20.04
Queue	AmpereUbuntu
Baseline	1eb702ce1efe3581488a37704cee47603a291069
Compare	2ed506c010b41c41c01997c81df977831f0fa217
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio
ScalarValueTypeObj - Duration of single invocation 📝 - Benchmark Source ADX - Test Multi Config Graph	300.28 μs	400.38 μs	1.33	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

PerfLabTests.CastingPerf2.CastingPerf.ScalarValueTypeObj

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 400.37517852564105 > 315.3061534375.
IsChangePoint: Marked as a change because one of 8/15/2023 10:39:13 PM, 8/23/2023 5:32:05 AM falls between 8/14/2023 7:19:10 AM and 8/23/2023 5:32:05 AM.
IsRegressionStdDev: Marked as regression because -24137.5892398953 (T) = (0 -400380.452382322) / Math.Sqrt((50.7310081059749 / (8)) + (130.24130265022006 / (12))) is less than -2.100922040224263 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (8) + (12) - 2, .025) and -0.33331267048151797 = (300289.99292246054 - 400380.452382322) / 300289.99292246054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository