Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/arm64: 81 Regressions on 4/26/2023 12:55:36 AM #17371

Closed
performanceautofiler bot opened this issue May 4, 2023 · 5 comments
Closed

[Perf] Windows/arm64: 81 Regressions on 4/26/2023 12:55:36 AM #17371

performanceautofiler bot opened this issue May 4, 2023 · 5 comments
Assignees
@tannergooding
Labels
arch-arm64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented May 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 16.28 ns 18.62 ns 1.14 0.10 False
TryParse - Duration of single invocation 16.24 ns 18.21 ns 1.12 0.14 False
TryParse - Duration of single invocation 15.89 ns 18.28 ns 1.15 0.13 False
Parse - Duration of single invocation 9.23 ns 11.22 ns 1.22 0.34 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.Parse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.62394193716033 > 17.127759243212203.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -8.318939031379447 (T) = (0 -17.82697130136037) / Math.Sqrt((0.05508051471060187 / (12)) + (0.6734136168345647 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09595143934936938 = (16.26620547343198 - 17.82697130136037) / 16.26620547343198 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "65535")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.2095368053242 > 17.05928072428638.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -6.71689479033339 (T) = (0 -17.716617191095004) / Math.Sqrt((0.3311296271862489 / (12)) + (0.13083458887424104 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.074615656168534 = (16.486468524255777 - 17.716617191095004) / 16.486468524255777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.282743691581814 > 16.74008429125149.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -10.164787673419013 (T) = (0 -17.60702126753151) / Math.Sqrt((0.1664233449759519 / (12)) + (0.22430788564947826 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09836655652189782 = (16.030186974452445 - 17.60702126753151) / 16.030186974452445 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.217521798223327 > 9.611071100466296.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -5.59610658917433 (T) = (0 -10.5944555227979) / Math.Sqrt((0.41262416121122647 / (12)) + (0.3373640814476946 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.13350470188107558 = (9.346635708891345 - 10.5944555227979) / 9.346635708891345 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FixedTimeEquals_256Bit_AllBitsDifferent - Duration of single invocation 191.45 ns 220.95 ns 1.15 0.01 True
FixedTimeEquals_256Bit_VersusZero - Duration of single invocation 191.73 ns 210.96 ns 1.10 0.01 True
FixedTimeEquals_256Bit_CascadingErrors - Duration of single invocation 191.25 ns 227.62 ns 1.19 0.01 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_AllBitsDifferent

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 220.9496243503409 > 201.14268162707882.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -18.380717668366124 (T) = (0 -210.67609518966964) / Math.Sqrt((0.04493579477787279 / (12)) + (24.7413371669378 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.09968195980516305 = (191.57911368028292 - 210.67609518966964) / 191.57911368028292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_VersusZero

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 210.95884315110334 > 200.84268477743873.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -23.307599181947438 (T) = (0 -209.2842994676127) / Math.Sqrt((0.5087435200225127 / (12)) + (12.387912346858467 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.09276305163399115 = (191.5184624467978 - 209.2842994676127) / 191.5184624467978 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_CascadingErrors

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 227.61528655772904 > 201.03064465759493.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -14.176902992230634 (T) = (0 -209.43050242902368) / Math.Sqrt((0.31834512194663955 / (12)) + (35.11761871971109 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.09214246487460322 = (191.7611567764375 - 209.43050242902368) / 191.7611567764375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 27.99 ns 31.21 ns 1.12 0.01 True
IndexOfString - Duration of single invocation 19.96 ns 33.38 ns 1.67 0.00 True
IndexOfString - Duration of single invocation 15.94 ns 19.39 ns 1.22 0.00 True
IndexOfString - Duration of single invocation 22.62 ns 25.89 ns 1.14 0.00 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.211358992375782 > 29.826838836477663.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -59.914000068733124 (T) = (0 -31.278107083779425) / Math.Sqrt((0.026771238592144586 / (12)) + (0.0037217256261668458 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10356183261359846 = (28.34286775730776 - 31.278107083779425) / 28.34286775730776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.38006803806828 > 20.939506728047114.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -55.218982893159264 (T) = (0 -34.381013373593454) / Math.Sqrt((0.0009892028008216655 / (28)) + (0.47950418353956553 / (7))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (28) + (7) - 2, .025) and -0.7255185058000226 = (19.925033117887644 - 34.381013373593454) / 19.925033117887644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.387679546683593 > 16.736094423314672.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -73.67122324722857 (T) = (0 -19.351762813360242) / Math.Sqrt((0.04654849368864218 / (27)) + (0.0010306191422304663 / (7))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (7) - 2, .025) and -0.19714728071343357 = (16.1648972729802 - 19.351762813360242) / 16.1648972729802 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.890850228671663 > 23.679144594796025.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -55.35362516452178 (T) = (0 -25.81690644429964) / Math.Sqrt((0.06310260226533439 / (27)) + (0.00394717421138148 / (7))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (27) + (7) - 2, .025) and -0.13056012323684574 = (22.835500663498237 - 25.81690644429964) / 22.835500663498237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HasExtension - Duration of single invocation 5.13 ns 6.25 ns 1.22 0.38 False
GetExtension - Duration of single invocation 19.06 ns 22.28 ns 1.17 0.16 False
GetFileName - Duration of single invocation 31.58 ns 34.24 ns 1.08 0.05 False
Combine - Duration of single invocation 3.33 ns 4.80 ns 1.44 0.61 False
GetFileNameWithoutExtension - Duration of single invocation 39.40 ns 42.54 ns 1.08 0.05 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.HasExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.247528349265935 > 5.341030762321178.
IsChangePoint: Marked as a change because one of 1/27/2023 3:43:27 AM, 2/1/2023 6:25:50 PM, 3/15/2023 9:07:40 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -5.781618330433489 (T) = (0 -5.534214633643079) / Math.Sqrt((0.02135413771984771 / (12)) + (0.11957818313224726 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09737885591886102 = (5.043121255520411 - 5.534214633643079) / 5.043121255520411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.27594573670113 > 20.10160443284385.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -3.120896093595543 (T) = (0 -20.56403143711302) / Math.Sqrt((0.9721117661519345 / (11)) + (0.5193587193151544 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.05324426428168011 = (19.524465629193653 - 20.56403143711302) / 19.524465629193653 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.238771818190095 > 33.58905321309849.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -15.407895725301563 (T) = (0 -34.39018659188685) / Math.Sqrt((0.21424659712904298 / (11)) + (0.15101569103261683 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.07793755249752722 = (31.903690999730472 - 34.39018659188685) / 31.903690999730472 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.Combine

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.804493899982922 > 3.8435686987265347.
IsChangePoint: Marked as a change because one of 3/15/2023 4:14:49 PM, 4/11/2023 8:14:56 PM, 4/25/2023 1:02:54 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -4.391136965522186 (T) = (0 -5.04430389706627) / Math.Sqrt((0.4397494602567275 / (11)) + (0.47894597037782166 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.2733176862449372 = (3.9615438877175393 - 5.04430389706627) / 3.9615438877175393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.543542129591124 > 40.501019048684824.
IsChangePoint: Marked as a change because one of 3/3/2023 4:19:26 AM, 3/10/2023 4:14:40 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -17.107194328456785 (T) = (0 -42.52825255014681) / Math.Sqrt((0.33375946696401737 / (12)) + (0.44539242533865336 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.09573659915099779 = (38.81247791038347 - 42.52825255014681) / 38.81247791038347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCustomAttributesClassMiss - Duration of single invocation 112.85 ns 126.14 ns 1.12 0.05 False
GetCustomAttributesMethodBaseMiss - Duration of single invocation 114.51 ns 131.42 ns 1.15 0.06 False
IsDefinedMethodOverrideMiss - Duration of single invocation 435.45 ns 460.91 ns 1.06 0.05 False
GetCustomAttributesClassMissInherit - Duration of single invocation 225.45 ns 242.07 ns 1.07 0.07 False
GetCustomAttributesMethodBaseMissInherit - Duration of single invocation 163.60 ns 174.55 ns 1.07 0.04 False
IsDefinedMethodOverrideMissInherit - Duration of single invocation 436.09 ns 461.93 ns 1.06 0.06 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.GetCustomAttributesClassMiss

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.14223025296853 > 118.52900853891653.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/1/2023 1:48:25 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -26.719163608705994 (T) = (0 -130.97635042218496) / Math.Sqrt((1.2282543432336968 / (12)) + (6.887999747633474 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.15139490858823929 = (113.75449851761034 - 130.97635042218496) / 113.75449851761034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesMethodBaseMiss

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.4221733096605 > 121.01156054480323.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -23.70859606566864 (T) = (0 -137.58555461135202) / Math.Sqrt((1.3319993031664883 / (11)) + (18.26377494210067 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.19738667782011177 = (114.90486503644068 - 137.58555461135202) / 114.90486503644068 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodOverrideMiss

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 460.9089989290696 > 457.0905188971295.
IsChangePoint: Marked as a change because one of 2/7/2023 2:48:42 AM, 3/10/2023 12:53:36 AM, 3/15/2023 2:24:48 AM, 4/1/2023 5:53:03 AM, 4/18/2023 7:42:00 PM, 4/26/2023 12:55:36 AM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -7.7689771321664365 (T) = (0 -465.59022819981) / Math.Sqrt((6.427122425628869 / (13)) + (278.4733948467291 / (21))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (21) - 2, .025) and -0.06596826507233933 = (436.77681921254367 - 465.59022819981) / 436.77681921254367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesClassMissInherit

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 242.0747642257093 > 235.39252417004565.
IsChangePoint: Marked as a change because one of 1/30/2023 1:22:46 AM, 2/4/2023 3:18:53 AM, 3/9/2023 9:18:02 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -16.28507119667129 (T) = (0 -245.82443065380477) / Math.Sqrt((4.338995570394197 / (12)) + (26.286602705526633 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09009338780227705 = (225.50767980476257 - 245.82443065380477) / 225.50767980476257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.GetCustomAttributesMethodBaseMissInherit

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 174.5529286984179 > 170.4554003349467.
IsChangePoint: Marked as a change because one of 1/28/2023 1:22:52 PM, 2/4/2023 3:18:53 AM, 2/28/2023 5:58:27 PM, 3/4/2023 1:07:20 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -13.145917653344503 (T) = (0 -174.06399781616923) / Math.Sqrt((3.887473839605613 / (12)) + (9.252881390834936 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.06878893287195068 = (162.8609657740753 - 174.06399781616923) / 162.8609657740753 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 461.9341276293059 > 457.9954578792807.
IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 3/10/2023 12:53:36 AM, 3/15/2023 2:24:48 AM, 4/1/2023 8:54:51 PM, 4/10/2023 1:33:42 PM, 4/26/2023 5:34:01 AM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -7.475884182902106 (T) = (0 -464.4150232531295) / Math.Sqrt((20.39502063565609 / (14)) + (228.5540216279481 / (20))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (20) - 2, .025) and -0.06132510927430049 = (437.58035986794033 - 464.4150232531295) / 437.58035986794033 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseHex - Duration of single invocation 20.74 ns 22.32 ns 1.08 0.09 False
ToStringHex - Duration of single invocation 25.06 ns 28.63 ns 1.14 0.13 False
ParseSpan - Duration of single invocation 10.61 ns 11.81 ns 1.11 0.19 False
ToString - Duration of single invocation 11.35 ns 13.27 ns 1.17 0.14 False
ParseHex - Duration of single invocation 20.64 ns 22.51 ns 1.09 0.07 False
ToString - Duration of single invocation 1.63 ns 3.53 ns 2.17 0.54 False
Parse - Duration of single invocation 10.18 ns 12.17 ns 1.20 0.15 False
Parse - Duration of single invocation 15.44 ns 16.89 ns 1.09 0.07 False
TryParseSpan - Duration of single invocation 9.29 ns 10.41 ns 1.12 0.36 False
TryParse - Duration of single invocation 13.70 ns 15.28 ns 1.11 0.03 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ParseHex(value: "80000000")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.322420298727575 > 21.7723249140503.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -12.438661824551696 (T) = (0 -22.383803047215682) / Math.Sqrt((0.15746847543988857 / (12)) + (0.061928732540618814 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.07544595460245919 = (20.813508062792334 - 22.383803047215682) / 20.813508062792334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToStringHex(value: 2147483647)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.627867772750122 > 26.63297687907891.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -12.102503240167716 (T) = (0 -29.003259772381956) / Math.Sqrt((0.7918120504365049 / (12)) + (0.11250016519646752 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.12519429923713585 = (25.7762235305011 - 29.003259772381956) / 25.7762235305011 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.807924479458375 > 11.068125384288082.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -3.4280391660285487 (T) = (0 -11.929245852934004) / Math.Sqrt((1.199533227091161 / (12)) + (0.10360675412693864 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10249991374110896 = (10.820178490948392 - 11.929245852934004) / 10.820178490948392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.266462279200649 > 11.917261247806799.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -11.554582310605824 (T) = (0 -14.02192237791488) / Math.Sqrt((0.2415406831065671 / (11)) + (0.6186273512704551 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.22270079945166574 = (11.467991502257275 - 14.02192237791488) / 11.467991502257275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.511192980474572 > 21.685266682064597.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -41.45225540530285 (T) = (0 -22.366789519270124) / Math.Sqrt((0.013849550286264644 / (12)) + (0.008717868996859793 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.07871794196851555 = (20.734604152827693 - 22.366789519270124) / 20.734604152827693 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ToString(value: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5343439514912705 > 1.9210620874243296.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -33.03066267706629 (T) = (0 -3.7109949688748514) / Math.Sqrt((0.03001305091430735 / (12)) + (0.025245237374460248 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -1.1456899757904229 = (1.7295112577984646 - 3.7109949688748514) / 1.7295112577984646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.173201618816055 > 11.059219191647575.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -3.1810615395359765 (T) = (0 -11.864175860514832) / Math.Sqrt((1.2620989514050935 / (12)) + (0.5201713407181708 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10648002484936245 = (10.722449202939778 - 11.864175860514832) / 10.722449202939778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.88673258099184 > 16.155556217700447.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -38.01974471159778 (T) = (0 -16.66765153601979) / Math.Sqrt((0.006984660031193682 / (12)) + (0.012084030135929322 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.08309933582907036 = (15.388848450600657 - 16.66765153601979) / 15.388848450600657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "4")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.412946847285802 > 9.786492211731984.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -2.9505852748336214 (T) = (0 -10.648140188041486) / Math.Sqrt((1.0231671099662258 / (11)) + (0.2511402758155961 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.09809672934329175 = (9.696905476086359 - 10.648140188041486) / 9.696905476086359 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.278086364530562 > 14.352844933547422.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -19.242838847862462 (T) = (0 -15.385056213166092) / Math.Sqrt((0.056294367120850314 / (12)) + (0.03376967579899905 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10949836823976976 = (13.866677638809543 - 15.385056213166092) / 13.866677638809543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 11.38 ns 13.27 ns 1.17 0.11 False
TryParse - Duration of single invocation 16.55 ns 17.84 ns 1.08 0.08 False
ToString - Duration of single invocation 1.94 ns 3.16 ns 1.62 0.53 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int16.ToString(value: 32767)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.266327388870705 > 11.85633883729068.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -11.164539283148487 (T) = (0 -13.860256901981392) / Math.Sqrt((0.1993344365521369 / (11)) + (0.6413862101641725 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.20886312582428446 = (11.465530386271427 - 13.860256901981392) / 11.465530386271427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "32767")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.83664156746386 > 17.39194049082555.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -17.839158531747685 (T) = (0 -17.71939897054414) / Math.Sqrt((0.04567703761700002 / (12)) + (0.02776815914951093 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.07675806709610078 = (16.456249098120463 - 17.71939897054414) / 16.456249098120463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.ToString(value: 0)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.157525476972092 > 1.8746740904539705.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 1:02:54 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -25.62541489055774 (T) = (0 -3.3017799861905197) / Math.Sqrt((0.024384247005557844 / (11)) + (0.026219211903167196 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.8170441310109341 = (1.8171160126713792 - 3.3017799861905197) / 1.8171160126713792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 12.99 ns 14.26 ns 1.10 0.12 False
Parse - Duration of single invocation 13.73 ns 15.11 ns 1.10 0.05 False
TryParse - Duration of single invocation 13.81 ns 14.90 ns 1.08 0.05 False
Parse - Duration of single invocation 12.98 ns 14.19 ns 1.09 0.10 False
ToString - Duration of single invocation 1.94 ns 3.16 ns 1.63 0.54 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.TryParse(value: "127")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.26467833730132 > 13.615225195282758.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -19.377953302665134 (T) = (0 -14.217419599024362) / Math.Sqrt((0.014286561350452552 / (12)) + (0.0644748581189991 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09588849546370312 = (12.973418060209262 - 14.217419599024362) / 12.973418060209262 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.114775240810241 > 14.455837753571164.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -10.913479266801604 (T) = (0 -14.968823045266054) / Math.Sqrt((0.003585796625573003 / (11)) + (0.25071091843435256 / (22))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (22) - 2, .025) and -0.08570089865702941 = (13.787243856739842 - 14.968823045266054) / 13.787243856739842 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.898140153293468 > 14.49080695636078.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -22.92839383584691 (T) = (0 -15.20889424261323) / Math.Sqrt((0.040251648777564215 / (12)) + (0.016693052169650714 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10703365368072487 = (13.738420861954722 - 15.20889424261323) / 13.738420861954722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "127")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.194358735185503 > 13.56917230938211.
IsChangePoint: Marked as a change because one of 4/17/2023 2:42:45 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -17.663904934511983 (T) = (0 -14.138444819567308) / Math.Sqrt((0.04108995510064385 / (12)) + (0.03903605831592997 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09899694860894385 = (12.864862670877343 - 14.138444819567308) / 12.864862670877343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.ToString(value: 127)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.163773714660688 > 1.9851230879785147.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -22.120316628129142 (T) = (0 -3.3441152462042734) / Math.Sqrt((0.025108176919725847 / (12)) + (0.06098588239856592 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.8564774142325343 = (1.8013228820167073 - 3.3441152462042734) / 1.8013228820167073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetNow - Duration of single invocation 130.91 ns 151.84 ns 1.16 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.GetNow

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 151.83785124631996 > 136.24549228434262.
IsChangePoint: Marked as a change because one of 3/11/2023 9:11:51 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -93.45911166859169 (T) = (0 -151.6225227557237) / Math.Sqrt((0.48214840584025803 / (11)) + (0.23667073960271034 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.16740355588108183 = (129.88012756333322 - 151.6225227557237) / 129.88012756333322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinarySearch - Duration of single invocation 10.50 ns 12.27 ns 1.17 0.04 False
BinarySearch - Duration of single invocation 3.73 ns 5.57 ns 1.49 0.45 False
SequenceCompareTo - Duration of single invocation 40.62 ns 50.34 ns 1.24 0.01 True
Reverse - Duration of single invocation 63.11 ns 99.72 ns 1.58 0.04 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.BinarySearch(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.265871771799892 > 10.97324216573782.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -28.646840629297962 (T) = (0 -12.278031407957007) / Math.Sqrt((0.007793737314465928 / (12)) + (0.07917107235642877 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.1793450904960341 = (10.410889490193961 - 12.278031407957007) / 10.410889490193961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.570616948973451 > 3.819512625539557.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -17.42375171488895 (T) = (0 -5.580431781343724) / Math.Sqrt((0.08050541177787576 / (12)) + (0.08750359639889999 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.47659151288694634 = (3.779265783827503 - 5.580431781343724) / 3.779265783827503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.33878065832626 > 43.02362709213771.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -35.8031454517337 (T) = (0 -49.84788838655911) / Math.Sqrt((0.41326106486656716 / (12)) + (0.4433354095025147 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.2016560562193655 = (41.48265897597178 - 49.84788838655911) / 41.48265897597178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.Reverse(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.72336697321663 > 66.19004156373491.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/21/2023 9:40:41 PM, 2/1/2023 6:25:50 PM, 2/7/2023 3:56:56 PM, 3/10/2023 4:14:40 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -200.8796649899259 (T) = (0 -99.78467718981261) / Math.Sqrt((0.46617866344503894 / (19)) + (0.1631654907573089 / (16))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (16) - 2, .025) and -0.6004766512429244 = (62.346849679012934 - 99.78467718981261) / 62.346849679012934 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 15.42 ns 16.48 ns 1.07 0.11 False
Parse - Duration of single invocation 10.36 ns 12.14 ns 1.17 0.23 False
ParseSpan - Duration of single invocation 10.31 ns 11.63 ns 1.13 0.21 False
TryParse - Duration of single invocation 14.06 ns 15.38 ns 1.09 0.09 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.Parse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.484824476164125 > 16.201712652702593.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -20.607284511981383 (T) = (0 -16.643212924497966) / Math.Sqrt((0.014390473403414782 / (12)) + (0.05326257487346465 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.08049613570011083 = (15.403306291062249 - 16.643212924497966) / 15.403306291062249 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.Parse(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.139205852525011 > 10.807856668661048.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -2.89190462478501 (T) = (0 -11.55503227119007) / Math.Sqrt((1.3416948100239865 / (12)) + (0.03635189270595687 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.09203053183997074 = (10.581235537179447 - 11.55503227119007) / 10.581235537179447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.ParseSpan(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.633742521472747 > 10.747131508551798.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -27.943699808062522 (T) = (0 -11.525813651197325) / Math.Sqrt((0.020040938151877286 / (11)) + (0.005945446417526892 / (24))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (24) - 2, .025) and -0.12396898281665365 = (10.254565586243995 - 11.525813651197325) / 10.254565586243995 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.37809388955967 > 14.754560890528385.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -24.974879369789946 (T) = (0 -15.415169628079546) / Math.Sqrt((0.01931913443055497 / (12)) + (0.03420349977243859 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.1002814278285956 = (14.010206151076611 - 15.415169628079546) / 14.010206151076611 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSetReverse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 2.01 μs 2.13 μs 1.06 0.09 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSetReverse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<String>.IList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1308296085255445 > 2.109864025451671.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -59.02174560355091 (T) = (0 -2129.83025496637) / Math.Sqrt((46.686876436704125 / (12)) + (2.392709294560492 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.05867076300480794 = (2011.796612689395 - 2129.83025496637) / 2011.796612689395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse2 - Duration of single invocation 33.88 ns 36.66 ns 1.08 0.03 False
TryParse4 - Duration of single invocation 61.52 ns 67.96 ns 1.10 0.00 False
Parse3 - Duration of single invocation 40.77 ns 45.34 ns 1.11 0.03 True
TryParse3 - Duration of single invocation 37.62 ns 52.10 ns 1.38 0.03 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.TryParse2

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.66354185792409 > 35.838042676527444.
IsChangePoint: Marked as a change because one of 3/17/2023 7:47:41 PM, 4/11/2023 5:52:15 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -13.64513310284177 (T) = (0 -37.351384214938356) / Math.Sqrt((0.3066586355312864 / (12)) + (0.4471411422541515 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.08489201642979881 = (34.428665387229614 - 37.351384214938356) / 34.428665387229614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.TryParse4

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.96103728041797 > 64.70394425057242.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -34.39987394786014 (T) = (0 -67.7475433739029) / Math.Sqrt((0.25412398066891845 / (12)) + (0.20960565173077073 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09766385824938045 = (61.719754062004654 - 67.7475433739029) / 61.719754062004654 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.Parse3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.343303454506035 > 43.49199028323505.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -18.19961187724433 (T) = (0 -46.352042675302485) / Math.Sqrt((0.5911806809365154 / (11)) + (0.44290741534678735 / (22))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (22) - 2, .025) and -0.11946941500306638 = (41.405367626926655 - 46.352042675302485) / 41.405367626926655 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.TryParse3

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.099100543219755 > 39.57006920997769.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -74.62519033161547 (T) = (0 -52.827752112880944) / Math.Sqrt((0.14191346101819216 / (12)) + (0.6272503068470742 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.39608661875581386 = (37.83988142509438 - 52.827752112880944) / 37.83988142509438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 14.58 ns 15.98 ns 1.10 0.08 False
ParseSpan - Duration of single invocation 15.79 ns 17.36 ns 1.10 0.07 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParse(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.980038046696496 > 15.284532572360051.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -15.173823612795637 (T) = (0 -15.933995088322732) / Math.Sqrt((0.09469845260504375 / (12)) + (0.010429621235246487 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.09528166263279987 = (14.547851600127359 - 15.933995088322732) / 14.547851600127359 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ParseSpan(value: "12345")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.36222170067147 > 16.794768422897924.
IsChangePoint: Marked as a change because one of 1/20/2023 9:27:13 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -6.73126739967738 (T) = (0 -17.06960467119794) / Math.Sqrt((0.013833752384392802 / (12)) + (0.5464622489303629 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.06789233531629597 = (15.984387289512798 - 17.06960467119794) / 15.984387289512798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 3.51 μs 3.74 μs 1.07 0.12 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7436759910958397 > 3.6837205989960813.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -11.52413779950704 (T) = (0 -3789.7487736518783) / Math.Sqrt((87.05380876155256 / (12)) + (13442.110754060706 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.07987683721364622 = (3509.426855964781 - 3789.7487736518783) / 3509.426855964781 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 715.49 ns 871.79 ns 1.22 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 871.7889996907855 > 752.9260025021198.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 3/28/2023 2:42:21 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -317.53103889217823 (T) = (0 -873.7809147410479) / Math.Sqrt((2.353758973921232 / (12)) + (1.0777542587352837 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.219397673386476 = (716.5676413949592 - 873.7809147410479) / 716.5676413949592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollectionNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ArrayList - Duration of single invocation 16.08 μs 17.98 μs 1.12 0.00 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollectionNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollectionNonGeneric<Int32>.ArrayList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.978686471738065 > 16.904922061094837.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -34.92995136254642 (T) = (0 -18005.850702133586) / Math.Sqrt((19097.483132379817 / (12)) + (30435.670052331345 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.1183285908915514 = (16100.679933237692 - 18005.850702133586) / 16100.679933237692 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Tests.Perf_WebUtility

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Decode_NoDecodingRequired - Duration of single invocation 81.32 ns 90.54 ns 1.11 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Tests.Perf_WebUtility*'

Payloads

Baseline
Compare

Histogram

System.Net.Tests.Perf_WebUtility.Decode_NoDecodingRequired

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.53964730995004 > 86.6746491773644.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -25.583361898639932 (T) = (0 -90.90357235693713) / Math.Sqrt((1.1395565437383952 / (12)) + (0.35119123110271167 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.10307045976271555 = (82.40957914555308 - 90.90357235693713) / 82.40957914555308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Method0_NoParms - Duration of single invocation 14.14 ns 18.45 ns 1.30 0.26 False
Property_Set_class - Duration of single invocation 39.12 ns 41.67 ns 1.07 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.Method0_NoParms

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.445845084214646 > 14.984619234695655.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -8.259601594125938 (T) = (0 -19.334485347678157) / Math.Sqrt((2.232894411432965 / (12)) + (1.8421204120251686 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.285178488479252 = (15.044202436469815 - 19.334485347678157) / 15.044202436469815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Reflection.Invoke.Property_Set_class

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 41.667283333333344 > 41.16092367029352.
IsChangePoint: Marked as a change because one of 1/13/2023 9:23:19 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -38.74284118426381 (T) = (0 -41.85842462177816) / Math.Sqrt((0.01553496496801199 / (12)) + (0.07411282949597156 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.06633152544711408 = (39.25460667987554 - 41.85842462177816) / 39.25460667987554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 12.62 ns 13.86 ns 1.10 0.08 False
Parse - Duration of single invocation 12.56 ns 13.65 ns 1.09 0.13 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Byte.TryParse(value: "255")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.860255838131925 > 12.776543540406728.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -7.9271313431596155 (T) = (0 -13.62985709521256) / Math.Sqrt((0.32876486175368175 / (12)) + (0.03354967117386261 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10976681798751586 = (12.281730607092172 - 13.62985709521256) / 12.281730607092172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.Parse(value: "255")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.651777550858155 > 13.16471654414016.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -21.814812926238442 (T) = (0 -13.610413414809019) / Math.Sqrt((0.02488263925698914 / (12)) + (0.01781663703646094 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09417115936090505 = (12.439016783040353 - 13.610413414809019) / 12.439016783040353 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EnumerateArray - Duration of single invocation 809.08 ns 888.28 ns 1.10 0.00 True
EnumerateArray - Duration of single invocation 810.46 ns 895.48 ns 1.10 0.00 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 888.276165087692 > 848.8029336802201.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -63.23122303351786 (T) = (0 -887.8251794841755) / Math.Sqrt((17.31514671100908 / (12)) + (3.02404238809308 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.09833806516283118 = (808.3350724556317 - 887.8251794841755) / 808.3350724556317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateArray(TestCase: ArrayOfNumbers)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 895.4822688979964 > 843.1433805687503.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -64.78189760429035 (T) = (0 -888.2606103089888) / Math.Sqrt((17.989111916158006 / (12)) + (3.3341468129831626 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.10338752617460832 = (805.0304985670318 - 888.2606103089888) / 805.0304985670318 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSet<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 15.82 μs 18.23 μs 1.15 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.23156411485248 > 16.569770277531802.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -45.99252105362969 (T) = (0 -17875.455774251725) / Math.Sqrt((3253.603702183957 / (12)) + (41835.813391837335 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.13330921801529566 = (15772.796594345242 - 17875.455774251725) / 15772.796594345242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_FileInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ctor_str - Duration of single invocation 185.84 ns 202.75 ns 1.09 0.20 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileInfo*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileInfo.ctor_str

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 202.75448312236287 > 201.7832538181338.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -6.722660100035502 (T) = (0 -204.45305316980713) / Math.Sqrt((33.581647136075276 / (11)) + (7.418717628452124 / (22))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (22) - 2, .025) and -0.06444298747056612 = (192.07515628023305 - 204.45305316980713) / 192.07515628023305 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CheckArrayIsInterfaceYes - Duration of single invocation 33.25 μs 35.32 μs 1.06 0.03 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.CheckArrayIsInterfaceYes

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.324447391055045 > 34.91859988234579.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -14.57607186074396 (T) = (0 -35465.496575576864) / Math.Sqrt((379.1107253938819 / (12)) + (504918.19709244824 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.0664463849259147 = (33255.77082624799 - 35465.496575576864) / 33255.77082624799 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 7.55 μs 8.21 μs 1.09 0.00 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.IDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.213661964684393 > 7.897695428491448.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/10/2023 12:53:36 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -23.860167835854103 (T) = (0 -8109.820086625796) / Math.Sqrt((4811.692084305634 / (12)) + (2744.5745674672094 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.07193835978488348 = (7565.565699368454 - 8109.820086625796) / 7565.565699368454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 6.09 μs 7.84 μs 1.29 0.04 False
FrozenDictionary - Duration of single invocation 7.95 μs 8.45 μs 1.06 0.02 True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.84477139471058 > 6.408697354899702.
IsChangePoint: Marked as a change because one of 1/9/2023 10:41:51 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -37.18826128250897 (T) = (0 -7498.186127288442) / Math.Sqrt((3824.9413807663263 / (12)) + (24421.120247212057 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.23072574741305577 = (6092.491477528099 - 7498.186127288442) / 6092.491477528099 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.TryGetValueFalse<Int32, Int32>.FrozenDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.447404840668314 > 8.421156193687752.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -19.468906828368823 (T) = (0 -8491.112634667941) / Math.Sqrt((1261.2532693619012 / (12)) + (11904.305120697622 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.06189341817521578 = (7996.200456030024 - 8491.112634667941) / 7996.200456030024 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Perf_Convert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToBase64CharArray - Duration of single invocation 1.31 μs 1.44 μs 1.10 0.02 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Perf_Convert*'

Payloads

Baseline
Compare

Histogram

System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: InsertLineBreaks)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.442889615654453 > 1.372292396340347.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -292.84593544610533 (T) = (0 -1443.656667440576) / Math.Sqrt((1.7908862910528969 / (12)) + (1.5269024904040525 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.1047913677077705 = (1306.723341291022 - 1443.656667440576) / 1306.723341291022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dequeue_And_Enqueue - Duration of single invocation 433.56 ns 462.49 ns 1.07 0.01 False
HeapSort - Duration of single invocation 72.38 μs 85.91 μs 1.19 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 10)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 462.4946451957174 > 455.79361629381276.
IsChangePoint: Marked as a change because one of 3/31/2023 8:29:50 AM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -89.40466500697887 (T) = (0 -461.0578036195795) / Math.Sqrt((0.7163554029678836 / (12)) + (0.6436783239117355 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.06138487693936674 = (434.39266343147466 - 461.0578036195795) / 434.39266343147466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.HeapSort(Size: 1000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.90697404371583 > 75.8848934111783.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -22.70529205555229 (T) = (0 -85714.37397237163) / Math.Sqrt((7270235.84400709 / (31)) + (30354.655848419716 / (3))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (3) - 2, .025) and -0.1507749944612738 = (74484.04282758867 - 85714.37397237163) / 74484.04282758867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SingleSerial - Duration of single invocation 1.84 μs 2.10 μs 1.14 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.SingleSerial(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1026629285714282 > 1.9361857375000004.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -131.1212303689624 (T) = (0 -2108.7878170995677) / Math.Sqrt((35.94015326948678 / (12)) + (23.54238974045334 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.14333429784595164 = (1844.4192753357754 - 2108.7878170995677) / 1844.4192753357754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Memory<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Pin - Duration of single invocation 45.60 ns 50.70 ns 1.11 0.16 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Memory&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Memory<Byte>.Pin(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.69746526472119 > 48.95158065907003.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -4.918309088206156 (T) = (0 -51.3979878436722) / Math.Sqrt((1.6229094814362268 / (12)) + (14.448981406028752 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.0930874494264039 = (47.020929451383985 - 51.3979878436722) / 47.020929451383985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTimeOffset

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetNow - Duration of single invocation 160.76 ns 181.00 ns 1.13 0.01 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTimeOffset*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTimeOffset.GetNow

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 180.99547278933065 > 169.30195907541525.
IsChangePoint: Marked as a change because one of 3/11/2023 9:11:51 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -94.59540235694034 (T) = (0 -180.00504468139573) / Math.Sqrt((0.22411953523435327 / (12)) + (0.4739761443257918 / (22))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (22) - 2, .025) and -0.11780927493709617 = (161.03377268141327 - 180.00504468139573) / 161.03377268141327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 549077434782bd435574cebcf9ab37d6c9d611c2
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Char_ToLowerInvariant - Duration of single invocation 17.42 ns 19.35 ns 1.11 0.12 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Char.Char_ToLowerInvariant(input: "Hello World!")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.347005641998486 > 18.19578016878353.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -8.616395261880601 (T) = (0 -19.163098721889945) / Math.Sqrt((0.18139852344401675 / (11)) + (0.5183114779588747 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.09748450926958764 = (17.460928660071588 - 19.163098721889945) / 17.460928660071588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 4, 2023 •
edited
Loading

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 4.34 μs 4.63 μs 1.07 0.16 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.625875382262996 > 4.55479332271792.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -27.686126170476665 (T) = (0 -4632.326289980611) / Math.Sqrt((892.7102055479405 / (12)) + (540.0620441873716 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.06284419407474079 = (4358.424608052062 - 4632.326289980611) / 4358.424608052062 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS Windows 10.0.19041
Queue SurfaceWindows
Baseline 195bf987887ea645a747ff0586440f7920c47b01
Compare 63a7a13190448f1ba5b3b449f1b9628d042d209d
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TrimStart - Duration of single invocation 2.32 ns 3.54 ns 1.53 0.58 False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.TrimStart(s: "Test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.537447245732433 > 2.6485727699821466.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/25/2023 6:46:56 PM, 5/4/2023 2:44:24 AM falls between 4/25/2023 1:02:54 PM and 5/4/2023 2:44:24 AM.
IsRegressionStdDev: Marked as regression because -17.989597211946112 (T) = (0 -3.4669886332537003) / Math.Sqrt((0.025893583527534937 / (12)) + (0.018168090547608103 / (23))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (23) - 2, .025) and -0.39220735801113743 = (2.490281791217171 - 3.4669886332537003) / 2.490281791217171 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo closed this as completed May 4, 2023
@EgorBo EgorBo removed the untriaged label May 4, 2023
@EgorBo EgorBo reopened this May 4, 2023
@EgorBo
Copy link
Member

EgorBo commented May 4, 2023

Could be dotnet/runtime#85265, but most likely dotnet/runtime#85275 (it updates managed PGO version)

@EgorBo EgorBo closed this as completed May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tannergooding tannergooding

Labels
arch-arm64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @tannergooding