Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Windows/x64: 67 Improvements on 4/25/2023 9:59:52 PM #17240

Closed
performanceautofiler bot opened this issue May 2, 2023 · 7 comments
Closed

[Perf] Windows/x64: 67 Improvements on 4/25/2023 9:59:52 PM #17240

performanceautofiler bot opened this issue May 2, 2023 · 7 comments
Assignees
@tannergooding
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 171.57 ns 157.65 ns 0.92 0.04 False 1392.205157925239 1340.295467031987 0.9627140507289806) Trace Trace
ToStringWithFormat - Duration of single invocation 229.17 ns 198.48 ns 0.87 0.01 False 1826.8876886196574 1721.0768226409766 0.9420813514493448) Trace Trace
ToStringWithFormat - Duration of single invocation 190.76 ns 175.73 ns 0.92 0.02 False 1782.6302740573574 1729.0948901048685 0.9699683188759946) Trace Trace
ToString - Duration of single invocation 158.26 ns 146.57 ns 0.93 0.03 False Trace Trace
ToStringWithCultureInfo - Duration of single invocation 150.19 ns 140.36 ns 0.93 0.00 False 1151.083880260902 1122.0988640995765 0.9748193709786328) Trace Trace
ToStringWithFormat - Duration of single invocation 201.32 ns 186.10 ns 0.92 0.01 False Trace Trace

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 157.65063027618126 < 162.5504298889227.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 6.47988110526923 (T) = (0 -159.2942002451505) / Math.Sqrt((7.509362883716798 / (299)) + (131.22947500349196 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08249515290594363 = (173.61673973676648 - 159.2942002451505) / 173.61673973676648 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 198.4755990829964 < 218.92631865581163.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 8:51:41 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 21.62262921775599 (T) = (0 -197.66655000953887) / Math.Sqrt((71.48203218713009 / (299)) + (1.510380959181054 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.05608257778856458 = (209.41085031192696 - 197.66655000953887) / 209.41085031192696 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 175.73385044573823 < 182.28969691686294.
IsChangePoint: Marked as a change because one of 2/21/2023 5:46:41 AM, 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 52.80767066881396 (T) = (0 -177.21521535569994) / Math.Sqrt((16.774291480251527 / (299)) + (1.609707887711128 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09203818127132 = (195.17914927726267 - 177.21521535569994) / 195.17914927726267 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Single.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 146.5686300820553 < 148.79502451876598.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 34.65211474651841 (T) = (0 -146.7161053817818) / Math.Sqrt((7.865754507138263 / (299)) + (1.0721481434255364 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.057212456586949456 = (155.61947801160446 - 146.7161053817818) / 155.61947801160446 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 140.35705990098356 < 143.29583176546072.
IsChangePoint: Marked as a change because one of 4/17/2023 1:30:24 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 76.70926661633406 (T) = (0 -140.12061689444283) / Math.Sqrt((2.275586289399877 / (299)) + (0.22844456373423883 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06489835831826742 = (149.84533300833806 - 140.12061689444283) / 149.84533300833806 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 186.104632076472 < 192.89049591352736.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 50.892514761368574 (T) = (0 -186.16886421607762) / Math.Sqrt((5.538392038247715 / (299)) + (2.8675885943081756 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08804520256624876 = (204.14264472313587 - 186.16886421607762) / 204.14264472313587 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation 527.40 ns 444.67 ns 0.84 0.00 True 6516.395804518211 5824.271148329069 0.8937871981764444) Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 526.59 ns 442.44 ns 0.84 0.00 True Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 648.15 ns 609.92 ns 0.94 0.00 True Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 659.39 ns 605.58 ns 0.92 0.01 False Trace Trace
IsSuffix_DifferentLastChar - Duration of single invocation 16.83 ns 13.12 ns 0.78 0.15 False 194.5287795663966 154.44757208154414 0.793957441288671) Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 648.33 ns 610.29 ns 0.94 0.01 True 9351.859717679707 8859.78530578666 0.9473821863513651) Trace Trace
IsSuffix_DifferentLastChar - Duration of single invocation 17.60 ns 13.13 ns 0.75 0.18 False 194.53580616223127 155.55465196713376 0.7996196434779234) Trace Trace
IsSuffix_DifferentLastChar - Duration of single invocation 16.75 ns 12.45 ns 0.74 0.22 False 183.47368032969837 144.53150936532384 0.7877506414304425) Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 40.12 ns 31.17 ns 0.78 0.30 False Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 527.97 ns 443.89 ns 0.84 0.00 True 6480.639181627819 5835.718571414521 0.9004850305442702) Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 525.83 ns 444.69 ns 0.85 0.00 True 6524.851385159173 5825.716115510014 0.8928503917745394) Trace Trace
IsSuffix_DifferentLastChar - Duration of single invocation 16.46 ns 12.43 ns 0.76 0.28 False 184.9174299779733 141.58938522653366 0.7656897743138614) Trace Trace
IsSuffix_DifferentLastChar - Duration of single invocation 17.05 ns 13.23 ns 0.78 0.21 False 184.4044413264403 143.56778579673883 0.7785484165350945) Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 656.54 ns 605.54 ns 0.92 0.00 False 9301.66312809625 8849.187459505203 0.9513554014631732) Trace Trace
LastIndexOf_Word_NotFound - Duration of single invocation 525.93 ns 444.71 ns 0.85 0.00 True Trace Trace
IndexOf_Word_NotFound - Duration of single invocation 526.53 ns 441.53 ns 0.84 0.00 True Trace Trace

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 444.6721663659019 < 499.7377415433641.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 236.52279321878495 (T) = (0 -445.2842805086523) / Math.Sqrt((29.878290066783652 / (299)) + (0.17598780545794798 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14770324550706423 = (522.4521601910465 - 445.2842805086523) / 522.4521601910465 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 442.43731605430554 < 501.20240969961753.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 94.5093780759672 (T) = (0 -449.7570100696118) / Math.Sqrt((27.390426874945835 / (299)) + (13.672152108228405 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13978160682946117 = (522.8404945073607 - 449.7570100696118) / 522.8404945073607 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 609.9164173313931 < 615.8360219596133.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 12:29:35 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 111.34890987252318 (T) = (0 -609.8788671280232) / Math.Sqrt((16.38331347656163 / (299)) + (1.3873168183355296 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05485095125380844 = (645.2726878761308 - 609.8788671280232) / 645.2726878761308 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 605.5779504495738 < 626.4352299701899.
IsChangePoint: Marked as a change because one of 2/13/2023 3:18:56 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 94.6534882023038 (T) = (0 -606.5197121820396) / Math.Sqrt((49.98797150797108 / (299)) + (0.48127286506286554 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.06270693867845781 = (647.0971963954084 - 606.5197121820396) / 647.0971963954084 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.123461437595227 < 16.21750491587734.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 7.401343564915417 (T) = (0 -13.15071628224483) / Math.Sqrt((2.539071807953424 / (299)) + (0.05703918969374994 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.054781899484912536 = (13.91289087151259 - 13.15071628224483) / 13.91289087151259 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 610.2906658320945 < 616.3493169034496.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 112.67629732801524 (T) = (0 -609.2236276904189) / Math.Sqrt((21.632261464348595 / (299)) + (0.9893580722717392 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.05662530240918929 = (645.7917826779259 - 609.2236276904189) / 645.7917826779259 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.132420907940054 < 16.194389426616304.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 6.024181261005628 (T) = (0 -13.220193446873392) / Math.Sqrt((2.574582259446533 / (299)) + (0.13423570830135084 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.05042839805382293 = (13.92227128504916 - 13.220193446873392) / 13.92227128504916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.445720247075972 < 15.616563570621944.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 7.160514266041495 (T) = (0 -12.540568546439863) / Math.Sqrt((2.449111943889903 / (299)) + (0.19291681720562656 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06604075870446886 = (13.427318872121607 - 12.540568546439863) / 13.427318872121607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 31.1713011942908 < 37.32247025981476.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 7.494056818419354 (T) = (0 -34.539976986506254) / Math.Sqrt((7.506660279788329 / (299)) + (1.8496296390745341 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06225072245048804 = (36.83284840993395 - 34.539976986506254) / 36.83284840993395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 443.88543480950614 < 500.4626678213687.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 106.66027796105647 (T) = (0 -449.7601770295407) / Math.Sqrt((30.303298774108043 / (299)) + (9.87353534922424 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13946496748388662 = (522.6517922396367 - 449.7601770295407) / 522.6517922396367 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 444.69389203867553 < 498.9664915263663.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 231.46474565826983 (T) = (0 -445.3636988700934) / Math.Sqrt((30.00675465042796 / (299)) + (0.28606861757240337 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14756959225621008 = (522.463411469425 - 445.3636988700934) / 522.463411469425 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.434419126156437 < 15.627479098816712.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 6.005354713065248 (T) = (0 -12.656092820864554) / Math.Sqrt((2.4605653374530534 / (299)) + (0.27485129750943327 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06048631702288028 = (13.47089781679398 - 12.656092820864554) / 13.47089781679398 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.227134163212678 < 15.644432379214027.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 5.995683355910002 (T) = (0 -12.632046509863208) / Math.Sqrt((2.694607114137194 / (299)) + (0.26565256343141597 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.061180737969402973 = (13.455248545434635 - 12.632046509863208) / 13.455248545434635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 605.5387309063208 < 625.3389582206007.
IsChangePoint: Marked as a change because one of 2/13/2023 6:54:14 PM, 4/17/2023 6:40:18 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 96.61288766330681 (T) = (0 -606.828655329797) / Math.Sqrt((40.68073462729566 / (299)) + (1.0979140901773858 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.062049206769336576 = (646.9728046603017 - 606.828655329797) / 646.9728046603017 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, None, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 444.70687177243605 < 499.32845541816334.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 220.5153060560246 (T) = (0 -445.5202929460497) / Math.Sqrt((30.092579896918977 / (299)) + (0.562890590450869 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14713765048920643 = (522.3824139986981 - 445.5202929460497) / 522.3824139986981 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreNonSpace, False))

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 441.5294419788778 < 500.27495972291484.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 103.54553356471006 (T) = (0 -450.03314794431765) / Math.Sqrt((28.830154869753002 / (299)) + (10.704458429647486 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13906815995888658 = (522.7279640660363 - 450.03314794431765) / 522.7279640660363 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Reflection.Invoke

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
StaticMethod5_ByRefParams_int_string_struct_class_bool - Duration of single invocation 349.15 ns 316.75 ns 0.91 0.13 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Invoke*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Invoke.StaticMethod5_ByRefParams_int_string_struct_class_bool

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 316.75277879008746 < 328.159759755291.
IsChangePoint: Marked as a change because one of 2/15/2023 2:23:18 PM, 4/10/2023 1:33:42 PM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 10.485540186864574 (T) = (0 -320.94658184832684) / Math.Sqrt((214.03153880323077 / (299)) + (114.89775641808316 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06789790782009775 = (344.3255674898559 - 320.94658184832684) / 344.3255674898559 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 83.43 μs 75.43 μs 0.90 0.01 True 682165.7509157509 626256.2814070352 0.9180412246823271) Trace Trace
ReadLine - Duration of single invocation 143.58 μs 116.89 μs 0.81 0.01 True 1514112.9032258065 1323057.6441102757 0.8738170325948025) Trace Trace
ReadLine - Duration of single invocation 25.38 μs 23.22 μs 0.91 0.00 True 203657.48709122202 188802.0833333333 0.927056923022748) Trace Trace
ReadLine - Duration of single invocation 9.41 μs 8.81 μs 0.94 0.02 False Trace Trace
ReadLine - Duration of single invocation 124.30 μs 109.64 μs 0.88 0.02 True 1342183.1955922865 1229660.4938271604 0.9161644236534556) Trace Trace

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 8])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 75.43014259708738 < 78.64965347730974.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 85.65330879570544 (T) = (0 -75510.77681930321) / Math.Sqrt((536020.624669357 / (299)) + (114574.85855481001 / (26))) is greater than 1.967335607330539 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (26) - 2, .975) and 0.08198962533122589 = (82254.81857604075 - 75510.77681930321) / 82254.81857604075 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 0])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 116.88911913646055 < 136.32808703309306.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 129.9683259458448 (T) = (0 -114539.22009311391) / Math.Sqrt((823797.413723968 / (299)) + (1272107.8055113754 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20216920326394167 = (143563.29758351788 - 114539.22009311391) / 143563.29758351788 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.218259587020654 < 24.132638244987405.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 99.682168767728 (T) = (0 -23059.654228080468) / Math.Sqrt((39050.794217879695 / (299)) + (13520.02313814996 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09797536544864444 = (25564.328672187276 - 23059.654228080468) / 25564.328672187276 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 33, 128])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.805302570943487 < 8.966738596085118.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 39.242380058680666 (T) = (0 -8824.690366607552) / Math.Sqrt((11632.612391770299 / (299)) + (6082.598492927856 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06740675582793323 = (9462.528730241762 - 8824.690366607552) / 9462.528730241762 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 109.63509963768117 < 118.06230117984691.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 40.34968198342307 (T) = (0 -110014.71254745101) / Math.Sqrt((1740716.0562996964 / (299)) + (3376918.582392723 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11714808069165487 = (124612.87124305071 - 110014.71254745101) / 124612.87124305071 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Xml.Linq.Perf_XDocument

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetRootElement - Duration of single invocation 9.27 ns 7.31 ns 0.79 0.04 False Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XDocument*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XDocument.GetRootElement

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.312709955614976 < 8.802076008356547.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 13.933565901326393 (T) = (0 -8.15143822674655) / Math.Sqrt((0.07817263184730608 / (299)) + (0.12986149117489582 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10851605768483456 = (9.143673643271052 - 8.15143822674655) / 9.143673643271052 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySequence

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Slice_StartPosition - Duration of single invocation 11.91 ns 10.68 ns 0.90 0.32 False 133.75413766038764 129.39444238890184 0.967405155849792) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySequence*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySequence.Slice_StartPosition(Segment: Multiple)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.67889821233053 < 12.763598705627109.
IsChangePoint: Marked as a change because one of 3/21/2023 2:16:57 AM, 3/25/2023 12:18:49 AM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 16.945775817209302 (T) = (0 -10.816212289389526) / Math.Sqrt((0.4118149680476661 / (299)) + (0.2714636682606284 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14347556801570374 = (12.628025407672006 - 10.816212289389526) / 12.628025407672006 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToList - Duration of single invocation 763.05 ns 710.50 ns 0.93 0.01 False Trace Trace
AnyWithPredicate_LastElementMatches - Duration of single invocation 677.71 ns 620.80 ns 0.92 0.02 False Trace Trace
SelectToList - Duration of single invocation 316.75 ns 283.68 ns 0.90 0.01 False Trace Trace
SelectToList - Duration of single invocation 570.72 ns 512.51 ns 0.90 0.05 False 4307.957929417727 3494.7939856834605 0.811241438041579) Trace Trace
SelectToList - Duration of single invocation 364.64 ns 322.30 ns 0.88 0.01 False 2625.8091236699634 1981.5066812412163 0.7546270836593646) Trace Trace
SelectToList - Duration of single invocation 353.93 ns 322.97 ns 0.91 0.00 True 3204.596789696271 1978.9621524637296 0.617538580462503) Trace Trace
Skip_One - Duration of single invocation 1.14 μs 1.03 μs 0.90 0.07 False Trace Trace
Sum - Duration of single invocation 554.89 ns 468.70 ns 0.84 0.03 False Trace Trace
FirstWithPredicate_LastElementMatches - Duration of single invocation 3.00 μs 2.64 μs 0.88 0.00 False Trace Trace
TakeLastHalf - Duration of single invocation 803.16 ns 757.59 ns 0.94 0.05 False Trace Trace

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.ToList(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 710.4964794064203 < 725.2865369290912.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/25/2023 12:18:49 AM, 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 24.827945409361444 (T) = (0 -690.7719867983641) / Math.Sqrt((232.40319760229676 / (299)) + (82.39988584844868 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06571078465397509 = (739.3556250593438 - 690.7719867983641) / 739.3556250593438 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.AnyWithPredicate_LastElementMatches(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 620.8039534177121 < 643.6511068395135.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 4/9/2023 11:04:55 PM, 4/12/2023 8:39:35 PM, 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 50.29559203884403 (T) = (0 -620.7432359316115) / Math.Sqrt((622.695514545029 / (299)) + (0.00737322623675314 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.10477041932888101 = (693.3899966378058 - 620.7432359316115) / 693.3899966378058 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Range)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 283.6758246565466 < 300.37268732225266.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 175.43351999721088 (T) = (0 -283.5432917348383) / Math.Sqrt((5.287951567419166 / (299)) + (0.03512995067306369 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.10407514141737088 = (316.4811077833128 - 283.5432917348383) / 316.4811077833128 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: IList)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 512.5103841238284 < 541.9993710883766.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 11.000661880482443 (T) = (0 -538.9913803987339) / Math.Sqrt((4004.6961978613836 / (299)) + (145.41931288024313 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08125917492613594 = (586.6631433901945 - 538.9913803987339) / 586.6631433901945 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: Array)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.2952386199741 < 346.3817681109164.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 100.46004552178496 (T) = (0 -320.9401134393726) / Math.Sqrt((66.25298692793051 / (299)) + (0.9936894488781861 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.13727067080411162 = (372.00556719047285 - 320.9401134393726) / 372.00556719047285 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.SelectToList(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 322.97159189621505 < 333.2901793635508.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 56.73418713953598 (T) = (0 -322.0118157160822) / Math.Sqrt((5.486995977088016 / (299)) + (7.130546562291737 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08561852795097151 = (352.16353957226306 - 322.0118157160822) / 352.16353957226306 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.Skip_One(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.0271953379689827 < 1.0800194925673465.
IsChangePoint: Marked as a change because one of 2/20/2023 10:31:48 AM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 54.06333040861707 (T) = (0 -1027.3959736055076) / Math.Sqrt((784.8852428113727 / (299)) + (0.08050931728750965 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.07911225710534389 = (1115.6582129935412 - 1027.3959736055076) / 1115.6582129935412 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.Sum(input: IEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 468.70034335681225 < 527.439091414396.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/14/2023 8:42:09 PM, 4/25/2023 3:46:39 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 14.752581568341489 (T) = (0 -505.26666697771884) / Math.Sqrt((220.98553923779644 / (299)) + (221.12823792090146 / (30))) is greater than 1.9672451058622664 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (30) - 2, .975) and 0.07676621319698208 = (547.2792202800124 - 505.26666697771884) / 547.2792202800124 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.FirstWithPredicate_LastElementMatches(input: IOrderedEnumerable)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.635555672563886 < 2.8519239183697396.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 57.16438436301058 (T) = (0 -2670.6088015837827) / Math.Sqrt((1337.620507975826 / (299)) + (527.9904418913349 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.0949617423600575 = (2950.824209959806 - 2670.6088015837827) / 2950.824209959806 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Linq.Tests.Perf_Enumerable.TakeLastHalf(input: List)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 757.5910437685184 < 764.5508665175396.
IsChangePoint: Marked as a change because one of 2/25/2023 3:45:56 AM, 2/26/2023 1:18:12 AM, 4/25/2023 6:46:56 PM, 5/1/2023 3:42:23 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 43.40487485085789 (T) = (0 -757.9910906417826) / Math.Sqrt((213.5016840595692 / (299)) + (0.3200750016169517 / (2))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (2) - 2, .975) and 0.050816267075319985 = (798.5715139746617 - 757.9910906417826) / 798.5715139746617 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.32 μs 2.92 μs 0.88 0.07 True 35675.42744100608 36645.4986925663 1.0271915803437635) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.918966367727801 < 3.1464939885769727.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 24.69220486013339 (T) = (0 -2943.4285810399588) / Math.Sqrt((21562.35734314431 / (299)) + (5253.971729882868 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1204931497301325 = (3346.6806769461755 - 2943.4285810399588) / 3346.6806769461755 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 4.74 μs 4.13 μs 0.87 0.00 True Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.130614046675473 < 4.51784905761268.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 133.6747577454917 (T) = (0 -4086.8035207985495) / Math.Sqrt((3076.7151004552397 / (299)) + (505.60505022675886 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.14979843944333968 = (4806.864290066415 - 4086.8035207985495) / 4806.864290066415 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyThreeValues - Duration of single invocation 16.58 ns 12.45 ns 0.75 0.19 False 143.6455092167867 147.63272811762477 1.0277573515703902) Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.44937866241213 < 13.81657411203996.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 35.557205117720976 (T) = (0 -12.644115535396425) / Math.Sqrt((0.3255941427772862 / (299)) + (0.04679536437416961 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.12997439384595982 = (14.533038391007716 - 12.644115535396425) / 14.533038391007716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.RegexRedux_1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 48.60 ms 44.93 ms 0.92 0.00 False 295666666.6666667 276700000 0.9358511837655016) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.RegexRedux_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.RegexRedux_1.RunBench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 44.928179166666666 < 46.074378063186806.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 22.47712100418857 (T) = (0 -45539745.54639806) / Math.Sqrt((203805481229.8447 / (299)) + (374428295623.5147 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.056189663945838644 = (48250950.22458487 - 45539745.54639806) / 48250950.22458487 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseHex - Duration of single invocation 6.86 ns 5.61 ns 0.82 0.04 True 98.50244350897565 86.93462780633874 0.8825631599526479) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "0")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.609814658927328 < 6.527864675361195.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 146.24382610421125 (T) = (0 -5.514500536783879) / Math.Sqrt((0.00882724429660948 / (299)) + (0.0016586702965458106 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.20186480471642865 = (6.909231129476277 - 5.514500536783879) / 6.909231129476277 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline f74138f056a6504b9df7acecdc0e0f52d9585b5c
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateGetPositionTenSegments - Duration of single invocation 99.33 ns 83.84 ns 0.84 0.13 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionTenSegments

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 83.84472675745211 < 94.6477186266568.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 25.95167941182113 (T) = (0 -83.55743300082419) / Math.Sqrt((11.711580292601546 / (299)) + (4.137127272776995 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11989789808236274 = (94.94061293429766 - 83.55743300082419) / 94.94061293429766 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19d545c62c3eea1836e8c3830eb654e2a022cac7
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 25.06 ns 21.12 ns 0.84 0.03 False 276.89946738917155 239.30434523148432 0.8642282612091531) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "ascii")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 21.116318851702154 < 24.22415023417327.
IsChangePoint: Marked as a change because one of 2/21/2023 12:38:54 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 52.15185895996728 (T) = (0 -21.125555451244313) / Math.Sqrt((0.2751991819351811 / (299)) + (0.13950922755577255 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.16150260922805046 = (25.19453928389139 - 21.125555451244313) / 25.19453928389139 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 8.37 μs 6.88 μs 0.82 0.01 True Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.876284171075837 < 7.905388032089662.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 62.867614369592836 (T) = (0 -6949.14564957866) / Math.Sqrt((17370.345280145084 / (299)) + (11468.794963567612 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.1658299645751901 = (8330.610492427642 - 6949.14564957866) / 8330.610492427642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IterateTryGetTenSegments - Duration of single invocation 50.29 ns 47.32 ns 0.94 0.03 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateTryGetTenSegments

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 47.318559033078536 < 47.76365403419051.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 33.779574357408485 (T) = (0 -47.917479056839205) / Math.Sqrt((2.9377406287643013 / (299)) + (0.26196045606810486 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08967653356682842 = (52.6378598638014 - 47.917479056839205) / 52.6378598638014 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19d545c62c3eea1836e8c3830eb654e2a022cac7
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 6.26 ns 4.84 ns 0.77 0.03 False 77.4171217681147 81.545679612906 1.0533287437003593) Trace Trace
TrimStart - Duration of single invocation 14.42 ns 11.94 ns 0.83 0.03 False 151.88504163968958 130.35118290535075 0.85822264982866)
TrimEnd - Duration of single invocation 14.61 ns 11.18 ns 0.77 0.03 True Trace Trace
Replace_Char - Duration of single invocation 5.92 ns 4.48 ns 0.76 0.11 False 67.42408344752842 70.54331179458971 1.0462628216442686) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "Hello", oldChar: 'a', newChar: 'b')

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.840118421322313 < 5.986002425440193.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 53.3931961169463 (T) = (0 -4.570203107766524) / Math.Sqrt((0.061080467709849054 / (299)) + (0.0144947452722482 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.24130374631157395 = (6.023758632718081 - 4.570203107766524) / 6.023758632718081 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_String.TrimStart(s: " Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.944175985892064 < 13.736143564231433.
IsChangePoint: Marked as a change because one of 2/24/2023 9:01:57 AM, 3/6/2023 11:14:52 AM, 3/20/2023 8:31:51 AM, 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 13.395667510277583 (T) = (0 -12.905951639710953) / Math.Sqrt((0.22063869656284524 / (299)) + (0.19042390014751417 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08392519616635073 = (14.08831635331666 - 12.905951639710953) / 14.08831635331666 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_String.TrimEnd(s: "Test ")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.184448143103975 < 13.861579638972497.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 36.17850066657133 (T) = (0 -12.016101697580112) / Math.Sqrt((0.12546529131524678 / (299)) + (0.14807482223039337 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18787823674074347 = (14.795936079038642 - 12.016101697580112) / 14.795936079038642 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_String.Replace_Char(text: "This is a very nice sentence", oldChar: 'z', newChar: 'y')

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.4790616996263175 < 5.602605983220419.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/19/2023 8:38:27 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 34.56531226960957 (T) = (0 -4.624840681009782) / Math.Sqrt((0.15730889537060638 / (299)) + (0.003920082853823819 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.16222914652726508 = (5.520412487303482 - 4.624840681009782) / 5.520412487303482 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Diagnostics.Perf_Process

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetCurrentProcess - Duration of single invocation 113.34 ns 104.04 ns 0.92 0.02 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Perf_Process*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Perf_Process.GetCurrentProcess

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 104.03519921292978 < 107.44775094833368.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 32.38204445734904 (T) = (0 -104.0935547204948) / Math.Sqrt((11.878394837617385 / (299)) + (0.7710842490574905 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07518005688014612 = (112.55548228051629 - 104.0935547204948) / 112.55548228051629 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBytes - Duration of single invocation 137.44 μs 123.62 μs 0.90 0.00 False 796386.0544217686 775300.3003003002 0.9735232002062391) Trace Trace
GetBytes - Duration of single invocation 217.04 μs 197.12 μs 0.91 0.01 False 1113717.9487179485 1083170.9956709954 0.9725720923487701) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 123.62339381720432 < 129.38461445932543.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 44.56651241452343 (T) = (0 -123446.6853215374) / Math.Sqrt((3113456.469351134 / (299)) + (2370112.776001592 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.10163144903155712 = (137412.0734619012 - 123446.6853215374) / 137412.0734619012 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Text.Perf_Utf8Encoding.GetBytes(Input: Greek)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 197.12248931623932 < 202.37001147349275.
IsChangePoint: Marked as a change because one of 2/16/2023 9:36:08 AM, 3/17/2023 8:17:18 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 29.364721234525323 (T) = (0 -198907.806034712) / Math.Sqrt((13090826.594191918 / (299)) + (8131906.73537832 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07978987740513983 = (216154.76851506546 - 198907.806034712) / 216154.76851506546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_GC<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AllocateUninitializedArray - Duration of single invocation 137.10 ns 123.23 ns 0.90 0.02 False Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Byte>.AllocateUninitializedArray(length: 10000, pinned: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 123.22810783687325 < 128.47563584621548.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 29.933856317855806 (T) = (0 -124.2034612320119) / Math.Sqrt((7.580186120873745 / (299)) + (2.6489640691920675 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07807143234787259 = (134.7213499938726 - 124.2034612320119) / 134.7213499938726 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Append_Strings - Duration of single invocation 195.09 ns 184.22 ns 0.94 0.01 False Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat: 1)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 184.22021218497875 < 187.87603133373335.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 26.732660165508314 (T) = (0 -183.8018763899572) / Math.Sqrt((5.655447180853203 / (299)) + (3.6440914392767896 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.053974358293551204 = (194.28847198942077 - 183.8018763899572) / 194.28847198942077 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 19d545c62c3eea1836e8c3830eb654e2a022cac7
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetComponents - Duration of single invocation 30.54 ns 24.92 ns 0.82 0.01 True Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Uri.GetComponents

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 24.920848746422884 < 28.831959685382376.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 63.02730154713715 (T) = (0 -24.813233678643176) / Math.Sqrt((0.6215765305594008 / (299)) + (0.15756835918131962 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18432359336255397 = (30.420438150140374 - 24.813233678643176) / 30.420438150140374 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 4.06 μs 3.26 μs 0.80 0.21 False 42231.91214470284 41062.69453090263 0.9723143576877594) Trace Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.2624466415568114 < 3.833238200880781.
IsChangePoint: Marked as a change because one of 2/24/2023 8:06:06 PM, 3/6/2023 11:14:52 AM, 3/14/2023 8:42:09 PM, 3/25/2023 12:18:49 AM, 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 29.933742412952036 (T) = (0 -3228.7785839877297) / Math.Sqrt((147953.97020944676 / (299)) + (2393.84560959372 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.18296920193830452 = (3951.844400049065 - 3228.7785839877297) / 3951.844400049065 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IDictionary - Duration of single invocation 5.14 μs 4.41 μs 0.86 0.48 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.IDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.407183644155599 < 4.92420362133935.
IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 4/1/2023 1:54:32 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 18.36452723825154 (T) = (0 -4612.175047740813) / Math.Sqrt((266403.3896073658 / (299)) + (81770.94282327846 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.19954032837468091 = (5761.908077612297 - 4612.175047740813) / 5761.908077612297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse4 - Duration of single invocation 60.01 ns 53.21 ns 0.89 0.02 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Version.Parse4

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.21421739065831 < 57.0168368267946.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 52.710866736561286 (T) = (0 -53.19377288070805) / Math.Sqrt((1.2341857895136674 / (299)) + (0.05433173388124922 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.07205216646608208 = (57.324098358125795 - 53.19377288070805) / 57.324098358125795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_GC<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
AllocateUninitializedArray - Duration of single invocation 191.46 ns 179.90 ns 0.94 0.01 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_GC&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_GC<Char>.AllocateUninitializedArray(length: 10000, pinned: False)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 179.89704341158748 < 180.49932285597188.
IsChangePoint: Marked as a change because one of 2/28/2023 10:21:25 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 25.461528472078015 (T) = (0 -177.3491702217989) / Math.Sqrt((8.656922230116079 / (299)) + (5.561483465461223 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.06505968221522172 = (189.69036509411114 - 177.3491702217989) / 189.69036509411114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Min - Duration of single invocation 7.04 μs 6.24 μs 0.89 0.02 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Min

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.243775313082866 < 6.578731757227571.
IsChangePoint: Marked as a change because one of 3/6/2023 11:14:52 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 81.93192647620559 (T) = (0 -6244.938304498514) / Math.Sqrt((20224.601045795338 / (299)) + (34.43482922489323 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.09821719602383978 = (6925.102449240768 - 6244.938304498514) / 6925.102449240768 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented May 2, 2023 •
edited
Loading

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.CreateAddAndRemove<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 12.29 μs 11.08 μs 0.90 0.04 False 124910.80530071356 121106.0948081264 0.9695405815099215) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.079691138503595 < 11.627554652185328.
IsChangePoint: Marked as a change because one of 2/25/2023 2:33:42 AM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 89.25039245471898 (T) = (0 -11110.090965106234) / Math.Sqrt((46217.01303560189 / (299)) + (2756.0678414631698 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.11402111593409028 = (12539.904917507867 - 11110.090965106234) / 12539.904917507867 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Next_int_int - Duration of single invocation 7.55 ns 6.29 ns 0.83 0.22 False 82.47009688148286 72.27166589023884 0.876337831809509) Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.Next_int_int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 6.287735903105484 < 7.15171102915089.
IsChangePoint: Marked as a change because one of 2/17/2023 9:53:52 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 20.712259385465448 (T) = (0 -6.370405909893968) / Math.Sqrt((0.21310761117625698 / (299)) + (0.09089334295482418 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.17194994571209915 = (7.693261870953361 - 6.370405909893968) / 7.693261870953361 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS Windows 10.0.18362
Queue TigerWindows
Baseline 2e0033cf25cee5457ee326dc1eaff013b939d00d
Compare 9a7db5556f4ad892c611b2e174594db17fdec3af
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
HashSet - Duration of single invocation 13.47 μs 10.89 μs 0.81 0.02 False Trace Trace

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.HashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.891076817558297 < 12.622895712010653.
IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 4/25/2023 6:46:56 PM, 5/2/2023 10:35:24 AM falls between 4/23/2023 8:38:14 PM and 5/2/2023 10:35:24 AM.
IsImprovementStdDev: Marked as improvement because 16.58147349196688 (T) = (0 -10847.623284639923) / Math.Sqrt((955026.8110148754 / (299)) + (21022.06820385985 / (27))) is greater than 1.9673127716992511 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (27) - 2, .975) and 0.08787841521029131 = (11892.738276927043 - 10847.623284639923) / 11892.738276927043 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@cincuranet cincuranet mentioned this issue May 4, 2023
Closed
@kunalspathak
Copy link
Member

dotnet/runtime#85437

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tannergooding tannergooding

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tannergooding @kunalspathak