[Perf] Linux/x64: 34 Regressions on 3/11/2023 1:35:17 AM #14021

performanceautofiler · 2023-03-14T17:37:57Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in System.MathBenchmarks.Double

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Exp2M1 - Duration of single invocation	24.85 μs	45.96 μs	1.85	0.18	False
Cosh - Duration of single invocation	82.00 μs	87.86 μs	1.07	0.01	False
Exp10 - Duration of single invocation	96.20 μs	123.53 μs	1.28	0.03	False
Pow - Duration of single invocation	98.00 μs	122.13 μs	1.25	0.01	False
Log2 - Duration of single invocation	36.96 μs	39.63 μs	1.07	0.03	False
Exp - Duration of single invocation	28.56 μs	46.47 μs	1.63	0.22	False
LogP1 - Duration of single invocation	33.36 μs	38.26 μs	1.15	0.04	False
Exp2 - Duration of single invocation	24.86 μs	42.74 μs	1.72	0.16	False
ExpM1 - Duration of single invocation	29.08 μs	47.51 μs	1.63	0.17	False
Log2P1 - Duration of single invocation	36.09 μs	40.94 μs	1.13	0.06	False
Log - Duration of single invocation	31.99 μs	35.90 μs	1.12	0.09	False
Exp10M1 - Duration of single invocation	97.00 μs	120.93 μs	1.25	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.Exp2M1

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.95970168621702 > 26.123284603781023.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -63.11589734440881 (T) = (0 -43858.50649266004) / Math.Sqrt((711454.439987181 / (48)) + (875096.3199781721 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.743026227367788 = (25162.275704188618 - 43858.50649266004) / 25162.275704188618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Cosh

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 87.85847292830539 > 84.80002840525873.
IsChangePoint: Marked as a change because one of 1/12/2023 10:41:19 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -53.63878153323664 (T) = (0 -87595.40433975554) / Math.Sqrt((526701.0381117769 / (48)) + (50031.46636901216 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.0814921614033472 = (80994.95074110523 - 87595.40433975554) / 80994.95074110523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Exp10

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.53353051181104 > 101.0173718399009.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -63.33032654392621 (T) = (0 -120912.84995721694) / Math.Sqrt((257381.06184266182 / (48)) + (1739177.653152954 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.2547886377408439 = (96361.12913399644 - 120912.84995721694) / 96361.12913399644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Pow

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.1331618962433 > 102.80846522734251.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -71.64715705447072 (T) = (0 -122139.1613371423) / Math.Sqrt((228841.78829200153 / (48)) + (1297869.8805714773 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.2455198283078112 = (98062.79961281955 - 122139.1613371423) / 98062.79961281955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Log2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 39.632478529283226 > 35.35235300690617.
IsChangePoint: Marked as a change because one of 1/20/2023 8:02:54 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -59.37197642974018 (T) = (0 -39511.11097627006) / Math.Sqrt((315413.51330356573 / (48)) + (37006.150799833486 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.17322886548840383 = (33677.2407656557 - 39511.11097627006) / 33677.2407656557 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Exp

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.4723107727466 > 29.94315471781071.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -61.31876041371742 (T) = (0 -46861.898952525924) / Math.Sqrt((442737.47006917285 / (48)) + (935832.482962906 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.6297707657832587 = (28753.675017605532 - 46861.898952525924) / 28753.675017605532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.LogP1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.25830488445378 > 34.97040273214215.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -57.66479404569125 (T) = (0 -38395.74990110128) / Math.Sqrt((198019.0178868507 / (48)) + (38770.7627685268 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.14785817254430564 = (33449.90767979156 - 38395.74990110128) / 33449.90767979156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Exp2

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.738524578779604 > 26.51757154906449.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -49.13357046111636 (T) = (0 -43083.2343577165) / Math.Sqrt((1139372.7862677532 / (48)) + (1302190.7584908665 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.7086328014941066 = (25215.034102144447 - 43083.2343577165) / 25215.034102144447 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.ExpM1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 47.50910497177595 > 30.540009616723303.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -13.67937549656884 (T) = (0 -49378.48432848635) / Math.Sqrt((961087.2874266129 / (48)) + (25395574.68321753 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.6804265696012909 = (29384.493926564264 - 49378.48432848635) / 29384.493926564264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Log2P1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.93837711074561 > 38.01907635737975.
IsChangePoint: Marked as a change because one of 1/20/2023 8:02:54 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -15.574149766870068 (T) = (0 -41565.32978560315) / Math.Sqrt((96677.96286182232 / (48)) + (1343389.0522687936 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.14480630396336355 = (36307.74013184796 - 41565.32978560315) / 36307.74013184796 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Log

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.899039535550465 > 32.29426527535526.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -15.98281269248146 (T) = (0 -36670.77880868079) / Math.Sqrt((1799384.6131269517 / (48)) + (1070895.6885318193 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.18365128672022235 = (30981.06614684786 - 36670.77880868079) / 30981.06614684786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Double.Exp10M1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 120.92857084478021 > 101.59918268600131.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -9.477950803624283 (T) = (0 -124796.78730295331) / Math.Sqrt((984493.3798235268 / (48)) + (102343046.71645065 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.28544689192000783 = (97084.35882290755 - 124796.78730295331) / 97084.35882290755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Benchstone.BenchF.BenchMk2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	174.39 ms	195.04 ms	1.12	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.BenchMk2*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.BenchMk2.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 195.04399980000002 > 183.1422372375.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -28.171430460839897 (T) = (0 -195920410.0055556) / Math.Sqrt((9022993420935.54 / (48)) + (4028288892924.777 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.11613845445108836 = (175534145.6288309 - 195920410.0055556) / 175534145.6288309 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	139.11 ns	170.90 ns	1.23	0.03	False
Span - Duration of single invocation	143.19 ns	171.20 ns	1.20	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.Array(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 170.8955361763138 > 147.03657293919204.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -89.37388755045757 (T) = (0 -170.77050011529204) / Math.Sqrt((1.6436865710192017 / (48)) + (0.9408342671029657 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.21308182647989288 = (140.7740981585982 - 170.77050011529204) / 140.7740981585982 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSet&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 171.19702132887565 > 151.83754108909915.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -68.58365649980283 (T) = (0 -171.98317216275385) / Math.Sqrt((3.5406303441455838 / (48)) + (1.105668456180202 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.19392711120878475 = (144.0483012305755 - 171.98317216275385) / 144.0483012305755 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Benchstone.BenchF.BenchMrk

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	180.27 ms	201.45 ms	1.12	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.BenchMrk*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.BenchMrk.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.4519287857143 > 189.348810245.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -21.217063608545256 (T) = (0 -203136802.118956) / Math.Sqrt((5348170707855.074 / (48)) + (11477731849925.824 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.12099430677583314 = (181211270.11180937 - 203136802.118956) / 181211270.11180937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in System.IO.Tests.Perf_FileStream

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
Read - Duration of single invocation	12.91 ms	13.76 ms	1.07	0.09	False
Read - Duration of single invocation	98.40 μs	105.09 μs	1.07	0.16	False
Read - Duration of single invocation	97.79 μs	109.28 μs	1.12	0.24	False
Read_NoBuffering - Duration of single invocation	47.43 μs	50.79 μs	1.07	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_FileStream*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_FileStream.Read(fileSize: 104857600, userBufferSize: 4096, options: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.764542012605041 > 13.543310273888888.
IsChangePoint: Marked as a change because one of 1/28/2023 4:01:39 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -11.823728512785573 (T) = (0 -13900738.873385139) / Math.Sqrt((173101028665.97275 / (48)) + (21543452846.372284 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06668240654260553 = (13031750.395547481 - 13900738.873385139) / 13031750.395547481 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.Read(fileSize: 1048576, userBufferSize: 4096, options: Asynchronous)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 105.08933001126124 > 102.6811993978974.
IsChangePoint: Marked as a change because one of 1/18/2023 9:06:44 AM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -10.2803717750665 (T) = (0 -108028.37568549208) / Math.Sqrt((12007698.438375138 / (48)) + (8311193.0038128635 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.10180684237098152 = (98046.5645439499 - 108028.37568549208) / 98046.5645439499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.Read(fileSize: 1048576, userBufferSize: 4096, options: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.27886542930527 > 99.46750656821645.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -12.735308131056222 (T) = (0 -107459.67343850556) / Math.Sqrt((6099502.9724331135 / (48)) + (6279982.840951731 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.1056779784450607 = (97188.94247096109 - 107459.67343850556) / 97188.94247096109 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_FileStream.Read_NoBuffering(fileSize: 1048576, userBufferSize: 16384, options: Asynchronous)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 50.7895631514658 > 50.036840080971544.
IsChangePoint: Marked as a change because one of 1/14/2023 10:23:46 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -13.41748675071045 (T) = (0 -51186.9179966741) / Math.Sqrt((269468.9751909951 / (48)) + (710123.3475261348 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.07149211080517075 = (47771.62377631487 - 51186.9179966741) / 47771.62377631487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Benchstone.BenchF.Secant

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	3.26 ms	109.67 ms	33.65	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Secant*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Secant.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.67189285714285 > 3.4225754370624997.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -1296.8840761344397 (T) = (0 -109798858.89761905) / Math.Sqrt((4364014.9536438715 / (48)) + (80981898908.99904 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -32.67910426107534 = (3260147.836666761 - 109798858.89761905) / 3260147.836666761 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-03-14T17:37:58Z

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BenchFourier - Duration of single invocation	433.42 ms	493.03 ms	1.14	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchFourier

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 493.03141675 > 456.1824622525.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -7.3208623861667155 (T) = (0 -497221188.956933) / Math.Sqrt((20103831012684.96 / (48)) + (828514542501483.1 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.13987590858551904 = (436206419.674172 - 497221188.956933) / 436206419.674172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Benchstone.BenchF.FFT

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	217.56 ms	246.30 ms	1.13	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.FFT*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.FFT.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 246.30107164285712 > 225.3597835442308.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -31.6811118384874 (T) = (0 -246591165.476801) / Math.Sqrt((19639484434860.312 / (48)) + (7799560731675.578 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.15236507673810304 = (213987017.18278778 - 246591165.476801) / 213987017.18278778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in System.Collections.CtorDefaultSize<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	840.54 ns	951.00 ns	1.13	0.55	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorDefaultSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorDefaultSize<String>.ConcurrentBag

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 951.001387191919 > 872.6614684229252.
IsChangePoint: Marked as a change because one of 3/1/2023 11:57:24 AM, 3/1/2023 10:00:31 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -12.818037898914525 (T) = (0 -956.8087682697341) / Math.Sqrt((2135.1975744309743 / (48)) + (427.8416877042175 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.13626770766087576 = (842.0628007104273 - 956.8087682697341) / 842.0628007104273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in System.Collections.IndexerSet<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	161.10 ns	182.88 ns	1.14	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 182.8848232589057 > 168.12826075096203.
IsChangePoint: Marked as a change because one of 3/1/2023 3:09:21 PM, 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -74.89919669306792 (T) = (0 -183.18116894435175) / Math.Sqrt((2.9119386836970538 / (48)) + (0.5103728898197698 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.15121111461737355 = (159.12039644026146 - 183.18116894435175) / 159.12039644026146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in SciMark2.kernel

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
benchmarkLU - Duration of single invocation	894.11 ms	943.31 ms	1.06	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchmarkLU

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 943.3126669166667 > 938.6530500375001.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -83.6944558605974 (T) = (0 -941619902.7116758) / Math.Sqrt((2643290696738.306 / (48)) + (3222032135904.552 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.05325230769585567 = (894011715.7413192 - 941619902.7116758) / 894011715.7413192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in PerfLabTests.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CheckObjIsInterfaceYes - Duration of single invocation	130.75 μs	159.39 μs	1.22	0.00	False
CheckIsInstAnyIsInterfaceYes - Duration of single invocation	130.42 μs	159.41 μs	1.22	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.CheckObjIsInterfaceYes

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 159.39281200396826 > 136.95539934765625.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -1738.1980140154249 (T) = (0 -159411.94320078622) / Math.Sqrt((8025.586489799508 / (48)) + (1324.7481432334996 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.22200172119499742 = (130451.4882719617 - 159411.94320078622) / 130451.4882719617 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 159.4060676931487 > 136.94923890625.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -1436.772325296048 (T) = (0 -159418.835751471) / Math.Sqrt((6004.527395716846 / (48)) + (3382.0091009296793 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.22220347235742888 = (130435.59387372575 - 159418.835751471) / 130435.59387372575 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in System.MathBenchmarks.Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Exp2 - Duration of single invocation	21.38 μs	31.85 μs	1.49	0.20	False
Exp2M1 - Duration of single invocation	21.74 μs	32.63 μs	1.50	0.15	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.Exp2

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.852155575356406 > 22.43871004777207.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -39.34192158573979 (T) = (0 -32156.688303690727) / Math.Sqrt((1172490.5620477535 / (48)) + (515675.78692819574 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.46546845333739606 = (21942.941337603304 - 32156.688303690727) / 21942.941337603304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Exp2M1

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.631965774048496 > 22.812592114041955.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -138.06829427720194 (T) = (0 -32693.348697726582) / Math.Sqrt((258380.81501663243 / (48)) + (9263.043662628204 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.49547643303845235 = (21861.49375239667 - 32693.348697726582) / 21861.49375239667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Benchstone.BenchF.Romber

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	133.07 ms	168.10 ms	1.26	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Romber*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Romber.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.09715369230767 > 137.63503253500002.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -10.95252033934631 (T) = (0 -174674573.1296856) / Math.Sqrt((1292984168794.4968 / (48)) + (190468497558729.9 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.33336765381531946 = (131002557.79407053 - 174674573.1296856) / 131002557.79407053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Span.Sorting

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BubbleSortSpan - Duration of single invocation	300.44 μs	319.31 μs	1.06	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Histogram

Span.Sorting.BubbleSortSpan(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 319.30873333333335 > 315.7690501875.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -39.370841042322716 (T) = (0 -320995.177190522) / Math.Sqrt((1982924.5907538307 / (48)) + (2449589.5463043433 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.06469600771753979 = (301489.979171295 - 320995.177190522) / 301489.979171295 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Span.IndexerBench

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CoveredIndex1 - Duration of single invocation	919.52 ns	1.12 μs	1.21	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.IndexerBench*'

Payloads

Baseline
Compare

Histogram

Span.IndexerBench.CoveredIndex1(length: 1024)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1157786095694584 > 963.8954446044544.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -550.2843739415368 (T) = (0 -1114.6580858330633) / Math.Sqrt((3.1319919473648814 / (48)) + (0.744562974581747 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.2137961725038859 = (918.3239419297928 - 1114.6580858330633) / 918.3239419297928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	b9002422acc79f65a2f1b83f342ce52eb1ce2326
Compare	2bacbca96ddcf7ae809a0e32a0d5dbffd6354199
Diff	Diff

Regressions in Benchstone.MDBenchF.MDRomber

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	138.93 ms	174.07 ms	1.25	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchF.MDRomber*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDRomber.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 174.0749334 > 146.4243172780769.
IsChangePoint: Marked as a change because one of 3/10/2023 8:06:53 PM, 3/14/2023 10:43:28 AM falls between 3/5/2023 5:50:49 PM and 3/14/2023 10:43:28 AM.
IsRegressionStdDev: Marked as regression because -26.698370756718088 (T) = (0 -174656693.45912698) / Math.Sqrt((74098380611567.83 / (48)) + (390855591512.6477 / (12))) is less than -2.001717484144427 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (12) - 2, .025) and -0.23749921752432543 = (141136811.22848368 - 174656693.45912698) / 141136811.22848368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

radekdoulik · 2023-03-15T10:57:12Z

Looks like this comes from emscripten bump dotnet/runtime#81215

kg · 2023-03-15T22:37:45Z

We use emscripten libc for math in the jiterpreter, so it would make sense if they changed math in musl and made some of the operations slower somehow

performanceautofiler bot added AOT=true untriaged labels Mar 14, 2023

radekdoulik self-assigned this Mar 15, 2023

radekdoulik removed the untriaged label Mar 15, 2023

performanceautofiler bot added perf-regression arch-x64 os-linux branch-refs/heads/main arch-wasm kind-micro wasm-aot and removed refs/heads/main labels Mar 23, 2023

DrewScoggins mentioned this issue Sep 6, 2023

[Perf] 7.0 -> 8.0-RC1 Microbenchmark Comparison dotnet/runtime#91626

Closed

[Perf] Linux/x64: 34 Regressions on 3/11/2023 1:35:17 AM #14021

[Perf] Linux/x64: 34 Regressions on 3/11/2023 1:35:17 AM #14021

Comments

performanceautofiler bot commented Mar 14, 2023 • edited Loading

Run Information

Regressions in System.MathBenchmarks.Double

Repro

Payloads

Payloads

Histogram

System.MathBenchmarks.Double.Exp2M1

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in Benchstone.BenchF.BenchMk2

Repro

Payloads

Payloads

Histogram

Benchstone.BenchF.BenchMk2.Test

Description of detection logic

Docs

Run Information

Regressions in System.Collections.IndexerSet<Int32>

Repro

Payloads

Payloads

Histogram

System.Collections.IndexerSet<Int32>.Array(Size: 512)

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in Benchstone.BenchF.BenchMrk

Repro

Payloads

Payloads

Histogram

Benchstone.BenchF.BenchMrk.Test

Description of detection logic

Docs

Run Information

Regressions in System.IO.Tests.Perf_FileStream

Repro

Payloads

Payloads

Histogram

System.IO.Tests.Perf_FileStream.Read(fileSize: 104857600, userBufferSize: 4096, options: 0)

Description of detection logic

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in Benchstone.BenchF.Secant

Repro

Payloads

Payloads

Histogram

Benchstone.BenchF.Secant.Test

Description of detection logic

Docs

performanceautofiler bot commented Mar 14, 2023 • edited Loading

Run Information

Regressions in ByteMark

Repro

Payloads

Payloads

Histogram

performanceautofiler bot commented Mar 14, 2023 •

edited

Loading

performanceautofiler bot commented Mar 14, 2023 •

edited

Loading