Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 7/5/2022 10:32:06 PM #6733

Closed
performanceautofiler bot opened this issue Jul 12, 2022 · 1 comment
Closed

[Perf] Changes at 7/5/2022 10:32:06 PM #6733

performanceautofiler bot opened this issue Jul 12, 2022 · 1 comment
Assignees
@tannergooding
Labels
arch-x64 branch-refs/heads/main kind-micro os-alpine perf-improvement runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jul 12, 2022 •
edited
Loading

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDXposMatrix

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 34.26 μs 24.45 μs 0.71 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDXposMatrix*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDXposMatrix.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.45421042118093 < 32.55272206504033.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 583.6000010622305 (T) = (0 -24537.33125379975) / Math.Sqrt((16553.90605766711 / (299)) + (10586.087139488065 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.2866684194961015 = (34398.212450465944 - 24537.33125379975) / 34398.212450465944 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchNumericSortRectangular - Duration of single invocation 974.34 ms 780.60 ms 0.80 0.00 False
BenchNeural - Duration of single invocation 586.96 ms 501.94 ms 0.86 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchNumericSortRectangular

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 780.6019800769232 < 925.463784275.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 320.40610047927186 (T) = (0 -779774264.7631271) / Math.Sqrt((23548174345917.758 / (299)) + (12616061500759.107 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1962289422396519 = (970144741.1355089 - 779774264.7631271) / 970144741.1355089 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### ByteMark.BenchNeural

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 501.935362 < 544.3153957384616.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 61.19961434157426 (T) = (0 -505393165.410965) / Math.Sqrt((40973128916618.43 / (299)) + (52109624252056.63 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.12007229979057063 = (574357603.8016279 - 505393165.410965) / 574357603.8016279 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDAddArray2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 16.42 ms 10.88 ms 0.66 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDAddArray2*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDAddArray2.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 10.878526305803572 < 15.606283548666667.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 349.44310911324885 (T) = (0 -10716633.942103762) / Math.Sqrt((4986767616.621469 / (299)) + (11657502308.781933 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.34891506649465576 = (16459655.861497212 - 10716633.942103762) / 16459655.861497212 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDInProd

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.35 secs 1.18 secs 0.87 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInProd*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDInProd.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1816533965 < 1.285773078298077.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 187.57439559610685 (T) = (0 -1187572352.3035078) / Math.Sqrt((38764396733194.16 / (299)) + (26998946916218.39 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1179359175567149 = (1346356093.5549898 - 1187572352.3035078) / 1346356093.5549898 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDMidpoint

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 442.55 ms 415.21 ms 0.94 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDMidpoint*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMidpoint.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 415.2065742857143 < 420.1461699265384.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 5/23/2022 10:48:42 PM, 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 22.84961124253839 (T) = (0 -412026184.0876931) / Math.Sqrt((386748016928479.44 / (299)) + (39085060853064.61 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.07508957407800027 = (445476851.098271 - 412026184.0876931) / 445476851.098271 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDRomber

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 150.84 ms 125.51 ms 0.83 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDRomber*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDRomber.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 125.50942542307693 < 143.30489618571426.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 233.23863439697644 (T) = (0 -125582429.31150663) / Math.Sqrt((367041668021.4863 / (299)) + (494114271154.9436 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.16887654140207087 = (151099608.62295836 - 125582429.31150663) / 151099608.62295836 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDGeneralArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 10.88 ms 9.30 ms 0.85 0.01 False
Test2 - Duration of single invocation 10.60 ms 9.30 ms 0.88 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDGeneralArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDGeneralArray.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.301361461538463 < 10.328925606160713.
IsChangePoint: Marked as a change because one of 6/14/2022 10:05:19 AM, 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 103.49999241528238 (T) = (0 -9300823.578864444) / Math.Sqrt((47229734811.12752 / (299)) + (6984232.149951507 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.12275038923620418 = (10602254.437897654 - 9300823.578864444) / 10602254.437897654 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Benchstone.MDBenchI.MDGeneralArray.Test2

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.302007109375 < 10.072081192885046.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 122.41772392182695 (T) = (0 -9303092.569888964) / Math.Sqrt((32111961171.875072 / (299)) + (82308306.18607251 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1208791389102665 = (10582268.015295545 - 9303092.569888964) / 10582268.015295545 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDLLoops

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 604.87 ms 572.30 ms 0.95 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDLLoops*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDLLoops.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 572.3012537857143 < 577.6544251313095.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 46.993955995024926 (T) = (0 -571582237.1537386) / Math.Sqrt((29013262355255.766 / (299)) + (24073119779601.887 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.060818507214710694 = (608596146.2662795 - 571582237.1537386) / 608596146.2662795 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDLogicArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 352.47 ms 293.53 ms 0.83 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDLogicArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDLogicArray.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 293.5326173846154 < 334.54280510102564.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 2962.672146326839 (T) = (0 -293611714.3854914) / Math.Sqrt((26999400692.321907 / (299)) + (13754354976.873549 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1660363841934536 = (352067774.68526894 - 293611714.3854914) / 352067774.68526894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchI.MDArray2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 1.07 secs 919.27 ms 0.86 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDArray2*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDArray2.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 919.2742426923077 < 1.0203072489772618.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 570.9822030792318 (T) = (0 -919853871.2594603) / Math.Sqrt((6737283087307.014 / (299)) + (2348044803693.1206 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.1441097472534866 = (1074733434.9324467 - 919853871.2594603) / 1074733434.9324467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDInvMt

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.64 ms 2.52 ms 0.54 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDInvMt*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDInvMt.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.5183204791666665 < 4.402104256529017.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 5889.06679597039 (T) = (0 -2515151.8836436435) / Math.Sqrt((3075126.4715031087 / (299)) + (5478197.044016493 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.45716722720541136 = (4633382.5253314115 - 2515151.8836436435) / 4633382.5253314115 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 338db1a1ba498f308797ce7a54c0ed54e7286d50
Compare ffcb0149763663a55a79784be216afe81f6bf76c
Diff Diff

Improvements in Benchstone.MDBenchF.MDSqMtx

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 752.22 ms 594.85 ms 0.79 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchF.MDSqMtx*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchF.MDSqMtx.Test

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 594.8503986153846 < 714.8829040633332.
IsChangePoint: Marked as a change because one of 7/5/2022 7:01:28 PM, 7/12/2022 10:14:57 AM falls between 7/3/2022 4:26:07 AM and 7/12/2022 10:14:57 AM.
IsImprovementStdDev: Marked as improvement because 1244.2540846235213 (T) = (0 -595300734.4144526) / Math.Sqrt((154400526873.38232 / (299)) + (709224673801.1129 / (46))) is greater than 1.9669042814877102 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (46) - 2, .975) and 0.20876012738626928 = (752364428.3091732 - 595300734.4144526) / 752364428.3091732 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tannergooding tannergooding

Labels
arch-x64 branch-refs/heads/main kind-micro os-alpine perf-improvement runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AndyAyersMS @tannergooding