[Perf] Linux/x64: 3 Regressions on 11/8/2022 2:29:58 AM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	bfa4812aec806f09f1e2927939ba98118e5e5bf7
Compare	f33badf332c43c0d325e8c802466ea5fd362fb91
Diff	Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadLine - Duration of single invocation	3.05 ms	4.78 ms	1.57	0.07	False
ReadLine - Duration of single invocation	1.49 ms	1.98 ms	1.33	0.04	False
ReadLine - Duration of single invocation	3.16 ms	6.27 ms	1.98	0.11	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Related Issues

Regressions

Improvements

• [Perf] Windows/x86: 1 Improvement on 11/8/2022 2:29:58 AM

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 1, 1])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 4.781691966517857 > 3.2166848428125.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsRegressionStdDev: Marked as regression because -57.09251996792648 (T) = (0 -4600162.224617006) / Math.Sqrt((4605864924.373739 / (10)) + (8595748692.52886 / (36))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (36) - 2, .025) and -0.48856679159305294 = (3090329.739046474 - 4600162.224617006) / 3090329.739046474 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   1,    8])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 1.9781770828125 > 1.5480122912478145.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsRegressionStdDev: Marked as regression because -89.13452586937301 (T) = (0 -1981355.6460121113) / Math.Sqrt((221593302.941373 / (10)) + (424404172.5873213 / (36))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (36) - 2, .025) and -0.355225713472459 = (1462011.5500430819 - 1981355.6460121113) / 1462011.5500430819 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [   0,    0])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 6.270424681089743 > 3.330590737103366.
IsChangePoint: Marked as a change because one of 11/7/2022 4:10:19 PM, 11/14/2022 6:23:13 PM falls between 11/5/2022 10:22:58 PM and 11/14/2022 6:23:13 PM.
IsRegressionStdDev: Marked as regression because -143.24380047732507 (T) = (0 -6254404.661465553) / Math.Sqrt((964507010.0760417 / (10)) + (13373204433.892677 / (37))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (37) - 2, .025) and -0.961104183815278 = (3189226.106946429 - 6254404.661465553) / 3189226.106946429 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[radekdoulik]

Looks like dotnet/runtime#69888. @GrabYourPitchforks @stephentoub please take a look

[stephentoub]

This is the same as dotnet/runtime#78401 and dotnet/runtime#78176. The regressions are for tests involving empty or otherwise really short lines, such that the overhead of the IndexOf now being used is observable. This is acceptable.

As an aside, is there anything we can do to better group such issues, rather than having at least three different issues created for the same tests? @DrewScoggins