[Perf] Changes at 4/6/2022 2:58:43 AM

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	0c5e7daf58e5d79cb7a2a8767d1f0e6ec4de4dd4
Compare	22870c2e31ae3c028cece52babd3cae57b560713
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsMatch - Duration of single invocation	183.32 ns	201.62 ns	1.10	0.09	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.62319232842881 > 192.53219449415403.
IsChangePoint: Marked as a change because one of 2/24/2022 10:17:16 AM, 2/26/2022 10:27:55 AM, 3/16/2022 5:02:20 PM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -4.934693326282283 (T) = (0 -199.63844173197674) / Math.Sqrt((152.17957837233513 / (23)) + (13.351311993112793 / (34))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (34) - 2, .025) and -0.07002339391360182 = (186.57390377401074 - 199.63844173197674) / 186.57390377401074 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	0c5e7daf58e5d79cb7a2a8767d1f0e6ec4de4dd4
Compare	22870c2e31ae3c028cece52babd3cae57b560713
Diff	Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	22.36 ms	26.83 ms	1.20	0.01	True
Count - Duration of single invocation	22.39 ms	26.81 ms	1.20	0.01	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.83443133333333 > 23.4838382625.
IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/6/2022 9:32:10 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -2102.1932277379365 (T) = (0 -26809750.833022095) / Math.Sqrt((40925051.142538585 / (23)) + (91087983.05575176 / (34))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (34) - 2, .025) and -0.19841637579515675 = (22370981.7176302 - 26809750.833022095) / 22370981.7176302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.80830285714286 > 23.507565253846153.
IsChangePoint: Marked as a change because one of 3/4/2022 7:13:54 PM, 3/6/2022 9:32:10 PM, 4/6/2022 12:51:22 AM, 4/12/2022 5:02:33 PM falls between 4/4/2022 4:14:58 AM and 4/12/2022 5:02:33 PM.
IsRegressionStdDev: Marked as regression because -1692.3518673275523 (T) = (0 -26821029.01345495) / Math.Sqrt((92354506.44324696 / (23)) + (96472162.4219048 / (34))) is less than -2.0040447832881556 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (23) + (34) - 2, .025) and -0.19785905149105662 = (22390805.479216434 - 26821029.01345495) / 22390805.479216434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[dakersnar]

@stephentoub This seems to be caused by dotnet/runtime#67365, dotnet/runtime#67576, or dotnet/runtime#67184

I saw your reply in dotnet/runtime#68565, but since this is a 64 bit test, I figured I would check to see if you had any insight.

[stephentoub]

The latter two cited regressions appear to just be bimodal and they've already gone back down. Is there actually an issue there?

[AndyAyersMS]

The latter two cited regressions appear to just be bimodal and they've already gone back down. Is there actually an issue there?

No issue. This test is unusually stable for long stretches, so it trips up our analysis.