[Perf] Linux/x64: 8 Improvements on 2/13/2023 3:18:56 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	0a51a37218dcda8085a0a9d8a46828094a0dd4c9
Compare	6f36be56480dc0a24a9713df5d81875a9ca2c980
Diff	Diff

Improvements in System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation	1.29 ms	1.17 ms	0.91	0.14	False
DeserializeFromReader - Duration of single invocation	1.32 ms	1.20 ms	0.91	0.12	False
DeserializeFromUtf8Bytes - Duration of single invocation	1.21 ms	1.01 ms	0.84	0.13	False
DeserializeFromReader - Duration of single invocation	1.47 ms	1.34 ms	0.91	0.12	False
DeserializeFromString - Duration of single invocation	1.26 ms	1.09 ms	0.87	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<MyEventsListerViewModel>.DeserializeFromUtf8Bytes(Mode: Reflection)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1708255955357143 < 1.251069805194805.
IsChangePoint: Marked as a change because one of 12/9/2022 9:29:01 AM, 1/13/2023 11:01:49 AM, 2/12/2023 9:12:37 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 21.240730833743786 (T) = (0 -1177281.4468721384) / Math.Sqrt((1921932679.5446668 / (299)) + (65096548.418235995 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.07915994284517608 = (1278486.3535472786 - 1177281.4468721384) / 1278486.3535472786 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;.DeserializeFromReader(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1989632613324177 < 1.2769094234308225.
IsChangePoint: Marked as a change because one of 12/9/2022 9:29:01 AM, 1/13/2023 11:01:49 AM, 2/12/2023 11:53:55 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 8.138216131369786 (T) = (0 -1211204.168222739) / Math.Sqrt((1306861347.865658 / (299)) + (1206796164.8277605 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.0952906000374148 = (1338777.0352257078 - 1211204.168222739) / 1338777.0352257078 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;.DeserializeFromUtf8Bytes(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0143119464285715 < 1.1319881265865381.
IsChangePoint: Marked as a change because one of 1/13/2023 11:01:49 AM, 2/12/2023 9:12:37 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 12.041467467938958 (T) = (0 -1025744.9496482944) / Math.Sqrt((2002178562.8597174 / (299)) + (641396199.1974273 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.12001675681674615 = (1165641.4569188405 - 1025744.9496482944) / 1165641.4569188405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;.DeserializeFromReader(Mode: Reflection)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.3393277976398603 < 1.3880477816056547.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/13/2023 11:01:49 AM, 1/18/2023 9:06:44 AM, 2/12/2023 11:53:55 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 10.712750134056634 (T) = (0 -1332378.1134932775) / Math.Sqrt((2023740938.2260754 / (299)) + (536626226.1597407 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.07908995955420038 = (1446805.9364933104 - 1332378.1134932775) / 1446805.9364933104 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;MyEventsListerViewModel&gt;.DeserializeFromString(Mode: SourceGen)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0888110169642857 < 1.152678464326923.
IsChangePoint: Marked as a change because one of 12/6/2022 4:40:30 PM, 1/13/2023 11:01:49 AM, 2/12/2023 11:53:55 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 14.006075531356695 (T) = (0 -1083772.4311031743) / Math.Sqrt((1817174220.7277026 / (299)) + (354939630.29082054 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.10189129431208538 = (1206727.4531907013 - 1083772.4311031743) / 1206727.4531907013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	252018c3d3fffdb592413cf61d5b80cf751e0a59
Compare	6f36be56480dc0a24a9713df5d81875a9ca2c980
Diff	Diff

Improvements in Microsoft.Extensions.Logging.FormattingOverhead

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TwoArguments - Duration of single invocation	1.02 μs	853.62 ns	0.84	0.22	False
TwoArguments_DefineMessage - Duration of single invocation	1.12 μs	983.76 ns	0.88	0.17	False
FourArguments_DefineMessage - Duration of single invocation	1.62 μs	1.29 μs	0.79	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Logging.FormattingOverhead*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 853.6188130406942 < 0.9708645986092251.
IsChangePoint: Marked as a change because one of 12/9/2022 1:52:32 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/13/2023 12:37:33 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 5.852051146818334 (T) = (0 -878.952226959078) / Math.Sqrt((2608.1669774210204 / (299)) + (2910.008365687258 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.1530119717573675 = (1037.7386664871358 - 878.952226959078) / 1037.7386664871358 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.TwoArguments_DefineMessage

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 983.7572110866956 < 1.0633167565425374.
IsChangePoint: Marked as a change because one of 12/7/2022 6:04:36 PM, 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/12/2023 11:53:55 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 14.58874625495942 (T) = (0 -957.3802511125256) / Math.Sqrt((2158.269439112714 / (299)) + (686.4965553232056 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15482482981629433 = (1132.7595567016376 - 957.3802511125256) / 1132.7595567016376 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Logging.FormattingOverhead.FourArguments_DefineMessage

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.286441150215138 < 1.5426834640550458.
IsChangePoint: Marked as a change because one of 1/12/2023 3:42:01 AM, 1/15/2023 4:08:29 PM, 2/12/2023 11:53:55 PM, 2/14/2023 10:45:34 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 10:45:34 AM.
IsImprovementStdDev: Marked as improvement because 26.72246439240735 (T) = (0 -1287.8564774047543) / Math.Sqrt((5655.891181149881 / (299)) + (324.3674693869655 / (5))) is greater than 1.9678502273660254 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (5) - 2, .975) and 0.15961761796891466 = (1532.464869494512 - 1287.8564774047543) / 1532.464869494512 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

this also looks like dotnet/runtime#81695 @jandupej @SamMonoRT