Regressions in System.Numerics.Tests.Perf_Matrix4x4

[performanceautofiler]

Run Information

Architecture	x64
OS	alpine 3.15
Baseline	3d3c23d79818373db6ae8752f33f68c56c1733c4
Compare	0c46b34ed70ad23d8323d32d1c8eeb9f4819ae68
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation	8.07 ns	11.63 ns	1.44	0.21	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.625000995348191 > 8.47419765122744.
IsChangePoint: Marked as a change because one of 11/25/2022 2:35:48 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -18.712345452882488 (T) = (0 -11.703298526863215) / Math.Sqrt((1.4615136939757034 / (44)) + (0.015439009616766749 / (11))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (11) - 2, .025) and -0.4234806616886659 = (8.221606967937076 - 11.703298526863215) / 8.221606967937076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[EgorBo]

@SingleAccretion can it be regressed by #76491 ?

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	alpine 3.15
Baseline	3d3c23d79818373db6ae8752f33f68c56c1733c4
Compare	0c46b34ed70ad23d8323d32d1c8eeb9f4819ae68
Diff	Diff

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
CreateScaleFromVectorWithCenterBenchmark - Duration of single invocation	8.07 ns	11.63 ns	1.44	0.21	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix4x4.CreateScaleFromVectorWithCenterBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.625000995348191 > 8.47419765122744.
IsChangePoint: Marked as a change because one of 11/25/2022 2:35:48 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsRegressionStdDev: Marked as regression because -18.712345452882488 (T) = (0 -11.703298526863215) / Math.Sqrt((1.4615136939757034 / (44)) + (0.015439009616766749 / (11))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (11) - 2, .025) and -0.4234806616886659 = (8.221606967937076 - 11.703298526863215) / 8.221606967937076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	tenet-performance, tenet-performance-benchmarks, area-CodeGen-coreclr, untriaged
Milestone:	-

[SingleAccretion]

Yes, there were a few diffs around Matrix4x4 benchmarks:

Top method regressions (bytes):
          16 (9.09 % of base) : 41847.dasm - System.Numerics.Matrix4x4:CreateScale(System.Numerics.Vector3,System.Numerics.Vector3):System.Numerics.Matrix4x4
           4 (3.51 % of base) : 30072.dasm - System.Numerics.Tests.Perf_Matrix4x4:CreateRotationXWithCenterBenchmark():System.Numerics.Matrix4x4:this
           3 (0.88 % of base) : 7944.dasm - System.Numerics.Matrix3x2:CreateRotation(float,System.Numerics.Vector2):System.Numerics.Matrix3x2

Top method improvements (bytes):
         -43 (-8.62 % of base) : 48643.dasm - System.Numerics.Matrix4x4:CreateShadow(System.Numerics.Vector3,System.Numerics.Plane):System.Numerics.Matrix4x4
         -25 (-5.66 % of base) : 53485.dasm - System.Numerics.Matrix4x4:CreateFromAxisAngle(System.Numerics.Vector3,float):System.Numerics.Matrix4x4
          -8 (-8.33 % of base) : 43985.dasm - Benchmarks.SIMD.RayTracer.Surfaces+<>c:<.cctor>b__3_2(Benchmarks.SIMD.RayTracer.Vector):double:this
          -3 (-2.27 % of base) : 43983.dasm - Benchmarks.SIMD.RayTracer.Surfaces+<>c:<.cctor>b__3_0(Benchmarks.SIMD.RayTracer.Vector):Benchmarks.SIMD.RayTracer.Color:this
          -3 (-0.37 % of base) : 52961.dasm - System.Numerics.Matrix4x4:CreateConstrainedBillboard(System.Numerics.Vector3,System.Numerics.Vector3,System.Numerics.Vector3,System.Numerics.Vector3,System.Numerics.Vector3):System.Numerics.Matrix4x4
          -1 (-1.09 % of base) : 15630.dasm - System.Numerics.Tests.Perf_Plane:CreateFromVector4Benchmark():System.Numerics.Plane:this

Will see what's up.

[dakersnar]

Another related regression: dotnet/perf-autofiling-issues#10179