[ASM] Rename snapshots for ASM ownership #6155

NachoEchevarria · 2024-10-15T11:00:12Z

Summary of changes

We have two snapshots that don't follow the naming conventions. This means that approval from other teams will be required when there is a change in either of them. Renaming them to follow the conventions will prevent this situation.

Reason for change

Implementation details

Test coverage

Other details

andrewlock · 2024-10-15T11:30:52Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6155) - mean (72ms)  : 68, 76
     .   : milestone, 72,
    master - mean (70ms)  : 68, 72
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6155) - mean (1,133ms)  : 1105, 1160
     .   : milestone, 1133,
    master - mean (1,111ms)  : 1087, 1135
     .   : milestone, 1111,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6155) - mean (112ms)  : 108, 117
     .   : milestone, 112,
    master - mean (109ms)  : 106, 111
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6155) - mean (803ms)  : 721, 885
     .   : milestone, 803,
    master - mean (772ms)  : 756, 789
     .   : milestone, 772,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6155) - mean (96ms)  : 90, 102
     .   : milestone, 96,
    master - mean (92ms)  : 89, 95
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (6155) - mean (746ms)  : 681, 810
     .   : milestone, 746,
    master - mean (730ms)  : 714, 745
     .   : milestone, 730,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6155) - mean (190ms)  : 187, 193
     .   : milestone, 190,
    master - mean (189ms)  : 186, 192
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (6155) - mean (1,199ms)  : 1169, 1228
     .   : milestone, 1199,
    master - mean (1,197ms)  : 1170, 1224
     .   : milestone, 1197,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6155) - mean (275ms)  : 271, 279
     .   : milestone, 275,
    master - mean (273ms)  : 269, 278
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (6155) - mean (942ms)  : 924, 960
     .   : milestone, 942,
    master - mean (939ms)  : 919, 958
     .   : milestone, 939,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6155) - mean (267ms)  : 262, 272
     .   : milestone, 267,
    master - mean (263ms)  : 259, 266
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (6155) - mean (945ms)  : 921, 970
     .   : milestone, 945,
    master - mean (926ms)  : 906, 947
     .   : milestone, 926,

datadog-ddstaging · 2024-10-15T11:31:37Z

Datadog Report

Branch report: nacho/RenameASMSnapshots
Commit report: 6368b06
Test service: dd-trace-dotnet

✅ 0 Failed, 367021 Passed, 2086 Skipped, 16h 25m 56.8s Total Time
❄️ 1 New Flaky
⌛ 4 Performance Regressions

New Flaky Tests (1)

SubmitsTraces - Datadog.Trace.ClrProfiler.IntegrationTests.StackExchangeRedisTests - Last Failure

Expand for error

 Results do not match.
 Differences:
 Received: StackExchangeRedisTests.Latest.SchemaV0.received.txt
 Verified: StackExchangeRedisTests.Latest.SchemaV0.verified.txt
 Received Content:
 [
   {
     TraceId: Id_1,
     SpanId: Id_2,
     Name: redis.command,
 ...

⌛ Performance Regressions vs Default Branch (4)

Baseline - Samples.FakeDbCommand.windows.net462.json.scenarios 72.14ms (+2.27ms, +3%) - Details
CallTarget+Inlining+NGEN - Samples.FakeDbCommand.windows.netcoreapp31.json.scenarios 803.07ms (+30.9ms, +4%) - Details
Baseline - Samples.FakeDbCommand.windows.net60.json.scenarios 95.91ms (+3.87ms, +4%) - Details
Baseline - Samples.FakeDbCommand.windows.netcoreapp31.json.scenarios 112.43ms (+3.55ms, +3%) - Details

andrewlock · 2024-10-15T11:57:13Z

Benchmarks Report for appsec 🐌

Benchmarks for #6155 compared to master:

1 benchmarks are faster, with geometric mean 1.234
2 benchmarks are slower, with geometric mean 1.252
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6155

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472	1.328	165.19	219.42

Faster 🎉 in #6155

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net6.0	1.234	192.95	156.33

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	72.8μs	107ns	414ns	0.0728	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	63.4μs	127ns	493ns	0.094	0	6.95 KB
master	`AllCycleSimpleBody`	net472	49μs	27.8ns	100ns	1.32	0	8.34 KB
master	`AllCycleMoreComplexBody`	net6.0	80.1μs	175ns	676ns	0.121	0	9.51 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	71μs	103ns	399ns	0.142	0	10.37 KB
master	`AllCycleMoreComplexBody`	net472	56.9μs	88.6ns	343ns	1.87	0.0284	11.85 KB
master	`ObjectExtractorSimpleBody`	net6.0	193ns	0.129ns	0.5ns	0.00395	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	224ns	0.156ns	0.604ns	0.00371	0	272 B
master	`ObjectExtractorSimpleBody`	net472	165ns	0.217ns	0.839ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.12μs	1.99ns	7.45ns	0.0532	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.92μs	2.63ns	9.83ns	0.0512	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	4μs	5.72ns	22.2ns	0.603	0.00597	3.8 KB
#6155	`AllCycleSimpleBody`	net6.0	72.5μs	128ns	496ns	0.0723	0	6 KB
#6155	`AllCycleSimpleBody`	netcoreapp3.1	63.9μs	134ns	502ns	0.0947	0	6.95 KB
#6155	`AllCycleSimpleBody`	net472	48.4μs	59.4ns	222ns	1.32	0	8.34 KB
#6155	`AllCycleMoreComplexBody`	net6.0	78.5μs	92.7ns	359ns	0.118	0	9.51 KB
#6155	`AllCycleMoreComplexBody`	netcoreapp3.1	72.5μs	184ns	711ns	0.107	0	10.37 KB
#6155	`AllCycleMoreComplexBody`	net472	56.5μs	142ns	550ns	1.87	0.0287	11.85 KB
#6155	`ObjectExtractorSimpleBody`	net6.0	156ns	0.187ns	0.724ns	0.004	0	280 B
#6155	`ObjectExtractorSimpleBody`	netcoreapp3.1	211ns	0.31ns	1.16ns	0.00371	0	272 B
#6155	`ObjectExtractorSimpleBody`	net472	220ns	0.141ns	0.51ns	0.0446	0	281 B
#6155	`ObjectExtractorMoreComplexBody`	net6.0	3.05μs	2.97ns	10.7ns	0.0534	0	3.78 KB
#6155	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.14μs	3.26ns	12.6ns	0.0517	0	3.69 KB
#6155	`ObjectExtractorMoreComplexBody`	net472	3.94μs	8.67ns	32.4ns	0.602	0.00585	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	37.5μs	21.8ns	81.5ns	0.449	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	54.4μs	23.4ns	90.5ns	0.432	0	32.4 KB
master	`EncodeArgs`	net472	67.2μs	45.6ns	176ns	5.16	0.0671	32.5 KB
master	`EncodeLegacyArgs`	net6.0	72.2μs	13ns	48.5ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	103μs	61.4ns	230ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	151μs	49.9ns	187ns	0.306	0	2.15 KB
#6155	`EncodeArgs`	net6.0	37.8μs	20.2ns	78ns	0.452	0	32.4 KB
#6155	`EncodeArgs`	netcoreapp3.1	54.9μs	17.6ns	68.1ns	0.438	0	32.4 KB
#6155	`EncodeArgs`	net472	68.7μs	56.8ns	212ns	5.15	0.0687	32.5 KB
#6155	`EncodeLegacyArgs`	net6.0	78μs	66.1ns	256ns	0	0	2.14 KB
#6155	`EncodeLegacyArgs`	netcoreapp3.1	107μs	283ns	1.1μs	0	0	2.14 KB
#6155	`EncodeLegacyArgs`	net472	152μs	122ns	474ns	0.305	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	185μs	181ns	701ns	0	2.44 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	195μs	110ns	411ns	0	2.39 KB
master	`RunWafRealisticBenchmark`	net472	209μs	150ns	580ns	0.314	2.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	52.9ns	205ns	0	1.47 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	130μs	123ns	476ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	139μs	25.9ns	100ns	0.209	1.49 KB
#6155	`RunWafRealisticBenchmark`	net6.0	186μs	246ns	952ns	0	2.44 KB
#6155	`RunWafRealisticBenchmark`	netcoreapp3.1	197μs	230ns	891ns	0	2.39 KB
#6155	`RunWafRealisticBenchmark`	net472	209μs	78.2ns	303ns	0.314	2.46 KB
#6155	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	83.7ns	313ns	0	1.47 KB
#6155	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	130μs	228ns	884ns	0	1.46 KB
#6155	`RunWafRealisticBenchmarkWithAttack`	net472	141μs	90.5ns	338ns	0.209	1.49 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #6155

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1	1.180	54,000.00	63,700.00	multimodal

More allocations ⚠️ in #6155

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	255.23 KB	264.47 KB	9.24 KB	3.62%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	61.3μs	701ns	6.97μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	54.2μs	283ns	1.44μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.5μs	65.2ns	235ns	59.07 KB
master	`StringConcatAspectBenchmark`	net6.0	314μs	1.77μs	12.6μs	255.23 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	328μs	1.8μs	12.1μs	252.73 KB
master	`StringConcatAspectBenchmark`	net472	272μs	5.43μs	51.8μs	278.53 KB
#6155	`StringConcatBenchmark`	net6.0	59.5μs	928ns	8.9μs	43.44 KB
#6155	`StringConcatBenchmark`	netcoreapp3.1	64.5μs	895ns	8.9μs	42.64 KB
#6155	`StringConcatBenchmark`	net472	37.9μs	126ns	505ns	59.07 KB
#6155	`StringConcatAspectBenchmark`	net6.0	302μs	5.39μs	53.1μs	264.47 KB
#6155	`StringConcatAspectBenchmark`	netcoreapp3.1	311μs	4.87μs	47.2μs	253.1 KB
#6155	`StringConcatAspectBenchmark`	net472	303μs	6.57μs	64.4μs	278.53 KB

andrewlock · 2024-10-15T12:12:08Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6155) (11.167M)   : 0, 11167475
    master (11.168M)   : 0, 11168199
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (6155) (7.336M)   : 0, 7335764
    master (7.377M)   : 0, 7377465
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.726M)   : 0, 7725756

    section Manual
    master (11.062M)   : 0, 11062026

    section Manual + Automatic
    This PR (6155) (6.800M)   : 0, 6799558
    master (6.902M)   : 0, 6901659

    section DD_TRACE_ENABLED=0
    master (10.195M)   : 0, 10194503

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6155) (9.569M)   : 0, 9569105
    master (9.584M)   : 0, 9583681
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (6155) (6.587M)   : 0, 6586928
    master (6.629M)   : 0, 6629382

    section Trace stats
    master (6.851M)   : 0, 6850733

    section Manual
    master (9.454M)   : 0, 9454208

    section Manual + Automatic
    This PR (6155) (6.052M)   : 0, 6052057
    master (6.166M)   : 0, 6165611

    section DD_TRACE_ENABLED=0
    master (8.858M)   : 0, 8858385

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6155) (10.165M)   : 0, 10165293
    master (10.308M)   : 0, 10307557
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (6155) (6.623M)   : 0, 6623049
    master (6.544M)   : 0, 6543815
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.344M)   : 0, 7343919

    section Manual
    master (10.030M)   : 0, 10029915

    section Manual + Automatic
    This PR (6155) (5.893M)   : 0, 5892746
    master (6.179M)   : 0, 6178568

    section DD_TRACE_ENABLED=0
    master (9.389M)   : 0, 9389459

andrewlock · 2024-10-15T12:27:19Z

Benchmarks Report for tracer 🐌

Benchmarks for #6155 compared to master:

1 benchmarks are faster, with geometric mean 1.140
1 benchmarks are slower, with geometric mean 1.117
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.09μs	45.8ns	327ns	0.0152	0.0038	0	5.42 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	55.5ns	328ns	0.015	0.00499	0	5.62 KB
master	`StartStopWithChild`	net472	17.4μs	86ns	365ns	1.03	0.327	0.106	6.06 KB
#6155	`StartStopWithChild`	net6.0	8.05μs	46.2ns	346ns	0.0155	0.00775	0	5.42 KB
#6155	`StartStopWithChild`	netcoreapp3.1	10.2μs	58.3ns	436ns	0.0201	0.0101	0	5.62 KB
#6155	`StartStopWithChild`	net472	16.9μs	81.3ns	335ns	1.03	0.315	0.0938	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	473μs	298ns	1.15μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	649μs	510ns	1.91μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	845μs	365ns	1.41μs	0.422	3.3 KB
#6155	`WriteAndFlushEnrichedTraces`	net6.0	479μs	375ns	1.4μs	0	2.7 KB
#6155	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	638μs	201ns	726ns	0	2.7 KB
#6155	`WriteAndFlushEnrichedTraces`	net472	846μs	532ns	2.06μs	0.422	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	190μs	976ns	4.68μs	0.193	18.45 KB
master	`SendRequest`	netcoreapp3.1	217μs	1.25μs	10.1μs	0.214	20.61 KB
master	`SendRequest`	net472	0.000988ns	0.000396ns	0.00143ns	0	0 b
#6155	`SendRequest`	net6.0	198μs	1.15μs	10.1μs	0.193	18.45 KB
#6155	`SendRequest`	netcoreapp3.1	223μs	1.28μs	9.21μs	0.211	20.61 KB
#6155	`SendRequest`	net472	0.000919ns	0.000335ns	0.0013ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	586μs	2.98μs	13.3μs	0.573	0	0	41.78 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	682μs	2.58μs	9.99μs	0.342	0	0	41.95 KB
master	`WriteAndFlushEnrichedTraces`	net472	862μs	4.29μs	18.2μs	8.74	2.62	0.437	53.28 KB
#6155	`WriteAndFlushEnrichedTraces`	net6.0	588μs	3.05μs	14.3μs	0.558	0	0	41.87 KB
#6155	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	684μs	2.25μs	8.41μs	0.342	0	0	41.79 KB
#6155	`WriteAndFlushEnrichedTraces`	net472	856μs	4.13μs	16μs	8.3	2.62	0.437	53.32 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.3μs	0.876ns	3.39ns	0.0143	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.79μs	1.47ns	5.67ns	0.0134	1.02 KB
master	`ExecuteNonQuery`	net472	2.15μs	2.21ns	8.56ns	0.156	987 B
#6155	`ExecuteNonQuery`	net6.0	1.32μs	1.4ns	5.44ns	0.0146	1.02 KB
#6155	`ExecuteNonQuery`	netcoreapp3.1	1.82μs	1.52ns	5.71ns	0.0141	1.02 KB
#6155	`ExecuteNonQuery`	net472	2.11μs	2.79ns	10.8ns	0.157	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.13μs	0.805ns	3.12ns	0.0136	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.47μs	0.643ns	2.49ns	0.0131	976 B
master	`CallElasticsearch`	net472	2.46μs	2ns	7.74ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.25μs	1.25ns	4.83ns	0.0131	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.67μs	1.42ns	5.49ns	0.0134	1.02 KB
master	`CallElasticsearchAsync`	net472	2.57μs	1.81ns	7.03ns	0.166	1.05 KB
#6155	`CallElasticsearch`	net6.0	1.16μs	1.08ns	4.16ns	0.0134	976 B
#6155	`CallElasticsearch`	netcoreapp3.1	1.55μs	1.44ns	5.38ns	0.0132	976 B
#6155	`CallElasticsearch`	net472	2.59μs	2.36ns	9.15ns	0.157	995 B
#6155	`CallElasticsearchAsync`	net6.0	1.29μs	1.14ns	4.43ns	0.013	952 B
#6155	`CallElasticsearchAsync`	netcoreapp3.1	1.62μs	2.23ns	8.65ns	0.0137	1.02 KB
#6155	`CallElasticsearchAsync`	net472	2.64μs	1.84ns	7.12ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6155

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.117	1,199.28	1,339.31

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.2μs	0.688ns	2.66ns	0.0133	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.61μs	0.942ns	3.65ns	0.0128	952 B
master	`ExecuteAsync`	net472	1.83μs	0.903ns	3.5ns	0.145	915 B
#6155	`ExecuteAsync`	net6.0	1.34μs	0.964ns	3.73ns	0.0133	952 B
#6155	`ExecuteAsync`	netcoreapp3.1	1.64μs	0.466ns	1.68ns	0.013	952 B
#6155	`ExecuteAsync`	net472	1.8μs	0.74ns	2.86ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.14μs	3.83ns	14.8ns	0.0311	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.01μs	3.45ns	13.4ns	0.0375	2.76 KB
master	`SendAsync`	net472	7.69μs	3.95ns	15.3ns	0.5	3.15 KB
#6155	`SendAsync`	net6.0	4.14μs	1.99ns	6.89ns	0.0312	2.22 KB
#6155	`SendAsync`	netcoreapp3.1	5.09μs	2.76ns	10.7ns	0.0377	2.76 KB
#6155	`SendAsync`	net472	7.84μs	2.83ns	11ns	0.499	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.62μs	7.34ns	28.4ns	0.0231	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.07μs	1.32ns	5.12ns	0.0218	1.64 KB
master	`EnrichedLog`	net472	2.59μs	1.73ns	6.7ns	0.249	1.57 KB
#6155	`EnrichedLog`	net6.0	1.55μs	0.594ns	2.22ns	0.0231	1.64 KB
#6155	`EnrichedLog`	netcoreapp3.1	2.26μs	1.56ns	5.83ns	0.0225	1.64 KB
#6155	`EnrichedLog`	net472	2.65μs	1.25ns	4.85ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	116μs	171ns	664ns	0.0576	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	122μs	225ns	871ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	107ns	372ns	0.681	0.227	4.46 KB
#6155	`EnrichedLog`	net6.0	118μs	162ns	626ns	0.0595	0	4.28 KB
#6155	`EnrichedLog`	netcoreapp3.1	122μs	195ns	755ns	0.0616	0	4.28 KB
#6155	`EnrichedLog`	net472	153μs	126ns	488ns	0.687	0.229	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.08μs	1.08ns	4.06ns	0.0307	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.21μs	1.2ns	4.63ns	0.0295	2.2 KB
master	`EnrichedLog`	net472	4.72μs	1.6ns	6.18ns	0.319	2.02 KB
#6155	`EnrichedLog`	net6.0	3.07μs	1.16ns	4.5ns	0.0309	2.2 KB
#6155	`EnrichedLog`	netcoreapp3.1	4.12μs	2.31ns	8.93ns	0.0288	2.2 KB
#6155	`EnrichedLog`	net472	4.82μs	1.44ns	5.39ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.34μs	0.703ns	2.54ns	0.0161	0	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.75μs	1.2ns	4.63ns	0.0148	0	1.14 KB
master	`SendReceive`	net472	2.16μs	0.969ns	3.75ns	0.183	0.00107	1.16 KB
#6155	`SendReceive`	net6.0	1.41μs	0.785ns	3.04ns	0.0163	0	1.14 KB
#6155	`SendReceive`	netcoreapp3.1	1.73μs	0.767ns	2.97ns	0.0157	0	1.14 KB
#6155	`SendReceive`	net472	2.12μs	0.827ns	3.1ns	0.183	0.00106	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.77μs	2.83ns	11ns	0.0221	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.96μs	1.69ns	6.08ns	0.0203	1.65 KB
master	`EnrichedLog`	net472	4.34μs	2.95ns	11.4ns	0.323	2.04 KB
#6155	`EnrichedLog`	net6.0	2.69μs	2.01ns	7.51ns	0.0216	1.6 KB
#6155	`EnrichedLog`	netcoreapp3.1	4.1μs	1.69ns	6.11ns	0.0207	1.65 KB
#6155	`EnrichedLog`	net472	4.52μs	1.89ns	7.32ns	0.322	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6155

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.140	477.21	418.54

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	477ns	0.236ns	0.913ns	0.00816	576 B
master	`StartFinishSpan`	netcoreapp3.1	615ns	0.275ns	1.07ns	0.00776	576 B
master	`StartFinishSpan`	net472	710ns	0.75ns	2.91ns	0.0917	578 B
master	`StartFinishScope`	net6.0	483ns	0.286ns	1.11ns	0.00972	696 B
master	`StartFinishScope`	netcoreapp3.1	780ns	1.52ns	5.89ns	0.00934	696 B
master	`StartFinishScope`	net472	927ns	1.44ns	5.59ns	0.104	658 B
#6155	`StartFinishSpan`	net6.0	419ns	0.195ns	0.754ns	0.00806	576 B
#6155	`StartFinishSpan`	netcoreapp3.1	564ns	0.987ns	3.82ns	0.00771	576 B
#6155	`StartFinishSpan`	net472	705ns	1.64ns	6.35ns	0.0918	578 B
#6155	`StartFinishScope`	net6.0	481ns	0.657ns	2.55ns	0.00968	696 B
#6155	`StartFinishScope`	netcoreapp3.1	853ns	0.691ns	2.59ns	0.00911	696 B
#6155	`StartFinishScope`	net472	915ns	4.41ns	18.7ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	616ns	0.426ns	1.59ns	0.0096	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.01μs	0.453ns	1.75ns	0.00922	696 B
master	`RunOnMethodBegin`	net472	1.14μs	1.32ns	4.76ns	0.105	658 B
#6155	`RunOnMethodBegin`	net6.0	668ns	0.431ns	1.61ns	0.00966	696 B
#6155	`RunOnMethodBegin`	netcoreapp3.1	940ns	2.52ns	9.76ns	0.00953	696 B
#6155	`RunOnMethodBegin`	net472	1.19μs	0.719ns	2.78ns	0.104	658 B

robertpi

Thank you!

andrewlock

Good idea 👍

Rename snapshots

5fffb6f

github-actions bot added the area:tests unit tests, integration tests label Oct 15, 2024

NachoEchevarria changed the title ~~Rename snapshots~~ [ASM] Rename snapshots for ASM ownership Oct 15, 2024

NachoEchevarria added the area:asm label Oct 15, 2024

Use Prefix

6368b06

NachoEchevarria marked this pull request as ready for review October 16, 2024 07:15

NachoEchevarria requested review from a team as code owners October 16, 2024 07:15

robertpi approved these changes Oct 16, 2024

View reviewed changes

andrewlock approved these changes Oct 16, 2024

View reviewed changes

NachoEchevarria merged commit 47fc17a into master Oct 16, 2024
75 checks passed

NachoEchevarria deleted the nacho/RenameASMSnapshots branch October 16, 2024 08:47

github-actions bot added this to the vNext-v3 milestone Oct 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Rename snapshots for ASM ownership #6155

[ASM] Rename snapshots for ASM ownership #6155

NachoEchevarria commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading

datadog-ddstaging bot commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading

Slower ⚠️ in #6155

Faster 🎉 in #6155

Raw results

Raw results

Raw results

Slower ⚠️ in #6155

More allocations ⚠️ in #6155

Raw results

andrewlock commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6155

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6155

Raw results

Raw results

robertpi left a comment

andrewlock left a comment

[ASM] Rename snapshots for ASM ownership #6155

[ASM] Rename snapshots for ASM ownership #6155

Conversation

NachoEchevarria commented Oct 15, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

andrewlock commented Oct 15, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

datadog-ddstaging bot commented Oct 15, 2024 • edited Loading

Datadog Report

New Flaky Tests (1)

⌛ Performance Regressions vs Default Branch (4)

andrewlock commented Oct 15, 2024 • edited Loading

Benchmarks Report for appsec 🐌

Benchmark details

Slower ⚠️ in #6155

Faster 🎉 in #6155

Raw results

Raw results

Raw results

Slower ⚠️ in #6155

More allocations ⚠️ in #6155

Raw results

andrewlock commented Oct 15, 2024 • edited Loading

Throughput/Crank Report ⚡

andrewlock commented Oct 15, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6155

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6155

Raw results

Raw results

robertpi left a comment

Choose a reason for hiding this comment

andrewlock left a comment

Choose a reason for hiding this comment

NachoEchevarria commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading

datadog-ddstaging bot commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading

andrewlock commented Oct 15, 2024 •

edited

Loading