Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IAST] Calculate method full name on demand always #6127

Merged
merged 1 commit into from
Oct 8, 2024
Merged

[IAST] Calculate method full name on demand always #6127

merged 1 commit into from
Oct 8, 2024

Conversation

daniel-romano-DD
Copy link
Contributor

Summary of changes

Stop caching method's fullname in MethodInfo and calculate it on demand

Reason for change

FullName caching could led to race conditions, and uses memory. It is only used on method creation for exclusion decision, on debug traces and when instrumentation fails, so there's no need for caching it, freeing up memory in the process.

Implementation details

Remove _fullName member in MethodReference class and calculate it always when calling GetFullName()

Test coverage

Other details

@daniel-romano-DD daniel-romano-DD added area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) area:asm-iast labels Oct 8, 2024
@daniel-romano-DD daniel-romano-DD requested a review from a team as a code owner October 8, 2024 08:39
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Oct 8, 2024 •
edited
Loading

Datadog Report

Branch report: dani/iast/method_fullname_optimization
Commit report: 883a659
Test service: dd-trace-dotnet

✅ 0 Failed, 371580 Passed, 2726 Skipped, 26h 47m 24.15s Total Time

@andrewlock
Copy link
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6127) - mean (70ms)  : 69, 72
     .   : milestone, 70,
    master - mean (71ms)  : 68, 74
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (6127) - mean (1,107ms)  : 1080, 1134
     .   : milestone, 1107,
    master - mean (1,109ms)  : 1084, 1134
     .   : milestone, 1109,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6127) - mean (109ms)  : 106, 112
     .   : milestone, 109,
    master - mean (109ms)  : 107, 112
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (6127) - mean (774ms)  : 758, 790
     .   : milestone, 774,
    master - mean (777ms)  : 762, 792
     .   : milestone, 777,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6127) - mean (93ms)  : 90, 95
     .   : milestone, 93,
    master - mean (93ms)  : 92, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (6127) - mean (730ms)  : 713, 747
     .   : milestone, 730,
    master - mean (731ms)  : 716, 747
     .   : milestone, 731,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6127) - mean (190ms)  : 187, 193
     .   : milestone, 190,
    master - mean (191ms)  : 188, 194
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (6127) - mean (1,197ms)  : 1174, 1220
     .   : milestone, 1197,
    master - mean (1,199ms)  : 1171, 1227
     .   : milestone, 1199,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6127) - mean (275ms)  : 270, 280
     .   : milestone, 275,
    master - mean (276ms)  : 271, 281
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6127) - mean (938ms)  : 920, 956
     .   : milestone, 938,
    master - mean (939ms)  : 921, 957
     .   : milestone, 939,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6127) - mean (264ms)  : 260, 268
     .   : milestone, 264,
    master - mean (265ms)  : 261, 269
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (6127) - mean (925ms)  : 912, 939
     .   : milestone, 925,
    master - mean (927ms)  : 908, 947
     .   : milestone, 927,
Loading

@andrewlock
Copy link
Member

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6127) (11.073M)   : 0, 11073055
    master (11.013M)   : 0, 11012653
    benchmarks/2.9.0 (11.081M)   : 0, 11080577

    section Automatic
    This PR (6127) (7.278M)   : 0, 7278207
    master (7.257M)   : 0, 7256944
    benchmarks/2.9.0 (7.732M)   : 0, 7732233

    section Trace stats
    master (7.393M)   : 0, 7393311

    section Manual
    master (11.210M)   : 0, 11210238

    section Manual + Automatic
    This PR (6127) (6.895M)   : 0, 6894632
    master (6.902M)   : 0, 6901880

    section DD_TRACE_ENABLED=0
    master (10.246M)   : 0, 10245809
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6127) (9.755M)   : 0, 9755378
    master (9.699M)   : 0, 9699374
    benchmarks/2.9.0 (9.798M)   : 0, 9798067

    section Automatic
    This PR (6127) (6.654M)   : 0, 6654462
    master (6.629M)   : 0, 6628939

    section Trace stats
    master (6.955M)   : 0, 6954928

    section Manual
    master (9.662M)   : 0, 9662390

    section Manual + Automatic
    This PR (6127) (5.946M)   : 0, 5946071
    master (6.096M)   : 0, 6095670

    section DD_TRACE_ENABLED=0
    master (8.710M)   : 0, 8710481
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6127) (9.961M)   : 0, 9961077
    master (10.071M)   : 0, 10071070
    benchmarks/2.9.0 (10.067M)   : 0, 10067315

    section Automatic
    This PR (6127) (6.757M)   : 0, 6757387
    master (6.612M)   : 0, 6612085
    benchmarks/2.9.0 (7.552M)   : 0, 7552193

    section Trace stats
    master (7.286M)   : 0, 7285519

    section Manual
    master (10.026M)   : 0, 10025950

    section Manual + Automatic
    This PR (6127) (6.079M)   : 0, 6079000
    master (6.346M)   : 0, 6345551

    section DD_TRACE_ENABLED=0
    master (9.203M)   : 0, 9203356
Loading

@andrewlock
Copy link
Member

Benchmarks Report for tracer 🐌

Benchmarks for #6127 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.143
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.76μs 41.8ns 217ns 0.0202 0.00807 0 5.42 KB
master StartStopWithChild netcoreapp3.1 9.93μs 56.1ns 372ns 0.0187 0.00933 0 5.61 KB
master StartStopWithChild net472 16.3μs 64.5ns 250ns 1.03 0.312 0.0903 6.08 KB
#6127 StartStopWithChild net6.0 7.77μs 44.6ns 357ns 0.0158 0.00788 0 5.42 KB
#6127 StartStopWithChild netcoreapp3.1 9.77μs 52.8ns 284ns 0.0142 0.00475 0 5.62 KB
#6127 StartStopWithChild net472 16.9μs 73.3ns 284ns 1.01 0.302 0.0899 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 467μs 219ns 849ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 650μs 229ns 857ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 835μs 452ns 1.75μs 0.414 0 0 3.3 KB
#6127 WriteAndFlushEnrichedTraces net6.0 482μs 355ns 1.37μs 0 0 0 2.7 KB
#6127 WriteAndFlushEnrichedTraces netcoreapp3.1 652μs 377ns 1.46μs 0 0 0 2.7 KB
#6127 WriteAndFlushEnrichedTraces net472 829μs 457ns 1.77μs 0.414 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 193μs 1.04μs 5.82μs 0.191 0 0 18.45 KB
master SendRequest netcoreapp3.1 213μs 854ns 3.19μs 0.207 0 0 20.61 KB
master SendRequest net472 0.00144ns 0.000398ns 0.00154ns 0 0 0 0 b
#6127 SendRequest net6.0 191μs 1.02μs 5.3μs 0.188 0 0 18.45 KB
#6127 SendRequest netcoreapp3.1 212μs 1.13μs 6.18μs 0.212 0 0 20.61 KB
#6127 SendRequest net472 0.00097ns 0.000246ns 0.000888ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6127

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.44 KB 41.72 KB 273 B 0.66%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 566μs 2.05μs 7.68μs 0.563 0 0 41.68 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 676μs 3.41μs 19μs 0.332 0 0 41.44 KB
master WriteAndFlushEnrichedTraces net472 852μs 2.91μs 10.5μs 8.08 2.55 0.425 53.32 KB
#6127 WriteAndFlushEnrichedTraces net6.0 585μs 1.78μs 6.42μs 0.571 0 0 41.74 KB
#6127 WriteAndFlushEnrichedTraces netcoreapp3.1 668μs 1.09μs 4.21μs 0.332 0 0 41.72 KB
#6127 WriteAndFlushEnrichedTraces net472 873μs 3.98μs 14.3μs 8.08 2.55 0.425 53.29 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.26μs 0.591ns 2.29ns 0.0145 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.77μs 1.73ns 6.71ns 0.0133 0 0 1.02 KB
master ExecuteNonQuery net472 2.06μs 1.61ns 6.24ns 0.156 0 0 987 B
#6127 ExecuteNonQuery net6.0 1.38μs 1.43ns 5.54ns 0.0144 0 0 1.02 KB
#6127 ExecuteNonQuery netcoreapp3.1 1.76μs 3.33ns 12.9ns 0.0132 0 0 1.02 KB
#6127 ExecuteNonQuery net472 2.08μs 1.17ns 4.37ns 0.156 0 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.21μs 0.627ns 2.43ns 0.0133 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.51μs 0.75ns 2.91ns 0.0133 0 0 976 B
master CallElasticsearch net472 2.4μs 0.999ns 3.87ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.25μs 0.663ns 2.57ns 0.0133 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.59μs 1.18ns 4.24ns 0.0133 0 0 1.02 KB
master CallElasticsearchAsync net472 2.46μs 1.37ns 5.13ns 0.167 0 0 1.05 KB
#6127 CallElasticsearch net6.0 1.13μs 0.69ns 2.58ns 0.0136 0 0 976 B
#6127 CallElasticsearch netcoreapp3.1 1.51μs 0.595ns 2.23ns 0.0135 0 0 976 B
#6127 CallElasticsearch net472 2.5μs 0.866ns 3.35ns 0.157 0 0 995 B
#6127 CallElasticsearchAsync net6.0 1.23μs 0.776ns 2.9ns 0.0135 0 0 952 B
#6127 CallElasticsearchAsync netcoreapp3.1 1.73μs 0.689ns 2.48ns 0.0138 0 0 1.02 KB
#6127 CallElasticsearchAsync net472 2.6μs 1.65ns 6.41ns 0.166 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.25μs 0.823ns 3.08ns 0.0132 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.63μs 4.27ns 16.5ns 0.0122 0 0 952 B
master ExecuteAsync net472 1.73μs 0.72ns 2.79ns 0.145 0 0 915 B
#6127 ExecuteAsync net6.0 1.32μs 1.04ns 4.01ns 0.0132 0 0 952 B
#6127 ExecuteAsync netcoreapp3.1 1.62μs 0.571ns 2.06ns 0.013 0 0 952 B
#6127 ExecuteAsync net472 1.82μs 0.776ns 3.01ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.16μs 3.67ns 13.7ns 0.0313 0 0 2.22 KB
master SendAsync netcoreapp3.1 5.07μs 2.01ns 7.51ns 0.038 0 0 2.76 KB
master SendAsync net472 7.74μs 5.7ns 22.1ns 0.496 0 0 3.15 KB
#6127 SendAsync net6.0 4.06μs 1.86ns 6.98ns 0.0307 0 0 2.22 KB
#6127 SendAsync netcoreapp3.1 5.05μs 2.61ns 9.77ns 0.0378 0 0 2.76 KB
#6127 SendAsync net472 7.85μs 3.93ns 15.2ns 0.497 0 0 3.15 KB
Benchmarks.Trace.ILoggerBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6127

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.163 2,491.20 2,141.32

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.52μs 1.1ns 4.27ns 0.0228 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 2.5μs 2.25ns 8.41ns 0.0224 0 0 1.64 KB
master EnrichedLog net472 2.54μs 1.24ns 4.62ns 0.249 0 0 1.57 KB
#6127 EnrichedLog net6.0 1.44μs 1.1ns 4.24ns 0.023 0 0 1.64 KB
#6127 EnrichedLog netcoreapp3.1 2.14μs 1.44ns 5.4ns 0.0219 0 0 1.64 KB
#6127 EnrichedLog net472 2.52μs 0.704ns 2.63ns 0.25 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 116μs 148ns 575ns 0.058 0 0 4.28 KB
master EnrichedLog netcoreapp3.1 120μs 200ns 749ns 0 0 0 4.28 KB
master EnrichedLog net472 147μs 99.3ns 372ns 0.658 0.219 0 4.46 KB
#6127 EnrichedLog net6.0 114μs 147ns 570ns 0 0 0 4.28 KB
#6127 EnrichedLog netcoreapp3.1 121μs 364ns 1.41μs 0 0 0 4.28 KB
#6127 EnrichedLog net472 146μs 135ns 524ns 0.659 0.22 0 4.46 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.21μs 9.32ns 36.1ns 0.0301 0 0 2.2 KB
master EnrichedLog netcoreapp3.1 4.16μs 2.83ns 11ns 0.029 0 0 2.2 KB
master EnrichedLog net472 4.92μs 5.75ns 22.3ns 0.32 0 0 2.02 KB
#6127 EnrichedLog net6.0 3.04μs 1.37ns 5.13ns 0.0304 0 0 2.2 KB
#6127 EnrichedLog netcoreapp3.1 4.44μs 2.32ns 8.98ns 0.0289 0 0 2.2 KB
#6127 EnrichedLog net472 4.8μs 1.43ns 5.54ns 0.319 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.38μs 0.464ns 1.74ns 0.0158 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.66μs 0.713ns 2.76ns 0.0149 0 0 1.14 KB
master SendReceive net472 2.15μs 1.43ns 5.34ns 0.183 0.00109 0 1.16 KB
#6127 SendReceive net6.0 1.39μs 0.745ns 2.69ns 0.016 0 0 1.14 KB
#6127 SendReceive netcoreapp3.1 1.8μs 0.995ns 3.85ns 0.0152 0 0 1.14 KB
#6127 SendReceive net472 2.12μs 1.17ns 4.55ns 0.183 0.00105 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.85μs 1.03ns 3.99ns 0.0229 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 3.88μs 2.5ns 9.7ns 0.0216 0 0 1.65 KB
master EnrichedLog net472 4.56μs 0.811ns 2.92ns 0.322 0 0 2.04 KB
#6127 EnrichedLog net6.0 2.77μs 0.788ns 3.05ns 0.0222 0 0 1.6 KB
#6127 EnrichedLog netcoreapp3.1 3.97μs 1.42ns 5.5ns 0.0217 0 0 1.65 KB
#6127 EnrichedLog net472 4.37μs 1.05ns 3.92ns 0.324 0 0 2.04 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6127

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 1.124 668.46 594.97

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 494ns 0.291ns 1.09ns 0.00801 0 0 576 B
master StartFinishSpan netcoreapp3.1 568ns 0.588ns 2.2ns 0.00795 0 0 576 B
master StartFinishSpan net472 668ns 0.53ns 2.05ns 0.0916 0 0 578 B
master StartFinishScope net6.0 535ns 0.23ns 0.893ns 0.0098 0 0 696 B
master StartFinishScope netcoreapp3.1 714ns 0.329ns 1.28ns 0.00929 0 0 696 B
master StartFinishScope net472 869ns 0.51ns 1.91ns 0.104 0 0 658 B
#6127 StartFinishSpan net6.0 497ns 0.177ns 0.684ns 0.00797 0 0 576 B
#6127 StartFinishSpan netcoreapp3.1 557ns 0.328ns 1.23ns 0.00783 0 0 576 B
#6127 StartFinishSpan net472 596ns 0.513ns 1.99ns 0.0917 0 0 578 B
#6127 StartFinishScope net6.0 502ns 0.384ns 1.49ns 0.00983 0 0 696 B
#6127 StartFinishScope netcoreapp3.1 762ns 0.546ns 2.11ns 0.00912 0 0 696 B
#6127 StartFinishScope net472 935ns 0.689ns 2.58ns 0.104 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 738ns 0.532ns 2.06ns 0.00985 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 996ns 0.348ns 1.25ns 0.00966 0 0 696 B
master RunOnMethodBegin net472 1.14μs 0.707ns 2.64ns 0.104 0 0 658 B
#6127 RunOnMethodBegin net6.0 681ns 0.323ns 1.25ns 0.00957 0 0 696 B
#6127 RunOnMethodBegin netcoreapp3.1 929ns 0.431ns 1.55ns 0.00928 0 0 696 B
#6127 RunOnMethodBegin net472 1.04μs 1.82ns 7.07ns 0.104 0 0 658 B

@andrewlock
Copy link
Member

Benchmarks Report for appsec 🐌

Benchmarks for #6127 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.219
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6127

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472 1.219 4,672.82 3,833.55

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 71.4μs 78.9ns 306ns 0.072 0 0 6 KB
master AllCycleSimpleBody netcoreapp3.1 62.9μs 40.9ns 148ns 0.0941 0 0 6.95 KB
master AllCycleSimpleBody net472 48.5μs 25ns 93.6ns 1.31 0 0 8.34 KB
master AllCycleMoreComplexBody net6.0 76.9μs 71ns 266ns 0.115 0 0 9.51 KB
master AllCycleMoreComplexBody netcoreapp3.1 69μs 64.2ns 249ns 0.138 0 0 10.36 KB
master AllCycleMoreComplexBody net472 54.7μs 26.7ns 99.8ns 1.86 0.0274 0 11.85 KB
master ObjectExtractorSimpleBody net6.0 140ns 0.129ns 0.483ns 0.00392 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 195ns 0.334ns 1.29ns 0.0037 0 0 272 B
master ObjectExtractorSimpleBody net472 164ns 0.29ns 1.12ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 3.15μs 1.12ns 4.05ns 0.0536 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.89μs 3.27ns 12.7ns 0.0506 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 4.67μs 3.58ns 13.9ns 0.603 0.0046 0 3.8 KB
#6127 AllCycleSimpleBody net6.0 71.7μs 98.8ns 370ns 0.0718 0 0 6.01 KB
#6127 AllCycleSimpleBody netcoreapp3.1 62.6μs 74ns 287ns 0.0935 0 0 6.95 KB
#6127 AllCycleSimpleBody net472 49.5μs 37.3ns 145ns 1.31 0 0 8.34 KB
#6127 AllCycleMoreComplexBody net6.0 77.5μs 95.2ns 369ns 0.116 0 0 9.51 KB
#6127 AllCycleMoreComplexBody netcoreapp3.1 68.7μs 93.8ns 351ns 0.137 0 0 10.37 KB
#6127 AllCycleMoreComplexBody net472 56.1μs 41ns 159ns 1.87 0.028 0 11.85 KB
#6127 ObjectExtractorSimpleBody net6.0 142ns 0.16ns 0.62ns 0.00395 0 0 280 B
#6127 ObjectExtractorSimpleBody netcoreapp3.1 196ns 0.225ns 0.842ns 0.00371 0 0 272 B
#6127 ObjectExtractorSimpleBody net472 165ns 0.12ns 0.464ns 0.0446 0 0 281 B
#6127 ObjectExtractorMoreComplexBody net6.0 3.03μs 1.17ns 4.52ns 0.0531 0 0 3.78 KB
#6127 ObjectExtractorMoreComplexBody netcoreapp3.1 3.97μs 3.1ns 12ns 0.0493 0 0 3.69 KB
#6127 ObjectExtractorMoreComplexBody net472 3.84μs 3.43ns 13.3ns 0.602 0.00582 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 38.1μs 19ns 73.6ns 0.455 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 54.4μs 17.8ns 69ns 0.433 0 0 32.4 KB
master EncodeArgs net472 65.8μs 37.3ns 144ns 5.14 0.0659 0 32.5 KB
master EncodeLegacyArgs net6.0 75.8μs 418ns 2.51μs 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 104μs 426ns 1.65μs 0 0 0 2.15 KB
master EncodeLegacyArgs net472 151μs 103ns 400ns 0.3 0 0 2.15 KB
#6127 EncodeArgs net6.0 37.3μs 23.6ns 88.2ns 0.446 0 0 32.4 KB
#6127 EncodeArgs netcoreapp3.1 55.4μs 25.8ns 92.9ns 0.441 0 0 32.4 KB
#6127 EncodeArgs net472 65.3μs 38.5ns 144ns 5.15 0.0655 0 32.5 KB
#6127 EncodeLegacyArgs net6.0 74.6μs 79.6ns 308ns 0 0 0 2.14 KB
#6127 EncodeLegacyArgs netcoreapp3.1 105μs 374ns 1.45μs 0 0 0 2.14 KB
#6127 EncodeLegacyArgs net472 153μs 117ns 455ns 0.305 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 186μs 142ns 548ns 0 0 0 2.44 KB
master RunWafRealisticBenchmark netcoreapp3.1 198μs 261ns 1.01μs 0 0 0 2.39 KB
master RunWafRealisticBenchmark net472 211μs 168ns 651ns 0.311 0 0 2.46 KB
master RunWafRealisticBenchmarkWithAttack net6.0 122μs 108ns 404ns 0 0 0 1.47 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 130μs 57.4ns 222ns 0 0 0 1.46 KB
master RunWafRealisticBenchmarkWithAttack net472 139μs 32.7ns 127ns 0.208 0 0 1.49 KB
#6127 RunWafRealisticBenchmark net6.0 184μs 98.9ns 383ns 0 0 0 2.44 KB
#6127 RunWafRealisticBenchmark netcoreapp3.1 195μs 128ns 497ns 0 0 0 2.39 KB
#6127 RunWafRealisticBenchmark net472 207μs 53.2ns 206ns 0.311 0 0 2.46 KB
#6127 RunWafRealisticBenchmarkWithAttack net6.0 123μs 113ns 439ns 0 0 0 1.47 KB
#6127 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 131μs 74.3ns 278ns 0 0 0 1.46 KB
#6127 RunWafRealisticBenchmarkWithAttack net472 139μs 68ns 263ns 0.207 0 0 1.49 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6127

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 265.54 KB 264.01 KB -1.54 KB -0.58%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 60μs 782ns 7.78μs 0 0 0 43.44 KB
master StringConcatBenchmark netcoreapp3.1 60μs 802ns 7.98μs 0 0 0 42.64 KB
master StringConcatBenchmark net472 37.3μs 50ns 180ns 0 0 0 59.07 KB
master StringConcatAspectBenchmark net6.0 306μs 1.63μs 9.92μs 0 0 0 253.55 KB
master StringConcatAspectBenchmark netcoreapp3.1 339μs 1μs 3.47μs 0 0 0 265.54 KB
master StringConcatAspectBenchmark net472 276μs 5.47μs 53.3μs 0 0 0 278.53 KB
#6127 StringConcatBenchmark net6.0 60.1μs 704ns 6.97μs 0 0 0 43.44 KB
#6127 StringConcatBenchmark netcoreapp3.1 53.3μs 276ns 1.23μs 0 0 0 42.64 KB
#6127 StringConcatBenchmark net472 36.7μs 90.1ns 325ns 0 0 0 59.1 KB
#6127 StringConcatAspectBenchmark net6.0 302μs 1.4μs 7μs 0 0 0 253.99 KB
#6127 StringConcatAspectBenchmark netcoreapp3.1 349μs 1.96μs 13.4μs 0 0 0 264.01 KB
#6127 StringConcatAspectBenchmark net472 275μs 5.86μs 57.1μs 0 0 0 278.53 KB

NachoEchevarria
NachoEchevarria approved these changes Oct 8, 2024
View reviewed changes
Copy link
Contributor

@NachoEchevarria NachoEchevarria left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@daniel-romano-DD daniel-romano-DD merged commit f6a5089 into master Oct 8, 2024
80 checks passed
@daniel-romano-DD daniel-romano-DD deleted the dani/iast/method_fullname_optimization branch October 8, 2024 15:15
@github-actions github-actions bot added this to the vNext-v3 milestone Oct 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

@tonyredondo tonyredondo tonyredondo approved these changes

Assignees
No one assigned
Labels
area:asm-iast area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native)
Projects
None yet
Milestone
3.4.0
Development

Successfully merging this pull request may close these issues.
4 participants
@daniel-romano-DD @andrewlock @tonyredondo @NachoEchevarria