[ASM] Avoid reporting unknown matcher WAF errors #5955

NachoEchevarria · 2024-08-27T16:41:48Z

Summary of changes

The unknown matcher error happens when a new operator is added and it is not supported by the WAF, so sending this particular error is causing a lot of noise because old versions of the WAF will return that error that actually is supported in newer versions. We already write warnings for this situation, but we are also reporting errors that are not actually errors, Which is generating a lot of noise. In particular, we return errors in the remote config operations and we add error tags to the span.

This PR corrects this.

Reason for change

It was reported and requested.

Implementation details

Test coverage

Other details

datadog-ddstaging · 2024-08-27T17:03:43Z

Datadog Report

Branch report: nacho/AvoidReportingUnknownMatcherErrors
Commit report: df29b01
Test service: dd-trace-dotnet

✅ 0 Failed, 358479 Passed, 2048 Skipped, 15h 34m 10.8s Total Time

andrewlock · 2024-08-27T17:04:26Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

andrewlock · 2024-08-27T17:43:02Z

Benchmarks Report for appsec 🐌

Benchmarks for #5955 compared to master:

All benchmarks have the same speed
1 benchmarks have fewer allocations
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	73.3μs	82ns	318ns	0.0729	0	6 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	62.7μs	68.4ns	265ns	0.0629	0	6.95 KB
master	`AllCycleSimpleBody`	net472	48.8μs	69.8ns	271ns	1.32	0	8.34 KB
master	`AllCycleMoreComplexBody`	net6.0	79.6μs	119ns	443ns	0.12	0	9.51 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	69.6μs	81.5ns	316ns	0.14	0	10.37 KB
master	`AllCycleMoreComplexBody`	net472	55.9μs	60.3ns	226ns	1.88	0.028	11.85 KB
master	`ObjectExtractorSimpleBody`	net6.0	141ns	0.243ns	0.939ns	0.00393	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	217ns	0.162ns	0.607ns	0.00374	0	272 B
master	`ObjectExtractorSimpleBody`	net472	165ns	0.113ns	0.437ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.03μs	1.14ns	4.41ns	0.0532	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	3.97μs	1.5ns	5.6ns	0.0498	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.84μs	2.57ns	9.96ns	0.602	0.00577	3.8 KB
#5955	`AllCycleSimpleBody`	net6.0	74.7μs	129ns	500ns	0.0746	0	6 KB
#5955	`AllCycleSimpleBody`	netcoreapp3.1	61.8μs	86.2ns	334ns	0.0927	0	6.95 KB
#5955	`AllCycleSimpleBody`	net472	49.5μs	65.3ns	253ns	1.31	0	8.34 KB
#5955	`AllCycleMoreComplexBody`	net6.0	79.2μs	90.5ns	351ns	0.118	0	9.51 KB
#5955	`AllCycleMoreComplexBody`	netcoreapp3.1	69.6μs	76.3ns	296ns	0.138	0	10.37 KB
#5955	`AllCycleMoreComplexBody`	net472	56.3μs	44.4ns	166ns	1.86	0.0281	11.85 KB
#5955	`ObjectExtractorSimpleBody`	net6.0	141ns	0.234ns	0.905ns	0.00397	0	280 B
#5955	`ObjectExtractorSimpleBody`	netcoreapp3.1	209ns	0.0852ns	0.319ns	0.00373	0	272 B
#5955	`ObjectExtractorSimpleBody`	net472	171ns	0.0986ns	0.369ns	0.0446	0	281 B
#5955	`ObjectExtractorMoreComplexBody`	net6.0	3.01μs	1.46ns	5.46ns	0.0528	0	3.78 KB
#5955	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.08μs	4.1ns	15.9ns	0.0507	0	3.69 KB
#5955	`ObjectExtractorMoreComplexBody`	net472	3.78μs	2.45ns	9.18ns	0.602	0.00568	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EncodeArgs`	net6.0	37.3μs	19.2ns	71.9ns	0.446	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	55.3μs	31.9ns	115ns	0.44	0	32.4 KB
master	`EncodeArgs`	net472	65.2μs	41.3ns	155ns	5.15	0.0652	32.5 KB
master	`EncodeLegacyArgs`	net6.0	71.8μs	20.6ns	77.2ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	105μs	101ns	392ns	0	0	2.14 KB
master	`EncodeLegacyArgs`	net472	151μs	85.1ns	318ns	0.303	0	2.15 KB
#5955	`EncodeArgs`	net6.0	36.7μs	28.4ns	102ns	0.446	0	32.4 KB
#5955	`EncodeArgs`	netcoreapp3.1	54.5μs	27ns	97.5ns	0.431	0	32.4 KB
#5955	`EncodeArgs`	net472	66.2μs	34ns	132ns	5.15	0.0657	32.5 KB
#5955	`EncodeLegacyArgs`	net6.0	73μs	24.4ns	94.4ns	0.0364	0	2.14 KB
#5955	`EncodeLegacyArgs`	netcoreapp3.1	106μs	200ns	747ns	0	0	2.15 KB
#5955	`EncodeLegacyArgs`	net472	151μs	105ns	405ns	0.302	0	2.15 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunWafRealisticBenchmark`	net6.0	185μs	68.4ns	265ns	0	2.42 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	195μs	158ns	612ns	0	2.37 KB
master	`RunWafRealisticBenchmark`	net472	208μs	97.6ns	352ns	0.311	2.43 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	76.7ns	297ns	0	1.46 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	131μs	51.4ns	178ns	0	1.45 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	138μs	37.4ns	145ns	0.207	1.48 KB
#5955	`RunWafRealisticBenchmark`	net6.0	183μs	321ns	1.24μs	0	2.42 KB
#5955	`RunWafRealisticBenchmark`	netcoreapp3.1	197μs	90ns	325ns	0	2.37 KB
#5955	`RunWafRealisticBenchmark`	net472	212μs	187ns	723ns	0.316	2.43 KB
#5955	`RunWafRealisticBenchmarkWithAttack`	net6.0	123μs	33.1ns	128ns	0	1.46 KB
#5955	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	132μs	135ns	505ns	0	1.45 KB
#5955	`RunWafRealisticBenchmarkWithAttack`	net472	140μs	104ns	377ns	0.21	1.48 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5955

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	253.15 KB	255.63 KB	2.48 KB	0.98%

Fewer allocations 🎉 in #5955

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	59.55 KB	59.07 KB	-480 B	-0.81%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	60μs	802ns	7.94μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	54.3μs	271ns	1.15μs	42.64 KB
master	`StringConcatBenchmark`	net472	37.6μs	96.7ns	362ns	59.55 KB
master	`StringConcatAspectBenchmark`	net6.0	306μs	1.72μs	11.2μs	253.15 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	333μs	1.87μs	11.7μs	251.8 KB
master	`StringConcatAspectBenchmark`	net472	290μs	6.63μs	64.3μs	278.53 KB
#5955	`StringConcatBenchmark`	net6.0	59.1μs	746ns	7.12μs	43.44 KB
#5955	`StringConcatBenchmark`	netcoreapp3.1	53.3μs	274ns	1.25μs	42.64 KB
#5955	`StringConcatBenchmark`	net472	37.1μs	144ns	539ns	59.07 KB
#5955	`StringConcatAspectBenchmark`	net6.0	316μs	1.83μs	14.3μs	255.63 KB
#5955	`StringConcatAspectBenchmark`	netcoreapp3.1	335μs	1.89μs	13.9μs	252.81 KB
#5955	`StringConcatAspectBenchmark`	net472	291μs	6.24μs	60.8μs	278.53 KB

andrewlock · 2024-08-27T21:39:53Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5955) (11.239M)   : 0, 11238560
    master (11.247M)   : 0, 11246521
    benchmarks/2.9.0 (11.120M)   : 0, 11120231

    section Automatic
    This PR (5955) (7.492M)   : 0, 7491949
    master (7.508M)   : 0, 7508439
    benchmarks/2.9.0 (7.874M)   : 0, 7874046

    section Trace stats
    master (7.707M)   : 0, 7707332

    section Manual
    master (11.233M)   : 0, 11232811

    section Manual + Automatic
    This PR (5955) (6.896M)   : 0, 6895505
    master (6.892M)   : 0, 6891553

    section DD_TRACE_ENABLED=0
    master (10.225M)   : 0, 10225326

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5955) (9.503M)   : 0, 9503007
    benchmarks/2.9.0 (9.342M)   : 0, 9341974

    section Automatic
    This PR (5955) (6.614M)   : 0, 6614083

    section Manual + Automatic
    This PR (5955) (6.055M)   : 0, 6055040

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5955) (10.031M)   : 0, 10030807
    master (10.060M)   : 0, 10059520
    benchmarks/2.9.0 (10.105M)   : 0, 10105239

    section Automatic
    This PR (5955) (6.684M)   : 0, 6684420
    master (6.690M)   : 0, 6690107
    benchmarks/2.9.0 (7.518M)   : 0, 7517801

    section Trace stats
    master (7.324M)   : 0, 7324117

    section Manual
    master (10.131M)   : 0, 10131126

    section Manual + Automatic
    This PR (5955) (6.129M)   : 0, 6129437
    master (6.279M)   : 0, 6279338

    section DD_TRACE_ENABLED=0
    master (9.441M)   : 0, 9441325

andrewlock · 2024-08-27T22:00:32Z

Benchmarks Report for tracer 🐌

Benchmarks for #5955 compared to master:

1 benchmarks are faster, with geometric mean 1.117
1 benchmarks are slower, with geometric mean 1.141
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.76μs	43.7ns	303ns	0.0148	0.00738	0	5.43 KB
master	`StartStopWithChild`	netcoreapp3.1	9.78μs	55.1ns	389ns	0.0241	0.00963	0	5.61 KB
master	`StartStopWithChild`	net472	15.8μs	46.9ns	175ns	1.02	0.302	0.103	6.07 KB
#5955	`StartStopWithChild`	net6.0	7.7μs	41.4ns	238ns	0.0113	0.00755	0	5.43 KB
#5955	`StartStopWithChild`	netcoreapp3.1	10.1μs	56.8ns	381ns	0.0155	0.00516	0	5.62 KB
#5955	`StartStopWithChild`	net472	16.3μs	37.6ns	146ns	1.02	0.308	0.0812	6.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	476μs	256ns	990ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	645μs	131ns	473ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	839μs	659ns	2.55μs	0.417	3.3 KB
#5955	`WriteAndFlushEnrichedTraces`	net6.0	469μs	228ns	885ns	0	2.7 KB
#5955	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	627μs	286ns	1.07μs	0	2.7 KB
#5955	`WriteAndFlushEnrichedTraces`	net472	839μs	259ns	970ns	0.414	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	187μs	884ns	3.54μs	0.183	18.45 KB
master	`SendRequest`	netcoreapp3.1	207μs	1.1μs	5.91μs	0.209	20.61 KB
master	`SendRequest`	net472	0.0059ns	0.00211ns	0.00818ns	0	0 b
#5955	`SendRequest`	net6.0	183μs	919ns	4.69μs	0.187	18.45 KB
#5955	`SendRequest`	netcoreapp3.1	204μs	1.01μs	4.62μs	0.198	20.61 KB
#5955	`SendRequest`	net472	0.000408ns	0.000216ns	0.00078ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5955

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.52 KB	41.83 KB	310 B	0.75%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	543μs	1.82μs	6.83μs	0.546	0	0	41.52 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	677μs	3.19μs	14.6μs	0.329	0	0	41.73 KB
master	`WriteAndFlushEnrichedTraces`	net472	875μs	3.56μs	13.8μs	8.13	2.57	0.428	53.27 KB
#5955	`WriteAndFlushEnrichedTraces`	net6.0	577μs	2.91μs	12.7μs	0.543	0	0	41.83 KB
#5955	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	666μs	1.97μs	7.1μs	0.329	0	0	41.8 KB
#5955	`WriteAndFlushEnrichedTraces`	net472	860μs	4.5μs	22.1μs	8.25	2.6	0.434	53.26 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.35μs	1.31ns	5.06ns	0.0142	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.72μs	1.68ns	6.5ns	0.0138	1.02 KB
master	`ExecuteNonQuery`	net472	2.02μs	0.813ns	2.93ns	0.157	987 B
#5955	`ExecuteNonQuery`	net6.0	1.36μs	0.84ns	3.14ns	0.0143	1.02 KB
#5955	`ExecuteNonQuery`	netcoreapp3.1	1.69μs	1.33ns	5.16ns	0.0136	1.02 KB
#5955	`ExecuteNonQuery`	net472	1.99μs	2.76ns	10.7ns	0.157	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.19μs	0.497ns	1.86ns	0.0137	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.46μs	0.583ns	2.1ns	0.0132	976 B
master	`CallElasticsearch`	net472	2.47μs	1.59ns	5.97ns	0.158	995 B
master	`CallElasticsearchAsync`	net6.0	1.35μs	1.64ns	6.37ns	0.0135	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.65μs	0.594ns	2.3ns	0.0142	1.02 KB
master	`CallElasticsearchAsync`	net472	2.51μs	1.25ns	4.67ns	0.166	1.05 KB
#5955	`CallElasticsearch`	net6.0	1.2μs	0.841ns	3.26ns	0.0137	976 B
#5955	`CallElasticsearch`	netcoreapp3.1	1.51μs	0.708ns	2.65ns	0.0127	976 B
#5955	`CallElasticsearch`	net472	2.54μs	1.13ns	4.38ns	0.157	995 B
#5955	`CallElasticsearchAsync`	net6.0	1.28μs	1.21ns	4.53ns	0.0135	952 B
#5955	`CallElasticsearchAsync`	netcoreapp3.1	1.63μs	1.14ns	4.4ns	0.0137	1.02 KB
#5955	`CallElasticsearchAsync`	net472	2.63μs	2.01ns	7.5ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5955

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0	1.117	1,342.72	1,202.49

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.34μs	0.561ns	2.1ns	0.013	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.57μs	0.783ns	2.82ns	0.0127	952 B
master	`ExecuteAsync`	net472	1.73μs	0.546ns	2.04ns	0.145	915 B
#5955	`ExecuteAsync`	net6.0	1.2μs	1ns	3.75ns	0.0132	952 B
#5955	`ExecuteAsync`	netcoreapp3.1	1.59μs	1.21ns	4.53ns	0.0127	952 B
#5955	`ExecuteAsync`	net472	1.81μs	0.673ns	2.61ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.14μs	0.957ns	3.58ns	0.0311	2.22 KB
master	`SendAsync`	netcoreapp3.1	5.1μs	2.15ns	8.05ns	0.0359	2.76 KB
master	`SendAsync`	net472	7.75μs	2.41ns	9.35ns	0.496	3.15 KB
#5955	`SendAsync`	net6.0	4.25μs	1.31ns	4.91ns	0.0321	2.22 KB
#5955	`SendAsync`	netcoreapp3.1	5.16μs	1.84ns	7.12ns	0.0387	2.76 KB
#5955	`SendAsync`	net472	7.7μs	1.68ns	6.5ns	0.497	3.15 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.47μs	0.435ns	1.68ns	0.0229	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.38μs	0.732ns	2.74ns	0.0225	1.64 KB
master	`EnrichedLog`	net472	2.71μs	1.55ns	5.99ns	0.249	1.57 KB
#5955	`EnrichedLog`	net6.0	1.47μs	0.684ns	2.56ns	0.0229	1.64 KB
#5955	`EnrichedLog`	netcoreapp3.1	2.17μs	0.855ns	3.2ns	0.0218	1.64 KB
#5955	`EnrichedLog`	net472	2.68μs	1.81ns	7.01ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	117μs	134ns	502ns	0.0589	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	121μs	386ns	1.5μs	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	215ns	832ns	0.682	0.227	4.46 KB
#5955	`EnrichedLog`	net6.0	117μs	206ns	798ns	0.0589	0	4.28 KB
#5955	`EnrichedLog`	netcoreapp3.1	121μs	212ns	823ns	0.0602	0	4.28 KB
#5955	`EnrichedLog`	net472	146μs	194ns	753ns	0.657	0.219	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.07μs	0.986ns	3.69ns	0.0305	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.39μs	0.973ns	3.77ns	0.0305	2.2 KB
master	`EnrichedLog`	net472	4.85μs	1.03ns	4.01ns	0.32	2.02 KB
#5955	`EnrichedLog`	net6.0	3.14μs	1.16ns	4.49ns	0.03	2.2 KB
#5955	`EnrichedLog`	netcoreapp3.1	4.03μs	1.4ns	5.42ns	0.028	2.2 KB
#5955	`EnrichedLog`	net472	4.87μs	1.23ns	4.43ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.31μs	0.747ns	2.79ns	0.0164	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.75μs	0.965ns	3.48ns	0.0158	1.14 KB
master	`SendReceive`	net472	2.14μs	1.44ns	5.38ns	0.183	1.16 KB
#5955	`SendReceive`	net6.0	1.28μs	0.447ns	1.61ns	0.016	1.14 KB
#5955	`SendReceive`	netcoreapp3.1	1.7μs	0.811ns	3.04ns	0.0152	1.14 KB
#5955	`SendReceive`	net472	2.12μs	0.926ns	3.59ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.79μs	1.15ns	4.44ns	0.0224	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.94μs	2.01ns	7.77ns	0.0218	1.65 KB
master	`EnrichedLog`	net472	4.35μs	1.25ns	4.67ns	0.322	2.04 KB
#5955	`EnrichedLog`	net6.0	2.86μs	0.97ns	3.76ns	0.0229	1.6 KB
#5955	`EnrichedLog`	netcoreapp3.1	3.93μs	1.7ns	6.57ns	0.0216	1.65 KB
#5955	`EnrichedLog`	net472	4.49μs	2.93ns	11.3ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5955

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.141	683.87	780.40

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	400ns	0.257ns	0.993ns	0.00803	576 B
master	`StartFinishSpan`	netcoreapp3.1	574ns	1.16ns	4.5ns	0.00761	576 B
master	`StartFinishSpan`	net472	618ns	0.706ns	2.73ns	0.0916	578 B
master	`StartFinishScope`	net6.0	537ns	0.549ns	2.05ns	0.00971	696 B
master	`StartFinishScope`	netcoreapp3.1	685ns	1.12ns	4.33ns	0.00929	696 B
master	`StartFinishScope`	net472	809ns	0.641ns	2.48ns	0.104	658 B
#5955	`StartFinishSpan`	net6.0	388ns	0.108ns	0.403ns	0.00812	576 B
#5955	`StartFinishSpan`	netcoreapp3.1	560ns	0.561ns	2.1ns	0.00782	576 B
#5955	`StartFinishSpan`	net472	559ns	0.715ns	2.77ns	0.0918	578 B
#5955	`StartFinishScope`	net6.0	552ns	0.395ns	1.53ns	0.00971	696 B
#5955	`StartFinishScope`	netcoreapp3.1	780ns	0.409ns	1.58ns	0.00926	696 B
#5955	`StartFinishScope`	net472	831ns	0.834ns	3.23ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	582ns	0.413ns	1.6ns	0.00987	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	899ns	1.01ns	3.77ns	0.00949	696 B
master	`RunOnMethodBegin`	net472	1.02μs	1.54ns	5.96ns	0.104	658 B
#5955	`RunOnMethodBegin`	net6.0	641ns	0.894ns	3.22ns	0.00968	696 B
#5955	`RunOnMethodBegin`	netcoreapp3.1	904ns	1.38ns	5.16ns	0.00919	696 B
#5955	`RunOnMethodBegin`	net472	1.06μs	1.19ns	4.61ns	0.104	658 B

NachoEchevarria · 2024-08-30T15:55:10Z

tracer/src/Datadog.Trace/AppSec/Security.cs

@@ -192,6 +192,11 @@ internal void SubscribeToChanges(params string[] productNames)
            }
        }

+        internal ApplyDetails[] UpdateFromRcmForTest(Dictionary<string, List<RemoteConfiguration>> configsByProduct)


I don't really like this but it's a fast and effective way to test this code... The alternatives for unit testing would be declaring UpdateFromRcm internal or use reflection in the tests.

FWIW, this is fine IMO 🙂 If you want to make sure people don't call it by accident, you could add [Obsolete] or [PublicApi]

NachoEchevarria · 2024-09-03T08:13:23Z

Thanks for you reviews!

NachoEchevarria changed the title ~~Avoid reporting unknown matcher errors in span tags~~ [ASM] Avoid reporting unknown matcher errors in span tags Aug 28, 2024

NachoEchevarria added the area:asm label Aug 28, 2024

NachoEchevarria marked this pull request as ready for review August 28, 2024 07:53

NachoEchevarria requested a review from a team as a code owner August 28, 2024 07:53

NachoEchevarria marked this pull request as draft August 28, 2024 08:14

NachoEchevarria changed the title ~~[ASM] Avoid reporting unknown matcher errors in span tags~~ [ASM] Avoid reporting unknown matcher errors Aug 30, 2024

NachoEchevarria added 4 commits August 30, 2024 13:47

Avoid reporting unknown matcher errors in span tags

1f56336

Add RCM unknown matcher errors exclusion

a8e7307

Add unit tests. Fix conditions.

a45886d

Add new operator to the RASP rule set of the integration tests

df29b01

NachoEchevarria force-pushed the nacho/AvoidReportingUnknownMatcherErrors branch from 2813112 to df29b01 Compare August 30, 2024 11:48

NachoEchevarria commented Aug 30, 2024

View reviewed changes

NachoEchevarria changed the title ~~[ASM] Avoid reporting unknown matcher errors~~ [ASM] Avoid reporting unknown matcher WAF errors Aug 30, 2024

NachoEchevarria marked this pull request as ready for review August 30, 2024 15:56

anna-git approved these changes Sep 2, 2024

View reviewed changes

daniel-romano-DD approved these changes Sep 2, 2024

View reviewed changes

NachoEchevarria merged commit 185755c into master Sep 3, 2024
70 of 71 checks passed

NachoEchevarria deleted the nacho/AvoidReportingUnknownMatcherErrors branch September 3, 2024 08:13

github-actions bot added this to the vNext-v3 milestone Sep 3, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ASM] Avoid reporting unknown matcher WAF errors #5955

[ASM] Avoid reporting unknown matcher WAF errors #5955

NachoEchevarria commented Aug 27, 2024 •

edited

Loading

datadog-ddstaging bot commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

Raw results

Raw results

Raw results

More allocations ⚠️ in #5955

Fewer allocations 🎉 in #5955

Raw results

andrewlock commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

Raw results

Raw results

Raw results

More allocations ⚠️ in #5955

Raw results

Raw results

Raw results

Faster 🎉 in #5955

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5955

Raw results

Raw results

NachoEchevarria Aug 30, 2024

andrewlock Aug 30, 2024

NachoEchevarria commented Sep 3, 2024

[ASM] Avoid reporting unknown matcher WAF errors #5955

[ASM] Avoid reporting unknown matcher WAF errors #5955

Conversation

NachoEchevarria commented Aug 27, 2024 • edited Loading

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Aug 27, 2024 • edited Loading

Datadog Report

andrewlock commented Aug 27, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Aug 27, 2024 • edited Loading

Benchmarks Report for appsec 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #5955

Fewer allocations 🎉 in #5955

Raw results

andrewlock commented Aug 27, 2024 • edited Loading

Throughput/Crank Report ⚡

andrewlock commented Aug 27, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #5955

Raw results

Raw results

Raw results

Faster 🎉 in #5955

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5955

Raw results

Raw results

NachoEchevarria Aug 30, 2024

Choose a reason for hiding this comment

andrewlock Aug 30, 2024

Choose a reason for hiding this comment

NachoEchevarria commented Sep 3, 2024

NachoEchevarria commented Aug 27, 2024 •

edited

Loading

datadog-ddstaging bot commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading

andrewlock commented Aug 27, 2024 •

edited

Loading