Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid double instrumentation of kafka-clients 3.8+ #8006

Merged
merged 1 commit into from
Nov 25, 2024
Merged

Avoid double instrumentation of kafka-clients 3.8+ #8006

merged 1 commit into from
Nov 25, 2024

Conversation

mcculls
Copy link
Contributor

@mcculls mcculls commented Nov 24, 2024 •
edited
Loading

What Does This Do

Use existence of MetadataRecoveryStrategy to decide whether to apply pre-3.8 or post-3.8 kafka-clients instrumentation

Motivation

Avoids double instrumentation of kafka-clients versions from 3.8 onwards.

Additional Notes

One exception: IAST's KafkaDeserializerInstrumentation is compatible with all 3.x versions of kafka-clients

Also note this doesn't address the warning instrumenting kafka-clients 3.7 where KafkaConsumerInfoInstrumentation doesn't apply because the metadata field has been removed from KafkaConsumer. This is unrelated to the double-instrumentation issue, and will be addressed in another PR.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@mcculls mcculls added type: bug inst: kafka Kafka instrumentation labels Nov 24, 2024
@pr-commenter
Copy link

pr-commenter bot commented Nov 24, 2024 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/kafka-client-muzzle-fix
git_commit_date 1732489991 1732491285
git_commit_sha 4925a50 b5db502
See matching parameters
Baseline Candidate
ci_job_date 1732492517 1732492517
ci_job_id 718268253 718268253
ci_pipeline_id 49818571 49818571
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@mcculls mcculls force-pushed the mcculls/kafka-client-muzzle-fix branch 2 times, most recently from be1e8ae to c03f7ee Compare November 24, 2024 23:33
@mcculls
Use existence of MetadataRecoveryStrategy to decide whether to apply …
b5db502 
…pre-3.8 or post-3.8 kafka-clients instrumentation

One exception: IAST's KafkaDeserializerInstrumentation is compatible with all 3.x versions of kafka-clients
@mcculls mcculls force-pushed the mcculls/kafka-client-muzzle-fix branch from c03f7ee to b5db502 Compare November 24, 2024 23:34
@pr-commenter
Copy link

pr-commenter bot commented Nov 25, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/kafka-client-muzzle-fix
git_commit_date 1732489991 1732491285
git_commit_sha 4925a50 b5db502
release_version 1.43.0-SNAPSHOT~4925a50233 1.43.0-SNAPSHOT~b5db5029f1
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732493673 1732493673
ci_job_id 718265435 718265435
ci_pipeline_id 49818571 49818571
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 9 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.088 s) : 0, 1088441
Total [baseline] (10.392 s) : 0, 10391831
Agent [candidate] (1.088 s) : 0, 1088087
Total [candidate] (10.429 s) : 0, 10428821
section appsec
Agent [baseline] (1.228 s) : 0, 1228002
Total [baseline] (10.722 s) : 0, 10722036
Agent [candidate] (1.224 s) : 0, 1223791
Total [candidate] (10.721 s) : 0, 10721254
section iast
Agent [baseline] (1.221 s) : 0, 1220912
Total [baseline] (10.975 s) : 0, 10975440
Agent [candidate] (1.227 s) : 0, 1226840
Total [candidate] (10.959 s) : 0, 10959083
section profiling
Agent [baseline] (1.313 s) : 0, 1313056
Total [baseline] (10.743 s) : 0, 10743230
Agent [candidate] (1.316 s) : 0, 1315541
Total [candidate] (10.723 s) : 0, 10723238
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent appsec 1.228 s 139.56 ms (12.8%)
Agent iast 1.221 s 132.471 ms (12.2%)
Agent profiling 1.313 s 224.615 ms (20.6%)
Total tracing 10.392 s -
Total appsec 10.722 s 330.205 ms (3.2%)
Total iast 10.975 s 583.609 ms (5.6%)
Total profiling 10.743 s 351.398 ms (3.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.088 s -
Agent appsec 1.224 s 135.704 ms (12.5%)
Agent iast 1.227 s 138.753 ms (12.8%)
Agent profiling 1.316 s 227.455 ms (20.9%)
Total tracing 10.429 s -
Total appsec 10.721 s 292.433 ms (2.8%)
Total iast 10.959 s 530.262 ms (5.1%)
Total profiling 10.723 s 294.417 ms (2.8%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.281 ms) : 0, 691281
BytebuddyAgent [candidate] (691.058 ms) : 0, 691058
GlobalTracer [baseline] (316.441 ms) : 0, 316441
GlobalTracer [candidate] (316.828 ms) : 0, 316828
AppSec [baseline] (54.757 ms) : 0, 54757
AppSec [candidate] (54.389 ms) : 0, 54389
Remote Config [baseline] (684.01 µs) : 0, 684
Remote Config [candidate] (673.817 µs) : 0, 674
Telemetry [baseline] (11.615 ms) : 0, 11615
Telemetry [candidate] (11.45 ms) : 0, 11450
section appsec
BytebuddyAgent [baseline] (713.569 ms) : 0, 713569
BytebuddyAgent [candidate] (711.073 ms) : 0, 711073
GlobalTracer [baseline] (315.465 ms) : 0, 315465
GlobalTracer [candidate] (313.697 ms) : 0, 313697
AppSec [baseline] (166.546 ms) : 0, 166546
AppSec [candidate] (166.921 ms) : 0, 166921
Remote Config [baseline] (647.529 µs) : 0, 648
Remote Config [candidate] (648.615 µs) : 0, 649
Telemetry [baseline] (7.824 ms) : 0, 7824
Telemetry [candidate] (7.888 ms) : 0, 7888
IAST [baseline] (20.418 ms) : 0, 20418
IAST [candidate] (19.743 ms) : 0, 19743
section iast
BytebuddyAgent [baseline] (812.381 ms) : 0, 812381
BytebuddyAgent [candidate] (815.936 ms) : 0, 815936
GlobalTracer [baseline] (306.976 ms) : 0, 306976
GlobalTracer [candidate] (309.57 ms) : 0, 309570
AppSec [baseline] (58.498 ms) : 0, 58498
AppSec [candidate] (58.146 ms) : 0, 58146
Remote Config [baseline] (626.905 µs) : 0, 627
Remote Config [candidate] (643.707 µs) : 0, 644
Telemetry [baseline] (7.628 ms) : 0, 7628
Telemetry [candidate] (7.6 ms) : 0, 7600
IAST [baseline] (21.104 ms) : 0, 21104
IAST [candidate] (21.169 ms) : 0, 21169
section profiling
BytebuddyAgent [baseline] (684.688 ms) : 0, 684688
BytebuddyAgent [candidate] (686.493 ms) : 0, 686493
GlobalTracer [baseline] (434.559 ms) : 0, 434559
GlobalTracer [candidate] (434.315 ms) : 0, 434315
AppSec [baseline] (53.749 ms) : 0, 53749
AppSec [candidate] (53.906 ms) : 0, 53906
Remote Config [baseline] (657.123 µs) : 0, 657
Remote Config [candidate] (673.024 µs) : 0, 673
Telemetry [baseline] (7.765 ms) : 0, 7765
Telemetry [candidate] (7.751 ms) : 0, 7751
ProfilingAgent [baseline] (92.814 ms) : 0, 92814
ProfilingAgent [candidate] (93.406 ms) : 0, 93406
Profiling [baseline] (92.838 ms) : 0, 92838
Profiling [candidate] (93.431 ms) : 0, 93431
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.092 s) : 0, 1092250
Total [baseline] (8.654 s) : 0, 8653995
Agent [candidate] (1.094 s) : 0, 1094407
Total [candidate] (8.626 s) : 0, 8625894
section iast
Agent [baseline] (1.216 s) : 0, 1215708
Total [baseline] (9.163 s) : 0, 9163054
Agent [candidate] (1.223 s) : 0, 1223409
Total [candidate] (9.214 s) : 0, 9213552
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.23 s) : 0, 1230039
Total [baseline] (9.168 s) : 0, 9167918
Agent [candidate] (1.226 s) : 0, 1226226
Total [candidate] (9.179 s) : 0, 9179097
section iast_TELEMETRY_OFF
Agent [baseline] (1.215 s) : 0, 1214795
Total [baseline] (9.219 s) : 0, 9218824
Agent [candidate] (1.214 s) : 0, 1213838
Total [candidate] (9.181 s) : 0, 9180905
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.092 s -
Agent iast 1.216 s 123.458 ms (11.3%)
Agent iast_HARDCODED_SECRET_DISABLED 1.23 s 137.788 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.215 s 122.544 ms (11.2%)
Total tracing 8.654 s -
Total iast 9.163 s 509.06 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.168 s 513.923 ms (5.9%)
Total iast_TELEMETRY_OFF 9.219 s 564.829 ms (6.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.094 s -
Agent iast 1.223 s 129.002 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.226 s 131.819 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.214 s 119.431 ms (10.9%)
Total tracing 8.626 s -
Total iast 9.214 s 587.658 ms (6.8%)
Total iast_HARDCODED_SECRET_DISABLED 9.179 s 553.203 ms (6.4%)
Total iast_TELEMETRY_OFF 9.181 s 555.011 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (693.421 ms) : 0, 693421
BytebuddyAgent [candidate] (696.796 ms) : 0, 696796
GlobalTracer [baseline] (317.044 ms) : 0, 317044
GlobalTracer [candidate] (317.047 ms) : 0, 317047
AppSec [baseline] (54.659 ms) : 0, 54659
AppSec [candidate] (54.701 ms) : 0, 54701
Remote Config [baseline] (687.437 µs) : 0, 687
Remote Config [candidate] (696.832 µs) : 0, 697
Telemetry [baseline] (12.736 ms) : 0, 12736
Telemetry [candidate] (11.335 ms) : 0, 11335
section iast
BytebuddyAgent [baseline] (809.667 ms) : 0, 809667
BytebuddyAgent [candidate] (814.926 ms) : 0, 814926
GlobalTracer [baseline] (306.082 ms) : 0, 306082
GlobalTracer [candidate] (308.247 ms) : 0, 308247
AppSec [baseline] (57.513 ms) : 0, 57513
AppSec [candidate] (56.036 ms) : 0, 56036
IAST [baseline] (20.704 ms) : 0, 20704
IAST [candidate] (22.377 ms) : 0, 22377
Remote Config [baseline] (611.422 µs) : 0, 611
Remote Config [candidate] (628.801 µs) : 0, 629
Telemetry [baseline] (7.432 ms) : 0, 7432
Telemetry [candidate] (7.406 ms) : 0, 7406
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (819.731 ms) : 0, 819731
BytebuddyAgent [candidate] (816.391 ms) : 0, 816391
GlobalTracer [baseline] (309.746 ms) : 0, 309746
GlobalTracer [candidate] (309.054 ms) : 0, 309054
AppSec [baseline] (58.043 ms) : 0, 58043
AppSec [candidate] (56.537 ms) : 0, 56537
IAST [baseline] (20.665 ms) : 0, 20665
IAST [candidate] (22.426 ms) : 0, 22426
Remote Config [baseline] (595.953 µs) : 0, 596
Remote Config [candidate] (604.662 µs) : 0, 605
Telemetry [baseline] (7.416 ms) : 0, 7416
Telemetry [candidate] (7.471 ms) : 0, 7471
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (807.763 ms) : 0, 807763
BytebuddyAgent [candidate] (807.508 ms) : 0, 807508
GlobalTracer [baseline] (305.954 ms) : 0, 305954
GlobalTracer [candidate] (306.455 ms) : 0, 306455
AppSec [baseline] (58.438 ms) : 0, 58438
AppSec [candidate] (57.803 ms) : 0, 57803
IAST [baseline] (20.85 ms) : 0, 20850
IAST [candidate] (20.357 ms) : 0, 20357
Remote Config [baseline] (641.76 µs) : 0, 642
Remote Config [candidate] (630.538 µs) : 0, 631
Telemetry [baseline] (7.46 ms) : 0, 7460
Telemetry [candidate] (7.386 ms) : 0, 7386
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-24T23:45:47 2024-11-24T23:52:43
git_branch master mcculls/kafka-client-muzzle-fix
git_commit_date 1732489991 1732491285
git_commit_sha 4925a50 b5db502
release_version 1.43.0-SNAPSHOT~4925a50233 1.43.0-SNAPSHOT~b5db5029f1
start_time 2024-11-24T23:45:34 2024-11-24T23:52:30
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1732492715 1732492715
ci_job_id 718268142 718268142
ci_pipeline_id 49818571 49818571
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.35 ms) : 1330, 1371
.   : milestone, 1350,
appsec (1.758 ms) : 1733, 1784
.   : milestone, 1758,
appsec_no_iast (1.755 ms) : 1731, 1779
.   : milestone, 1755,
iast (1.494 ms) : 1472, 1517
.   : milestone, 1494,
profiling (1.488 ms) : 1465, 1512
.   : milestone, 1488,
tracing (1.507 ms) : 1483, 1532
.   : milestone, 1507,
section candidate
no_agent (1.342 ms) : 1322, 1361
.   : milestone, 1342,
appsec (1.76 ms) : 1737, 1783
.   : milestone, 1760,
appsec_no_iast (1.712 ms) : 1687, 1737
.   : milestone, 1712,
iast (1.499 ms) : 1476, 1522
.   : milestone, 1499,
profiling (1.534 ms) : 1510, 1557
.   : milestone, 1534,
tracing (1.472 ms) : 1447, 1497
.   : milestone, 1472,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.33 ms, 1.371 ms] -
appsec 1.758 ms [1.733 ms, 1.784 ms] 407.954 µs (30.2%)
appsec_no_iast 1.755 ms [1.731 ms, 1.779 ms] 404.252 µs (29.9%)
iast 1.494 ms [1.472 ms, 1.517 ms] 143.837 µs (10.7%)
profiling 1.488 ms [1.465 ms, 1.512 ms] 137.922 µs (10.2%)
tracing 1.507 ms [1.483 ms, 1.532 ms] 156.848 µs (11.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.322 ms, 1.361 ms] -
appsec 1.76 ms [1.737 ms, 1.783 ms] 418.64 µs (31.2%)
appsec_no_iast 1.712 ms [1.687 ms, 1.737 ms] 370.222 µs (27.6%)
iast 1.499 ms [1.476 ms, 1.522 ms] 157.771 µs (11.8%)
profiling 1.534 ms [1.51 ms, 1.557 ms] 192.006 µs (14.3%)
tracing 1.472 ms [1.447 ms, 1.497 ms] 129.99 µs (9.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233
    dateFormat X
    axisFormat %s
section baseline
no_agent (376.078 µs) : 355, 397
.   : milestone, 376,
iast (489.384 µs) : 468, 511
.   : milestone, 489,
iast_FULL (650.914 µs) : 629, 673
.   : milestone, 651,
iast_GLOBAL (519.918 µs) : 498, 542
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (489.419 µs) : 468, 511
.   : milestone, 489,
iast_INACTIVE (444.356 µs) : 424, 465
.   : milestone, 444,
iast_TELEMETRY_OFF (470.094 µs) : 449, 491
.   : milestone, 470,
tracing (449.414 µs) : 428, 471
.   : milestone, 449,
section candidate
no_agent (367.391 µs) : 347, 388
.   : milestone, 367,
iast (481.686 µs) : 461, 503
.   : milestone, 482,
iast_FULL (648.827 µs) : 627, 670
.   : milestone, 649,
iast_GLOBAL (520.993 µs) : 498, 544
.   : milestone, 521,
iast_HARDCODED_SECRET_DISABLED (482.576 µs) : 461, 504
.   : milestone, 483,
iast_INACTIVE (444.661 µs) : 424, 465
.   : milestone, 445,
iast_TELEMETRY_OFF (476.408 µs) : 455, 498
.   : milestone, 476,
tracing (443.061 µs) : 423, 463
.   : milestone, 443,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 376.078 µs [355.444 µs, 396.713 µs] -
iast 489.384 µs [467.873 µs, 510.894 µs] 113.305 µs (30.1%)
iast_FULL 650.914 µs [629.063 µs, 672.764 µs] 274.835 µs (73.1%)
iast_GLOBAL 519.918 µs [497.97 µs, 541.867 µs] 143.84 µs (38.2%)
iast_HARDCODED_SECRET_DISABLED 489.419 µs [467.894 µs, 510.944 µs] 113.34 µs (30.1%)
iast_INACTIVE 444.356 µs [423.735 µs, 464.977 µs] 68.278 µs (18.2%)
iast_TELEMETRY_OFF 470.094 µs [448.907 µs, 491.282 µs] 94.016 µs (25.0%)
tracing 449.414 µs [427.964 µs, 470.864 µs] 73.336 µs (19.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.391 µs [346.876 µs, 387.905 µs] -
iast 481.686 µs [460.598 µs, 502.775 µs] 114.296 µs (31.1%)
iast_FULL 648.827 µs [627.371 µs, 670.283 µs] 281.436 µs (76.6%)
iast_GLOBAL 520.993 µs [497.99 µs, 543.997 µs] 153.602 µs (41.8%)
iast_HARDCODED_SECRET_DISABLED 482.576 µs [461.301 µs, 503.85 µs] 115.185 µs (31.4%)
iast_INACTIVE 444.661 µs [423.875 µs, 465.448 µs] 77.27 µs (21.0%)
iast_TELEMETRY_OFF 476.408 µs [454.91 µs, 497.906 µs] 109.017 µs (29.7%)
tracing 443.061 µs [422.628 µs, 463.495 µs] 75.671 µs (20.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/kafka-client-muzzle-fix
git_commit_date 1732489991 1732491285
git_commit_sha 4925a50 b5db502
release_version 1.43.0-SNAPSHOT~4925a50233 1.43.0-SNAPSHOT~b5db5029f1
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1732493364 1732493364
ci_job_id 718268136 718268136
ci_pipeline_id 49818571 49818571
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.46 ms) : 1449, 1472
.   : milestone, 1460,
appsec (2.324 ms) : 2283, 2365
.   : milestone, 2324,
iast (2.073 ms) : 2021, 2125
.   : milestone, 2073,
iast_GLOBAL (2.114 ms) : 2062, 2166
.   : milestone, 2114,
profiling (1.959 ms) : 1916, 2002
.   : milestone, 1959,
tracing (1.908 ms) : 1868, 1948
.   : milestone, 1908,
section candidate
no_agent (1.464 ms) : 1452, 1475
.   : milestone, 1464,
appsec (2.329 ms) : 2288, 2370
.   : milestone, 2329,
iast (2.071 ms) : 2019, 2124
.   : milestone, 2071,
iast_GLOBAL (2.114 ms) : 2062, 2167
.   : milestone, 2114,
profiling (1.935 ms) : 1893, 1977
.   : milestone, 1935,
tracing (1.915 ms) : 1876, 1955
.   : milestone, 1915,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.46 ms [1.449 ms, 1.472 ms] -
appsec 2.324 ms [2.283 ms, 2.365 ms] 863.96 µs (59.2%)
iast 2.073 ms [2.021 ms, 2.125 ms] 612.96 µs (42.0%)
iast_GLOBAL 2.114 ms [2.062 ms, 2.166 ms] 653.989 µs (44.8%)
profiling 1.959 ms [1.916 ms, 2.002 ms] 499.095 µs (34.2%)
tracing 1.908 ms [1.868 ms, 1.948 ms] 447.999 µs (30.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.464 ms [1.452 ms, 1.475 ms] -
appsec 2.329 ms [2.288 ms, 2.37 ms] 865.163 µs (59.1%)
iast 2.071 ms [2.019 ms, 2.124 ms] 607.759 µs (41.5%)
iast_GLOBAL 2.114 ms [2.062 ms, 2.167 ms] 650.463 µs (44.4%)
profiling 1.935 ms [1.893 ms, 1.977 ms] 471.356 µs (32.2%)
tracing 1.915 ms [1.876 ms, 1.955 ms] 451.734 µs (30.9%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~b5db5029f1, baseline=1.43.0-SNAPSHOT~4925a50233
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.934 s) : 14934000, 14934000
.   : milestone, 14934000,
appsec (14.639 s) : 14639000, 14639000
.   : milestone, 14639000,
iast (18.737 s) : 18737000, 18737000
.   : milestone, 18737000,
iast_GLOBAL (18.185 s) : 18185000, 18185000
.   : milestone, 18185000,
profiling (15.354 s) : 15354000, 15354000
.   : milestone, 15354000,
tracing (15.026 s) : 15026000, 15026000
.   : milestone, 15026000,
section candidate
no_agent (15.016 s) : 15016000, 15016000
.   : milestone, 15016000,
appsec (14.984 s) : 14984000, 14984000
.   : milestone, 14984000,
iast (19.415 s) : 19415000, 19415000
.   : milestone, 19415000,
iast_GLOBAL (17.993 s) : 17993000, 17993000
.   : milestone, 17993000,
profiling (15.068 s) : 15068000, 15068000
.   : milestone, 15068000,
tracing (14.773 s) : 14773000, 14773000
.   : milestone, 14773000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.934 s [14.934 s, 14.934 s] -
appsec 14.639 s [14.639 s, 14.639 s] -295.0 ms (-2.0%)
iast 18.737 s [18.737 s, 18.737 s] 3.803 s (25.5%)
iast_GLOBAL 18.185 s [18.185 s, 18.185 s] 3.251 s (21.8%)
profiling 15.354 s [15.354 s, 15.354 s] 420.0 ms (2.8%)
tracing 15.026 s [15.026 s, 15.026 s] 92.0 ms (0.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.016 s [15.016 s, 15.016 s] -
appsec 14.984 s [14.984 s, 14.984 s] -32.0 ms (-0.2%)
iast 19.415 s [19.415 s, 19.415 s] 4.399 s (29.3%)
iast_GLOBAL 17.993 s [17.993 s, 17.993 s] 2.977 s (19.8%)
profiling 15.068 s [15.068 s, 15.068 s] 52.0 ms (0.3%)
tracing 14.773 s [14.773 s, 14.773 s] -243.0 ms (-1.6%)

@pr-commenter
Copy link

pr-commenter bot commented Nov 25, 2024 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/kafka-client-muzzle-fix
git_commit_date 1732489991 1732491285
git_commit_sha 4925a50 b5db502
See matching parameters
Baseline Candidate
ci_job_date 1732494229 1732494229
ci_job_id 718273073 718273073
ci_pipeline_id 49818571 49818571
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.25 11.0.25
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.25+9-post-Ubuntu-1ubuntu122.04 11.0.25+9-post-Ubuntu-1ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@mcculls mcculls marked this pull request as ready for review November 25, 2024 00:40
@mcculls mcculls requested a review from a team as a code owner November 25, 2024 00:40
@nayeem-kamal nayeem-kamal mentioned this pull request Nov 25, 2024
Merged
5 tasks
@amarziali amarziali mentioned this pull request Nov 25, 2024
Closed
5 tasks
amarziali
amarziali approved these changes Nov 25, 2024
View reviewed changes
Copy link
Collaborator

@amarziali amarziali left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

@mcculls mcculls merged commit b36006c into master Nov 25, 2024
105 checks passed
@mcculls mcculls deleted the mcculls/kafka-client-muzzle-fix branch November 25, 2024 09:29
@github-actions github-actions bot added this to the 1.43.0 milestone Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@amarziali amarziali amarziali approved these changes

Assignees
No one assigned
Labels
inst: kafka Kafka instrumentation type: bug
Projects
None yet
Milestone
1.43.0
Development

Successfully merging this pull request may close these issues.
2 participants
@mcculls @amarziali