Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Kafka lag instrumentation for version 2.7 of Kafka #7941

Merged
merged 9 commits into from
Nov 14, 2024
Merged

Fix Kafka lag instrumentation for version 2.7 of Kafka #7941

merged 9 commits into from
Nov 14, 2024

Conversation

piochelepiotr
Copy link
Contributor

@piochelepiotr piochelepiotr commented Nov 12, 2024 •
edited
Loading

What Does This Do

Instruments the new constructor for KafkaConsumer in version 2.7.
Before that, we failed to instrument it, and though were not able to capture Kafka lag or Kafka cluster ID for this version.

Motivation

Additional Notes

Also, fixing latest dependency tests that were marked as flaky a while ago. I made them simpler, as it's better to have easy tests enabled, rather than complex one that are disabled (these tests should have caught the issue when Kafka published the 2.7 version).

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@pr-commenter
Copy link

pr-commenter bot commented Nov 12, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master piotr-wolski/fix-kafka-instrumentation-27
git_commit_date 1731594304 1731600113
git_commit_sha ea008a2 06e9209
release_version 1.43.0-SNAPSHOT~ea008a2aad 1.43.0-SNAPSHOT~06e92093a6
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1731602412 1731602412
ci_job_id 707247066 707247066
ci_pipeline_id 49030319 49030319
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 9 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.083 s) : 0, 1083290
Total [baseline] (8.631 s) : 0, 8631331
Agent [candidate] (1.082 s) : 0, 1081523
Total [candidate] (8.57 s) : 0, 8570423
section iast
Agent [baseline] (1.206 s) : 0, 1206388
Total [baseline] (9.141 s) : 0, 9141196
Agent [candidate] (1.217 s) : 0, 1217329
Total [candidate] (9.173 s) : 0, 9173104
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.205 s) : 0, 1205405
Total [baseline] (9.106 s) : 0, 9106427
Agent [candidate] (1.209 s) : 0, 1208628
Total [candidate] (9.144 s) : 0, 9143564
section iast_TELEMETRY_OFF
Agent [baseline] (1.206 s) : 0, 1206207
Total [baseline] (9.138 s) : 0, 9137653
Agent [candidate] (1.206 s) : 0, 1205619
Total [candidate] (9.191 s) : 0, 9191074
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent iast 1.206 s 123.098 ms (11.4%)
Agent iast_HARDCODED_SECRET_DISABLED 1.205 s 122.115 ms (11.3%)
Agent iast_TELEMETRY_OFF 1.206 s 122.917 ms (11.3%)
Total tracing 8.631 s -
Total iast 9.141 s 509.865 ms (5.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.106 s 475.096 ms (5.5%)
Total iast_TELEMETRY_OFF 9.138 s 506.322 ms (5.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent iast 1.217 s 135.807 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.209 s 127.105 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.206 s 124.097 ms (11.5%)
Total tracing 8.57 s -
Total iast 9.173 s 602.681 ms (7.0%)
Total iast_HARDCODED_SECRET_DISABLED 9.144 s 573.141 ms (6.7%)
Total iast_TELEMETRY_OFF 9.191 s 620.651 ms (7.2%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (688.022 ms) : 0, 688022
BytebuddyAgent [candidate] (687.697 ms) : 0, 687697
GlobalTracer [baseline] (316.291 ms) : 0, 316291
GlobalTracer [candidate] (315.84 ms) : 0, 315840
AppSec [baseline] (54.539 ms) : 0, 54539
AppSec [candidate] (54.341 ms) : 0, 54341
Remote Config [baseline] (691.589 µs) : 0, 692
Remote Config [candidate] (674.511 µs) : 0, 675
Telemetry [baseline] (9.991 ms) : 0, 9991
Telemetry [candidate] (9.231 ms) : 0, 9231
section iast
BytebuddyAgent [baseline] (802.403 ms) : 0, 802403
BytebuddyAgent [candidate] (809.692 ms) : 0, 809692
GlobalTracer [baseline] (304.338 ms) : 0, 304338
GlobalTracer [candidate] (307.444 ms) : 0, 307444
AppSec [baseline] (57.315 ms) : 0, 57315
AppSec [candidate] (57.892 ms) : 0, 57892
IAST [baseline] (20.536 ms) : 0, 20536
IAST [candidate] (20.451 ms) : 0, 20451
Remote Config [baseline] (620.273 µs) : 0, 620
Remote Config [candidate] (607.246 µs) : 0, 607
Telemetry [baseline] (7.442 ms) : 0, 7442
Telemetry [candidate] (7.37 ms) : 0, 7370
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.772 ms) : 0, 800772
BytebuddyAgent [candidate] (803.036 ms) : 0, 803036
GlobalTracer [baseline] (304.411 ms) : 0, 304411
GlobalTracer [candidate] (305.909 ms) : 0, 305909
AppSec [baseline] (56.041 ms) : 0, 56041
AppSec [candidate] (57.305 ms) : 0, 57305
IAST [baseline] (22.317 ms) : 0, 22317
IAST [candidate] (20.558 ms) : 0, 20558
Remote Config [baseline] (634.845 µs) : 0, 635
Remote Config [candidate] (640.234 µs) : 0, 640
Telemetry [baseline] (7.508 ms) : 0, 7508
Telemetry [candidate] (7.446 ms) : 0, 7446
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.861 ms) : 0, 800861
BytebuddyAgent [candidate] (800.506 ms) : 0, 800506
GlobalTracer [baseline] (305.509 ms) : 0, 305509
GlobalTracer [candidate] (304.924 ms) : 0, 304924
AppSec [baseline] (57.894 ms) : 0, 57894
AppSec [candidate] (57.328 ms) : 0, 57328
IAST [baseline] (20.192 ms) : 0, 20192
IAST [candidate] (21.122 ms) : 0, 21122
Remote Config [baseline] (611.407 µs) : 0, 611
Remote Config [candidate] (618.894 µs) : 0, 619
Telemetry [baseline] (7.371 ms) : 0, 7371
Telemetry [candidate] (7.381 ms) : 0, 7381
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.09 s) : 0, 1090495
Total [baseline] (10.467 s) : 0, 10467002
Agent [candidate] (1.081 s) : 0, 1080679
Total [candidate] (10.41 s) : 0, 10409732
section appsec
Agent [baseline] (1.221 s) : 0, 1221224
Total [baseline] (10.7 s) : 0, 10700092
Agent [candidate] (1.223 s) : 0, 1223444
Total [candidate] (10.672 s) : 0, 10671929
section iast
Agent [baseline] (1.208 s) : 0, 1208197
Total [baseline] (10.917 s) : 0, 10917461
Agent [candidate] (1.211 s) : 0, 1210905
Total [candidate] (11.011 s) : 0, 11011281
section profiling
Agent [baseline] (1.284 s) : 0, 1283696
Total [baseline] (10.802 s) : 0, 10802055
Agent [candidate] (1.289 s) : 0, 1288788
Total [candidate] (10.776 s) : 0, 10776392
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.09 s -
Agent appsec 1.221 s 130.729 ms (12.0%)
Agent iast 1.208 s 117.702 ms (10.8%)
Agent profiling 1.284 s 193.201 ms (17.7%)
Total tracing 10.467 s -
Total appsec 10.7 s 233.09 ms (2.2%)
Total iast 10.917 s 450.459 ms (4.3%)
Total profiling 10.802 s 335.054 ms (3.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent appsec 1.223 s 142.765 ms (13.2%)
Agent iast 1.211 s 130.225 ms (12.1%)
Agent profiling 1.289 s 208.108 ms (19.3%)
Total tracing 10.41 s -
Total appsec 10.672 s 262.197 ms (2.5%)
Total iast 11.011 s 601.549 ms (5.8%)
Total profiling 10.776 s 366.66 ms (3.5%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (692.574 ms) : 0, 692574
BytebuddyAgent [candidate] (687.091 ms) : 0, 687091
GlobalTracer [baseline] (318.199 ms) : 0, 318199
GlobalTracer [candidate] (316.379 ms) : 0, 316379
AppSec [baseline] (54.459 ms) : 0, 54459
AppSec [candidate] (54.285 ms) : 0, 54285
Remote Config [baseline] (679.263 µs) : 0, 679
Remote Config [candidate] (674.617 µs) : 0, 675
Telemetry [baseline] (10.751 ms) : 0, 10751
Telemetry [candidate] (8.51 ms) : 0, 8510
section appsec
BytebuddyAgent [baseline] (706.47 ms) : 0, 706470
BytebuddyAgent [candidate] (709.646 ms) : 0, 709646
GlobalTracer [baseline] (316.195 ms) : 0, 316195
GlobalTracer [candidate] (315.568 ms) : 0, 315568
AppSec [baseline] (165.937 ms) : 0, 165937
AppSec [candidate] (165.152 ms) : 0, 165152
IAST [baseline] (20.759 ms) : 0, 20759
IAST [candidate] (20.14 ms) : 0, 20140
Remote Config [baseline] (641.725 µs) : 0, 642
Remote Config [candidate] (648.098 µs) : 0, 648
Telemetry [baseline] (7.86 ms) : 0, 7860
Telemetry [candidate] (8.548 ms) : 0, 8548
section iast
BytebuddyAgent [baseline] (803.433 ms) : 0, 803433
BytebuddyAgent [candidate] (804.402 ms) : 0, 804402
GlobalTracer [baseline] (305.328 ms) : 0, 305328
GlobalTracer [candidate] (305.582 ms) : 0, 305582
AppSec [baseline] (56.257 ms) : 0, 56257
AppSec [candidate] (57.455 ms) : 0, 57455
IAST [baseline] (21.418 ms) : 0, 21418
IAST [candidate] (21.522 ms) : 0, 21522
Remote Config [baseline] (610.343 µs) : 0, 610
Remote Config [candidate] (615.374 µs) : 0, 615
Telemetry [baseline] (7.424 ms) : 0, 7424
Telemetry [candidate] (7.587 ms) : 0, 7587
section profiling
BytebuddyAgent [baseline] (685.411 ms) : 0, 685411
BytebuddyAgent [candidate] (686.256 ms) : 0, 686256
GlobalTracer [baseline] (398.86 ms) : 0, 398860
GlobalTracer [candidate] (402.025 ms) : 0, 402025
AppSec [baseline] (54.692 ms) : 0, 54692
AppSec [candidate] (55.098 ms) : 0, 55098
Remote Config [baseline] (663.547 µs) : 0, 664
Remote Config [candidate] (679.12 µs) : 0, 679
Telemetry [baseline] (12.747 ms) : 0, 12747
Telemetry [candidate] (10.662 ms) : 0, 10662
ProfilingAgent [baseline] (92.13 ms) : 0, 92130
ProfilingAgent [candidate] (94.791 ms) : 0, 94791
Profiling [baseline] (92.154 ms) : 0, 92154
Profiling [candidate] (94.815 ms) : 0, 94815
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-14T16:12:35 2024-11-14T16:19:30
git_branch master piotr-wolski/fix-kafka-instrumentation-27
git_commit_date 1731594304 1731600113
git_commit_sha ea008a2 06e9209
release_version 1.43.0-SNAPSHOT~ea008a2aad 1.43.0-SNAPSHOT~06e92093a6
start_time 2024-11-14T16:12:21 2024-11-14T16:19:17
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1731601520 1731601520
ci_job_id 707247067 707247067
ci_pipeline_id 49030319 49030319
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad
    dateFormat X
    axisFormat %s
section baseline
no_agent (373.23 µs) : 353, 393
.   : milestone, 373,
iast (500.079 µs) : 478, 522
.   : milestone, 500,
iast_FULL (646.361 µs) : 625, 668
.   : milestone, 646,
iast_GLOBAL (517.923 µs) : 496, 540
.   : milestone, 518,
iast_HARDCODED_SECRET_DISABLED (491.44 µs) : 470, 513
.   : milestone, 491,
iast_INACTIVE (450.956 µs) : 430, 472
.   : milestone, 451,
iast_TELEMETRY_OFF (470.289 µs) : 449, 491
.   : milestone, 470,
tracing (446.313 µs) : 425, 467
.   : milestone, 446,
section candidate
no_agent (371.889 µs) : 351, 393
.   : milestone, 372,
iast (481.968 µs) : 460, 503
.   : milestone, 482,
iast_FULL (643.034 µs) : 622, 664
.   : milestone, 643,
iast_GLOBAL (517.892 µs) : 495, 541
.   : milestone, 518,
iast_HARDCODED_SECRET_DISABLED (481.736 µs) : 461, 503
.   : milestone, 482,
iast_INACTIVE (449.232 µs) : 428, 470
.   : milestone, 449,
iast_TELEMETRY_OFF (475.028 µs) : 454, 497
.   : milestone, 475,
tracing (446.244 µs) : 424, 468
.   : milestone, 446,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 373.23 µs [353.353 µs, 393.106 µs] -
iast 500.079 µs [477.888 µs, 522.271 µs] 126.85 µs (34.0%)
iast_FULL 646.361 µs [624.874 µs, 667.848 µs] 273.131 µs (73.2%)
iast_GLOBAL 517.923 µs [495.781 µs, 540.064 µs] 144.693 µs (38.8%)
iast_HARDCODED_SECRET_DISABLED 491.44 µs [470.024 µs, 512.856 µs] 118.211 µs (31.7%)
iast_INACTIVE 450.956 µs [430.094 µs, 471.818 µs] 77.726 µs (20.8%)
iast_TELEMETRY_OFF 470.289 µs [449.363 µs, 491.215 µs] 97.06 µs (26.0%)
tracing 446.313 µs [425.187 µs, 467.44 µs] 73.084 µs (19.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 371.889 µs [351.249 µs, 392.53 µs] -
iast 481.968 µs [460.457 µs, 503.479 µs] 110.079 µs (29.6%)
iast_FULL 643.034 µs [621.692 µs, 664.377 µs] 271.145 µs (72.9%)
iast_GLOBAL 517.892 µs [495.278 µs, 540.505 µs] 146.002 µs (39.3%)
iast_HARDCODED_SECRET_DISABLED 481.736 µs [460.614 µs, 502.858 µs] 109.846 µs (29.5%)
iast_INACTIVE 449.232 µs [428.24 µs, 470.223 µs] 77.342 µs (20.8%)
iast_TELEMETRY_OFF 475.028 µs [453.541 µs, 496.515 µs] 103.138 µs (27.7%)
tracing 446.244 µs [424.381 µs, 468.108 µs] 74.355 µs (20.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1321, 1361
.   : milestone, 1341,
appsec (1.738 ms) : 1714, 1763
.   : milestone, 1738,
appsec_no_iast (1.716 ms) : 1691, 1741
.   : milestone, 1716,
iast (1.484 ms) : 1462, 1505
.   : milestone, 1484,
profiling (1.534 ms) : 1510, 1558
.   : milestone, 1534,
tracing (1.478 ms) : 1454, 1502
.   : milestone, 1478,
section candidate
no_agent (1.342 ms) : 1321, 1362
.   : milestone, 1342,
appsec (1.725 ms) : 1701, 1750
.   : milestone, 1725,
appsec_no_iast (1.749 ms) : 1725, 1773
.   : milestone, 1749,
iast (1.488 ms) : 1467, 1510
.   : milestone, 1488,
profiling (1.487 ms) : 1464, 1510
.   : milestone, 1487,
tracing (1.473 ms) : 1446, 1501
.   : milestone, 1473,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.321 ms, 1.361 ms] -
appsec 1.738 ms [1.714 ms, 1.763 ms] 397.522 µs (29.6%)
appsec_no_iast 1.716 ms [1.691 ms, 1.741 ms] 375.177 µs (28.0%)
iast 1.484 ms [1.462 ms, 1.505 ms] 142.706 µs (10.6%)
profiling 1.534 ms [1.51 ms, 1.558 ms] 193.217 µs (14.4%)
tracing 1.478 ms [1.454 ms, 1.502 ms] 136.824 µs (10.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.321 ms, 1.362 ms] -
appsec 1.725 ms [1.701 ms, 1.75 ms] 383.886 µs (28.6%)
appsec_no_iast 1.749 ms [1.725 ms, 1.773 ms] 407.151 µs (30.3%)
iast 1.488 ms [1.467 ms, 1.51 ms] 146.724 µs (10.9%)
profiling 1.487 ms [1.464 ms, 1.51 ms] 145.595 µs (10.9%)
tracing 1.473 ms [1.446 ms, 1.501 ms] 131.745 µs (9.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master piotr-wolski/fix-kafka-instrumentation-27
git_commit_date 1731594304 1731600113
git_commit_sha ea008a2 06e9209
release_version 1.43.0-SNAPSHOT~ea008a2aad 1.43.0-SNAPSHOT~06e92093a6
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1731601923 1731601923
ci_job_id 707247068 707247068
ci_pipeline_id 49030319 49030319
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.341 ms) : 2300, 2382
.   : milestone, 2341,
iast (2.085 ms) : 2033, 2137
.   : milestone, 2085,
iast_GLOBAL (2.122 ms) : 2070, 2175
.   : milestone, 2122,
profiling (1.948 ms) : 1906, 1990
.   : milestone, 1948,
tracing (1.925 ms) : 1885, 1965
.   : milestone, 1925,
section candidate
no_agent (1.467 ms) : 1456, 1479
.   : milestone, 1467,
appsec (2.356 ms) : 2314, 2397
.   : milestone, 2356,
iast (2.077 ms) : 2025, 2129
.   : milestone, 2077,
iast_GLOBAL (2.114 ms) : 2062, 2165
.   : milestone, 2114,
profiling (1.936 ms) : 1893, 1978
.   : milestone, 1936,
tracing (1.929 ms) : 1889, 1969
.   : milestone, 1929,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.341 ms [2.3 ms, 2.382 ms] 870.673 µs (59.2%)
iast 2.085 ms [2.033 ms, 2.137 ms] 614.855 µs (41.8%)
iast_GLOBAL 2.122 ms [2.07 ms, 2.175 ms] 652.32 µs (44.4%)
profiling 1.948 ms [1.906 ms, 1.99 ms] 478.117 µs (32.5%)
tracing 1.925 ms [1.885 ms, 1.965 ms] 455.122 µs (31.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.467 ms [1.456 ms, 1.479 ms] -
appsec 2.356 ms [2.314 ms, 2.397 ms] 888.608 µs (60.6%)
iast 2.077 ms [2.025 ms, 2.129 ms] 610.153 µs (41.6%)
iast_GLOBAL 2.114 ms [2.062 ms, 2.165 ms] 646.401 µs (44.1%)
profiling 1.936 ms [1.893 ms, 1.978 ms] 468.626 µs (31.9%)
tracing 1.929 ms [1.889 ms, 1.969 ms] 461.855 µs (31.5%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~06e92093a6, baseline=1.43.0-SNAPSHOT~ea008a2aad
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.473 s) : 15473000, 15473000
.   : milestone, 15473000,
appsec (15.065 s) : 15065000, 15065000
.   : milestone, 15065000,
iast (18.451 s) : 18451000, 18451000
.   : milestone, 18451000,
iast_GLOBAL (18.535 s) : 18535000, 18535000
.   : milestone, 18535000,
profiling (15.179 s) : 15179000, 15179000
.   : milestone, 15179000,
tracing (14.892 s) : 14892000, 14892000
.   : milestone, 14892000,
section candidate
no_agent (15.526 s) : 15526000, 15526000
.   : milestone, 15526000,
appsec (15.272 s) : 15272000, 15272000
.   : milestone, 15272000,
iast (18.819 s) : 18819000, 18819000
.   : milestone, 18819000,
iast_GLOBAL (17.952 s) : 17952000, 17952000
.   : milestone, 17952000,
profiling (15.125 s) : 15125000, 15125000
.   : milestone, 15125000,
tracing (15.291 s) : 15291000, 15291000
.   : milestone, 15291000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.473 s [15.473 s, 15.473 s] -
appsec 15.065 s [15.065 s, 15.065 s] -408.0 ms (-2.6%)
iast 18.451 s [18.451 s, 18.451 s] 2.978 s (19.2%)
iast_GLOBAL 18.535 s [18.535 s, 18.535 s] 3.062 s (19.8%)
profiling 15.179 s [15.179 s, 15.179 s] -294.0 ms (-1.9%)
tracing 14.892 s [14.892 s, 14.892 s] -581.0 ms (-3.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.526 s [15.526 s, 15.526 s] -
appsec 15.272 s [15.272 s, 15.272 s] -254.0 ms (-1.6%)
iast 18.819 s [18.819 s, 18.819 s] 3.293 s (21.2%)
iast_GLOBAL 17.952 s [17.952 s, 17.952 s] 2.426 s (15.6%)
profiling 15.125 s [15.125 s, 15.125 s] -401.0 ms (-2.6%)
tracing 15.291 s [15.291 s, 15.291 s] -235.0 ms (-1.5%)

@pr-commenter
Copy link

pr-commenter bot commented Nov 12, 2024 •
edited
Loading

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master piotr-wolski/fix-kafka-instrumentation-27
git_commit_date 1731594304 1731600113
git_commit_sha ea008a2 06e9209
See matching parameters
Baseline Candidate
ci_job_date 1731601153 1731601153
ci_job_id 707247070 707247070
ci_pipeline_id 49030319 49030319
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.21 11.0.21
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/producer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.21+9-post-Ubuntu-0ubuntu122.04 11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter
Copy link

pr-commenter bot commented Nov 12, 2024 •
edited
Loading

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master piotr-wolski/fix-kafka-instrumentation-27
git_commit_date 1731594304 1731600113
git_commit_sha ea008a2 06e9209
See matching parameters
Baseline Candidate
ci_job_date 1731601194 1731601194
ci_job_id 707247071 707247071
ci_pipeline_id 49030319 49030319
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.21 11.0.21
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/go/src/github.com/DataDog/apm-reliability/dd-trace-java/platform/src/consumer-benchmark/build/tmp/jmh -Duser.country=US -Duser.language=en -Duser.variant
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.21+9-post-Ubuntu-0ubuntu122.04 11.0.21+9-post-Ubuntu-0ubuntu122.04

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

@piochelepiotr piochelepiotr force-pushed the piotr-wolski/fix-kafka-instrumentation-27 branch from 27a4688 to e3e573d Compare November 13, 2024 15:46
@piochelepiotr piochelepiotr marked this pull request as ready for review November 13, 2024 15:46
@piochelepiotr piochelepiotr requested a review from a team as a code owner November 13, 2024 15:46
@piochelepiotr piochelepiotr changed the title Fix Kafka lag instrumentation for version 2.7+ of Kafka Fix Kafka lag instrumentation for version 2.7 of Kafka Nov 13, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 13, 2024 •
edited
Loading

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

@piochelepiotr piochelepiotr added inst: kafka Kafka instrumentation type: bug labels Nov 13, 2024
nayeem-kamal
nayeem-kamal reviewed Nov 13, 2024
View reviewed changes
dd-java-agent/instrumentation/kafka-clients-0.11/build.gradle
@@ -38,11 +38,8 @@ dependencies {
testImplementation(group: 'com.fasterxml.jackson.core', name: 'jackson-databind', version: '2.9.10')

// Include latest version of kafka itself along with latest version of client libs.
// This seems to help with jar compatibility hell.
latestDepTestImplementation group: 'org.apache.kafka', name: 'kafka_2.13', version: '2.+'
Copy link
Contributor

@nayeem-kamal nayeem-kamal Nov 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand, why are we removing these? Was this just always flaky?

Copy link
Contributor Author

@piochelepiotr piochelepiotr Nov 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were requiring different versions of kafka-clients. So it was not testing with the versions listed bellow.
Now, the versions it's testing for are more clearly defined.

dd-java-agent/instrumentation/kafka-clients-0.11/build.gradle
latestDepTestImplementation group: 'org.apache.kafka', name: 'kafka-clients', version: '2.+'
// latest depending to kafka client 2.x -> to be fixed when this instrumentation will test 3.x as well
latestDepTestImplementation group: 'io.projectreactor.kafka', name: 'reactor-kafka', version: '1.3.21'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we plan on bringing the reactor tests back?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know, they have been disabled for 2 years as far as I understand, so I prefer to have a simple test case on, rather than having a bunch of "tests" that are not actually running.

@piochelepiotr piochelepiotr merged commit f19e110 into master Nov 14, 2024
104 checks passed
@piochelepiotr piochelepiotr deleted the piotr-wolski/fix-kafka-instrumentation-27 branch November 14, 2024 17:48
@github-actions github-actions bot added this to the 1.43.0 milestone Nov 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nayeem-kamal nayeem-kamal nayeem-kamal approved these changes

Assignees
No one assigned
Labels
inst: kafka Kafka instrumentation type: bug
Projects
None yet
Milestone
1.43.0
Development

Successfully merging this pull request may close these issues.
2 participants
@piochelepiotr @nayeem-kamal