Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Exception Replay config parameters #7647

Merged
merged 1 commit into from
Sep 24, 2024
Merged

Add Exception Replay config parameters #7647

merged 1 commit into from
Sep 24, 2024

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Sep 19, 2024 •
edited by jira bot
Loading

What Does This Do

  • DD_EXCEPTION_REPLAY_CAPTURE_MAX_FRAMES alias onDD_EXCEPTION_REPLAY_MAX_FRAMES_TO_CAPTURE
  • DD_EXCEPTION_REPLAY_CAPTURE_INTERVAL_SECONDS interval in seconds between 2 capture of the same exceptions
  • DD_EXCEPTION_REPLAY_CAPTURE_INTERMEDIATE_SPANS_ENABLED inverted alias on DD_EXCEPTION_REPLAY_ONLY_LOCAL_ROOT

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-2882

@jpbempel
Add Exception Replay config parameters
f485063 
DD_EXCEPTION_REPLAY_CAPTURE_MAX_FRAMES alias on
DD_EXCEPTION_REPLAY_MAX_FRAMES_TO_CAPTURE
DD_EXCEPTION_REPLAY_CAPTURE_INTERVAL_SECONDS interval in seconds
between 2 capture of the same exceptions
DD_EXCEPTION_REPLAY_CAPTURE_INTERMEDIATE_SPANS_ENABLED inverted alias
on DD_EXCEPTION_REPLAY_ONLY_LOCAL_ROOT
@jpbempel jpbempel requested review from a team as code owners September 19, 2024 12:35
@jpbempel jpbempel requested review from evanchooly and ygree and removed request for a team September 19, 2024 12:35
@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Sep 19, 2024
@pr-commenter
Copy link

pr-commenter bot commented Sep 19, 2024

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1726749873 1726750248
end_time 2024-09-19T12:45:49 2024-09-19T12:52:04
git_branch master jpbempel/er-config-params
git_commit_sha a8ea730 f485063
start_time 2024-09-19T12:44:34 2024-09-19T12:50:49
See matching parameters
Baseline Candidate
ci_job_id 644818494 644818494
ci_pipeline_id 44684956 44684956
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1726749251 1726749251

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 4 metrics, 6 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+210.337µs; +298.714µs] or [+2.071%; +2.941%]		 worse
[+204.524µs; +308.186µs] or [+1.982%; +2.987%]		 worse
[+187.203µs; +316.693µs] or [+1.798%; +3.043%]		 worse
[+118.826µs; +471.504µs] or [+1.110%; +4.403%]		 worse
[-3.325op/s; -1.146op/s] or [-3.475%; -1.198%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-21.799µs; +25.095µs] or [-8.271%; +9.522%]		 unstable
[-35.460µs; +37.614µs] or [-11.659%; +12.367%]		 unstable
[-47.154µs; +51.241µs] or [-14.843%; +16.130%]		 unstable
[-85.612µs; +126.465µs] or [-14.344%; +21.189%]		 unstable
[-138.897op/s; +138.897op/s] or [-5.278%; +5.278%]
scenario:basic unsure
[+0.360µs; +11.655µs] or [+0.134%; +4.331%]		 same same unstable
[-16.523µs; +77.924µs] or [-3.086%; +14.554%]		 same
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (304.138 µs) : 272, 337
.   : milestone, 304,
basic (298.432 µs) : 288, 309
.   : milestone, 298,
loop (10.319 ms) : 10287, 10351
.   : milestone, 10319,
section candidate
noprobe (305.214 µs) : 268, 342
.   : milestone, 305,
basic (304.68 µs) : 296, 314
.   : milestone, 305,
loop (10.575 ms) : 10515, 10636
.   : milestone, 10575,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 304.138 µs [271.623 µs, 336.653 µs]
basic 298.432 µs [287.705 µs, 309.159 µs]
loop 10.319 ms [10.287 ms, 10.351 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 305.214 µs [268.197 µs, 342.231 µs]
basic 304.68 µs [295.643 µs, 313.717 µs]
loop 10.575 ms [10.515 ms, 10.636 ms]

@pr-commenter
Copy link

pr-commenter bot commented Sep 19, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/er-config-params
git_commit_date 1726743087 1726749251
git_commit_sha a8ea730 f485063
release_version 1.40.0-SNAPSHOT~a8ea730bf7 1.40.0-SNAPSHOT~f485063f6f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726751579 1726751579
ci_job_id 644818488 644818488
ci_pipeline_id 44684956 44684956
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 15 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:tracing:Telemetry better
[-424.636µs; -205.252µs] or [-5.512%; -2.664%]		 7.389ms 7.704ms
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1064203
Total [baseline] (10.315 s) : 0, 10315046
Agent [candidate] (1.07 s) : 0, 1069895
Total [candidate] (10.408 s) : 0, 10408494
section appsec
Agent [baseline] (1.204 s) : 0, 1203787
Total [baseline] (10.599 s) : 0, 10599140
Agent [candidate] (1.198 s) : 0, 1197763
Total [candidate] (10.61 s) : 0, 10610291
section iast
Agent [baseline] (1.19 s) : 0, 1190252
Total [baseline] (10.825 s) : 0, 10824972
Agent [candidate] (1.196 s) : 0, 1196058
Total [candidate] (10.772 s) : 0, 10772005
section profiling
Agent [baseline] (1.264 s) : 0, 1264262
Total [baseline] (10.541 s) : 0, 10541354
Agent [candidate] (1.263 s) : 0, 1263250
Total [candidate] (10.532 s) : 0, 10532465
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.204 s 139.583 ms (13.1%)
Agent iast 1.19 s 126.049 ms (11.8%)
Agent profiling 1.264 s 200.059 ms (18.8%)
Total tracing 10.315 s -
Total appsec 10.599 s 284.094 ms (2.8%)
Total iast 10.825 s 509.925 ms (4.9%)
Total profiling 10.541 s 226.307 ms (2.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.07 s -
Agent appsec 1.198 s 127.868 ms (12.0%)
Agent iast 1.196 s 126.163 ms (11.8%)
Agent profiling 1.263 s 193.355 ms (18.1%)
Total tracing 10.408 s -
Total appsec 10.61 s 201.798 ms (1.9%)
Total iast 10.772 s 363.511 ms (3.5%)
Total profiling 10.532 s 123.971 ms (1.2%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (679.037 ms) : 0, 679037
BytebuddyAgent [candidate] (683.602 ms) : 0, 683602
GlobalTracer [baseline] (309.89 ms) : 0, 309890
GlobalTracer [candidate] (311.216 ms) : 0, 311216
AppSec [baseline] (53.489 ms) : 0, 53489
AppSec [candidate] (53.143 ms) : 0, 53143
Remote Config [baseline] (668.441 µs) : 0, 668
Remote Config [candidate] (675.575 µs) : 0, 676
Telemetry [baseline] (7.528 ms) : 0, 7528
Telemetry [candidate] (7.559 ms) : 0, 7559
section appsec
BytebuddyAgent [baseline] (705.041 ms) : 0, 705041
BytebuddyAgent [candidate] (700.729 ms) : 0, 700729
GlobalTracer [baseline] (303.296 ms) : 0, 303296
GlobalTracer [candidate] (302.502 ms) : 0, 302502
AppSec [baseline] (163.424 ms) : 0, 163424
AppSec [candidate] (161.827 ms) : 0, 161827
IAST [baseline] (20.666 ms) : 0, 20666
IAST [candidate] (19.634 ms) : 0, 19634
Remote Config [baseline] (638.945 µs) : 0, 639
Remote Config [candidate] (625.276 µs) : 0, 625
Telemetry [baseline] (7.398 ms) : 0, 7398
Telemetry [candidate] (8.692 ms) : 0, 8692
section iast
BytebuddyAgent [baseline] (792.099 ms) : 0, 792099
BytebuddyAgent [candidate] (796.709 ms) : 0, 796709
GlobalTracer [baseline] (298.576 ms) : 0, 298576
GlobalTracer [candidate] (299.452 ms) : 0, 299452
AppSec [baseline] (53.67 ms) : 0, 53670
AppSec [candidate] (51.983 ms) : 0, 51983
IAST [baseline] (23.572 ms) : 0, 23572
IAST [candidate] (26.11 ms) : 0, 26110
Remote Config [baseline] (645.178 µs) : 0, 645
Remote Config [candidate] (650.21 µs) : 0, 650
Telemetry [baseline] (8.062 ms) : 0, 8062
Telemetry [candidate] (7.434 ms) : 0, 7434
section profiling
BytebuddyAgent [baseline] (673.173 ms) : 0, 673173
BytebuddyAgent [candidate] (674.89 ms) : 0, 674890
GlobalTracer [baseline] (393.996 ms) : 0, 393996
GlobalTracer [candidate] (393.126 ms) : 0, 393126
AppSec [baseline] (54.292 ms) : 0, 54292
AppSec [candidate] (53.232 ms) : 0, 53232
Remote Config [baseline] (652.904 µs) : 0, 653
Remote Config [candidate] (663.473 µs) : 0, 663
Telemetry [baseline] (7.455 ms) : 0, 7455
Telemetry [candidate] (7.427 ms) : 0, 7427
ProfilingAgent [baseline] (96.659 ms) : 0, 96659
ProfilingAgent [candidate] (95.876 ms) : 0, 95876
Profiling [baseline] (96.683 ms) : 0, 96683
Profiling [candidate] (95.9 ms) : 0, 95900
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063466
Total [baseline] (8.533 s) : 0, 8532635
Agent [candidate] (1.06 s) : 0, 1059713
Total [candidate] (8.505 s) : 0, 8504691
section iast
Agent [baseline] (1.193 s) : 0, 1193026
Total [baseline] (9.008 s) : 0, 9007902
Agent [candidate] (1.186 s) : 0, 1186323
Total [candidate] (8.981 s) : 0, 8980926
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.189 s) : 0, 1189160
Total [baseline] (8.952 s) : 0, 8952399
Agent [candidate] (1.188 s) : 0, 1187998
Total [candidate] (8.954 s) : 0, 8954487
section iast_TELEMETRY_OFF
Agent [baseline] (1.184 s) : 0, 1183577
Total [baseline] (8.981 s) : 0, 8980653
Agent [candidate] (1.188 s) : 0, 1188372
Total [candidate] (8.989 s) : 0, 8989362
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.193 s 129.56 ms (12.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.189 s 125.694 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.184 s 120.11 ms (11.3%)
Total tracing 8.533 s -
Total iast 9.008 s 475.267 ms (5.6%)
Total iast_HARDCODED_SECRET_DISABLED 8.952 s 419.763 ms (4.9%)
Total iast_TELEMETRY_OFF 8.981 s 448.017 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.06 s -
Agent iast 1.186 s 126.61 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.188 s 128.285 ms (12.1%)
Agent iast_TELEMETRY_OFF 1.188 s 128.659 ms (12.1%)
Total tracing 8.505 s -
Total iast 8.981 s 476.235 ms (5.6%)
Total iast_HARDCODED_SECRET_DISABLED 8.954 s 449.796 ms (5.3%)
Total iast_TELEMETRY_OFF 8.989 s 484.671 ms (5.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.715 ms) : 0, 678715
BytebuddyAgent [candidate] (676.921 ms) : 0, 676921
GlobalTracer [baseline] (309.3 ms) : 0, 309300
GlobalTracer [candidate] (308.865 ms) : 0, 308865
AppSec [baseline] (53.52 ms) : 0, 53520
AppSec [candidate] (52.332 ms) : 0, 52332
Remote Config [baseline] (662.322 µs) : 0, 662
Remote Config [candidate] (652.762 µs) : 0, 653
Telemetry [baseline] (7.704 ms) : 0, 7704
Telemetry [candidate] (7.389 ms) : 0, 7389
section iast
BytebuddyAgent [baseline] (794.536 ms) : 0, 794536
BytebuddyAgent [candidate] (790.155 ms) : 0, 790155
GlobalTracer [baseline] (298.687 ms) : 0, 298687
GlobalTracer [candidate] (297.802 ms) : 0, 297802
AppSec [baseline] (53.714 ms) : 0, 53714
AppSec [candidate] (53.473 ms) : 0, 53473
IAST [baseline] (24.467 ms) : 0, 24467
IAST [candidate] (23.416 ms) : 0, 23416
Remote Config [baseline] (608.924 µs) : 0, 609
Remote Config [candidate] (591.167 µs) : 0, 591
Telemetry [baseline] (7.369 ms) : 0, 7369
Telemetry [candidate] (7.277 ms) : 0, 7277
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (791.545 ms) : 0, 791545
BytebuddyAgent [candidate] (790.827 ms) : 0, 790827
GlobalTracer [baseline] (297.905 ms) : 0, 297905
GlobalTracer [candidate] (298.469 ms) : 0, 298469
AppSec [baseline] (54.654 ms) : 0, 54654
AppSec [candidate] (52.869 ms) : 0, 52869
IAST [baseline] (23.481 ms) : 0, 23481
IAST [candidate] (24.341 ms) : 0, 24341
Remote Config [baseline] (610.441 µs) : 0, 610
Remote Config [candidate] (628.057 µs) : 0, 628
Telemetry [baseline] (7.331 ms) : 0, 7331
Telemetry [candidate] (7.281 ms) : 0, 7281
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (786.388 ms) : 0, 786388
BytebuddyAgent [candidate] (790.126 ms) : 0, 790126
GlobalTracer [baseline] (297.851 ms) : 0, 297851
GlobalTracer [candidate] (299.101 ms) : 0, 299101
AppSec [baseline] (55.764 ms) : 0, 55764
AppSec [candidate] (55.831 ms) : 0, 55831
IAST [baseline] (21.43 ms) : 0, 21430
IAST [candidate] (21.753 ms) : 0, 21753
Remote Config [baseline] (1.368 ms) : 0, 1368
Remote Config [candidate] (607.68 µs) : 0, 608
Telemetry [baseline] (7.179 ms) : 0, 7179
Telemetry [candidate] (7.318 ms) : 0, 7318
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-19T12:45:44 2024-09-19T12:52:37
git_branch master jpbempel/er-config-params
git_commit_date 1726743087 1726749251
git_commit_sha a8ea730 f485063
release_version 1.40.0-SNAPSHOT~a8ea730bf7 1.40.0-SNAPSHOT~f485063f6f
start_time 2024-09-19T12:45:31 2024-09-19T12:52:24
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726750702 1726750702
ci_job_id 644818489 644818489
ci_pipeline_id 44684956 44684956
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 1 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 16 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:petclinic:profiling better
[-84.846µs; -31.778µs] or [-5.569%; -2.086%]		 unstable
[-442.443op/s; +688.596op/s] or [-14.379%; +22.379%]		 1.465ms 3200.000op/s 1.524ms 3076.923op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.652 µs) : 353, 392
.   : milestone, 373,
iast (483.398 µs) : 462, 505
.   : milestone, 483,
iast_FULL (552.317 µs) : 531, 574
.   : milestone, 552,
iast_GLOBAL (511.662 µs) : 490, 534
.   : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (485.05 µs) : 464, 506
.   : milestone, 485,
iast_INACTIVE (459.689 µs) : 439, 480
.   : milestone, 460,
iast_TELEMETRY_OFF (480.358 µs) : 458, 503
.   : milestone, 480,
tracing (442.842 µs) : 422, 463
.   : milestone, 443,
section candidate
no_agent (367.424 µs) : 348, 387
.   : milestone, 367,
iast (481.524 µs) : 460, 503
.   : milestone, 482,
iast_FULL (553.682 µs) : 532, 575
.   : milestone, 554,
iast_GLOBAL (506.149 µs) : 485, 527
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (481.4 µs) : 460, 503
.   : milestone, 481,
iast_INACTIVE (445.063 µs) : 425, 465
.   : milestone, 445,
iast_TELEMETRY_OFF (483.242 µs) : 460, 506
.   : milestone, 483,
tracing (443.812 µs) : 423, 464
.   : milestone, 444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.652 µs [353.169 µs, 392.135 µs] -
iast 483.398 µs [462.161 µs, 504.635 µs] 110.746 µs (29.7%)
iast_FULL 552.317 µs [531.019 µs, 573.616 µs] 179.665 µs (48.2%)
iast_GLOBAL 511.662 µs [489.594 µs, 533.73 µs] 139.01 µs (37.3%)
iast_HARDCODED_SECRET_DISABLED 485.05 µs [463.9 µs, 506.2 µs] 112.398 µs (30.2%)
iast_INACTIVE 459.689 µs [438.943 µs, 480.434 µs] 87.037 µs (23.4%)
iast_TELEMETRY_OFF 480.358 µs [457.552 µs, 503.164 µs] 107.706 µs (28.9%)
tracing 442.842 µs [422.378 µs, 463.307 µs] 70.19 µs (18.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.424 µs [347.952 µs, 386.896 µs] -
iast 481.524 µs [460.472 µs, 502.576 µs] 114.1 µs (31.1%)
iast_FULL 553.682 µs [532.44 µs, 574.925 µs] 186.258 µs (50.7%)
iast_GLOBAL 506.149 µs [484.99 µs, 527.307 µs] 138.724 µs (37.8%)
iast_HARDCODED_SECRET_DISABLED 481.4 µs [460.294 µs, 502.506 µs] 113.976 µs (31.0%)
iast_INACTIVE 445.063 µs [424.738 µs, 465.388 µs] 77.639 µs (21.1%)
iast_TELEMETRY_OFF 483.242 µs [460.471 µs, 506.014 µs] 115.818 µs (31.5%)
tracing 443.812 µs [423.455 µs, 464.169 µs] 76.388 µs (20.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.344 ms) : 1324, 1363
.   : milestone, 1344,
appsec (1.737 ms) : 1713, 1760
.   : milestone, 1737,
appsec_no_iast (1.737 ms) : 1713, 1761
.   : milestone, 1737,
iast (1.469 ms) : 1446, 1492
.   : milestone, 1469,
profiling (1.524 ms) : 1498, 1549
.   : milestone, 1524,
tracing (1.451 ms) : 1427, 1475
.   : milestone, 1451,
section candidate
no_agent (1.334 ms) : 1314, 1353
.   : milestone, 1334,
appsec (1.719 ms) : 1695, 1743
.   : milestone, 1719,
appsec_no_iast (1.734 ms) : 1710, 1759
.   : milestone, 1734,
iast (1.449 ms) : 1426, 1472
.   : milestone, 1449,
profiling (1.465 ms) : 1442, 1489
.   : milestone, 1465,
tracing (1.46 ms) : 1435, 1484
.   : milestone, 1460,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.324 ms, 1.363 ms] -
appsec 1.737 ms [1.713 ms, 1.76 ms] 392.673 µs (29.2%)
appsec_no_iast 1.737 ms [1.713 ms, 1.761 ms] 392.93 µs (29.2%)
iast 1.469 ms [1.446 ms, 1.492 ms] 125.298 µs (9.3%)
profiling 1.524 ms [1.498 ms, 1.549 ms] 179.731 µs (13.4%)
tracing 1.451 ms [1.427 ms, 1.475 ms] 107.097 µs (8.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.334 ms [1.314 ms, 1.353 ms] -
appsec 1.719 ms [1.695 ms, 1.743 ms] 385.059 µs (28.9%)
appsec_no_iast 1.734 ms [1.71 ms, 1.759 ms] 400.8 µs (30.1%)
iast 1.449 ms [1.426 ms, 1.472 ms] 115.392 µs (8.7%)
profiling 1.465 ms [1.442 ms, 1.489 ms] 131.692 µs (9.9%)
tracing 1.46 ms [1.435 ms, 1.484 ms] 125.974 µs (9.4%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/er-config-params
git_commit_date 1726743087 1726749251
git_commit_sha a8ea730 f485063
release_version 1.40.0-SNAPSHOT~a8ea730bf7 1.40.0-SNAPSHOT~f485063f6f
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1726751241 1726751241
ci_job_id 644818490 644818490
ci_pipeline_id 44684956 44684956
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.744 s) : 15744000, 15744000
.   : milestone, 15744000,
appsec (15.454 s) : 15454000, 15454000
.   : milestone, 15454000,
iast (18.674 s) : 18674000, 18674000
.   : milestone, 18674000,
iast_GLOBAL (18.164 s) : 18164000, 18164000
.   : milestone, 18164000,
profiling (15.333 s) : 15333000, 15333000
.   : milestone, 15333000,
tracing (14.897 s) : 14897000, 14897000
.   : milestone, 14897000,
section candidate
no_agent (15.155 s) : 15155000, 15155000
.   : milestone, 15155000,
appsec (16.174 s) : 16174000, 16174000
.   : milestone, 16174000,
iast (19.241 s) : 19241000, 19241000
.   : milestone, 19241000,
iast_GLOBAL (18.01 s) : 18010000, 18010000
.   : milestone, 18010000,
profiling (15.592 s) : 15592000, 15592000
.   : milestone, 15592000,
tracing (15.117 s) : 15117000, 15117000
.   : milestone, 15117000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.744 s [15.744 s, 15.744 s] -
appsec 15.454 s [15.454 s, 15.454 s] -290.0 ms (-1.8%)
iast 18.674 s [18.674 s, 18.674 s] 2.93 s (18.6%)
iast_GLOBAL 18.164 s [18.164 s, 18.164 s] 2.42 s (15.4%)
profiling 15.333 s [15.333 s, 15.333 s] -411.0 ms (-2.6%)
tracing 14.897 s [14.897 s, 14.897 s] -847.0 ms (-5.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.155 s [15.155 s, 15.155 s] -
appsec 16.174 s [16.174 s, 16.174 s] 1.019 s (6.7%)
iast 19.241 s [19.241 s, 19.241 s] 4.086 s (27.0%)
iast_GLOBAL 18.01 s [18.01 s, 18.01 s] 2.855 s (18.8%)
profiling 15.592 s [15.592 s, 15.592 s] 437.0 ms (2.9%)
tracing 15.117 s [15.117 s, 15.117 s] -38.0 ms (-0.3%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~f485063f6f, baseline=1.40.0-SNAPSHOT~a8ea730bf7
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.316 ms) : 2275, 2356
.   : milestone, 2316,
iast (2.049 ms) : 1999, 2099
.   : milestone, 2049,
iast_GLOBAL (2.111 ms) : 2060, 2162
.   : milestone, 2111,
profiling (1.929 ms) : 1889, 1968
.   : milestone, 1929,
tracing (1.901 ms) : 1863, 1939
.   : milestone, 1901,
section candidate
no_agent (1.468 ms) : 1457, 1480
.   : milestone, 1468,
appsec (2.282 ms) : 2242, 2322
.   : milestone, 2282,
iast (2.053 ms) : 2003, 2102
.   : milestone, 2053,
iast_GLOBAL (2.108 ms) : 2057, 2159
.   : milestone, 2108,
profiling (1.926 ms) : 1884, 1968
.   : milestone, 1926,
tracing (1.919 ms) : 1880, 1957
.   : milestone, 1919,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.457 ms, 1.48 ms] -
appsec 2.316 ms [2.275 ms, 2.356 ms] 847.124 µs (57.7%)
iast 2.049 ms [1.999 ms, 2.099 ms] 580.565 µs (39.5%)
iast_GLOBAL 2.111 ms [2.06 ms, 2.162 ms] 642.607 µs (43.8%)
profiling 1.929 ms [1.889 ms, 1.968 ms] 460.154 µs (31.3%)
tracing 1.901 ms [1.863 ms, 1.939 ms] 432.098 µs (29.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.468 ms [1.457 ms, 1.48 ms] -
appsec 2.282 ms [2.242 ms, 2.322 ms] 813.547 µs (55.4%)
iast 2.053 ms [2.003 ms, 2.102 ms] 584.47 µs (39.8%)
iast_GLOBAL 2.108 ms [2.057 ms, 2.159 ms] 639.765 µs (43.6%)
profiling 1.926 ms [1.884 ms, 1.968 ms] 457.917 µs (31.2%)
tracing 1.919 ms [1.88 ms, 1.957 ms] 450.392 µs (30.7%)

smola
smola approved these changes Sep 19, 2024
View reviewed changes
dd-trace-api/src/main/java/datadog/trace/api/config/DebuggerConfig.java
@@ -40,6 +40,12 @@ public final class DebuggerConfig {
"internal.exception.replay.only.local.root";
public static final String DEBUGGER_EXCEPTION_MAX_CAPTURED_FRAMES =
"exception.replay.max.frames.to.capture";
public static final String DEBUGGER_EXCEPTION_CAPTURE_MAX_FRAMES =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick and not blocking, but given the similarity between both variable names (DEBUGGER_EXCEPTION_CAPTURE_MAX_FRAMES, DEBUGGER_EXCEPTION_MAX_CAPTURED_FRAMES) it was somewhat confusing when trying to follow in Config.java. It would make sense that the old variable (DEBUGGER_EXCEPTION_MAX_CAPTURED_FRAMES) was aligned with its actual env var form (e.g. DEBUGGER_REPLAY_MAX_FRAMES_TO_CAPTURE).

Copy link
Member Author

@jpbempel jpbempel Sep 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am trying to decouple what is inside the code what is exposed to the user because more than once we had renaming (debugger-> dynamic instrumentation, exception debugging -> exception replay) and if the code need to follow all the wave of renaming could be worse :)

@jpbempel jpbempel merged commit 0dcde79 into master Sep 24, 2024
105 checks passed
@jpbempel jpbempel deleted the jpbempel/er-config-params branch September 24, 2024 08:26
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 24, 2024
shatzi pushed a commit that referenced this pull request Sep 25, 2024
@jpbempel @shatzi
Add Exception Replay config parameters (#7647)
80ad360 
DD_EXCEPTION_REPLAY_CAPTURE_MAX_FRAMES alias on
DD_EXCEPTION_REPLAY_MAX_FRAMES_TO_CAPTURE
DD_EXCEPTION_REPLAY_CAPTURE_INTERVAL_SECONDS interval in seconds
between 2 capture of the same exceptions
DD_EXCEPTION_REPLAY_CAPTURE_INTERMEDIATE_SPANS_ENABLED inverted alias
on DD_EXCEPTION_REPLAY_ONLY_LOCAL_ROOT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@smola smola smola approved these changes

@evanchooly evanchooly Awaiting requested review from evanchooly evanchooly is a code owner automatically assigned from DataDog/debugger-java

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @smola