Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add watches support through probe tags #7573

Merged
merged 1 commit into from
Sep 19, 2024
Merged

Add watches support through probe tags #7573

merged 1 commit into from
Sep 19, 2024

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Sep 6, 2024 •
edited
Loading

What Does This Do

Watches collect specific (deep) fields to add them exclusively into
the snapshot instead of locals/arguments/statics/...
Watches are defined using probe definition predefined tags:

  • dd_watches_dsl
  • dd_watches_json

When you just want to reference deep fields just a.b.c you can use the
dsl syntax (like Expression Language). But if you need more advanced
(accessing map or list, using filter, any, ... functions) you need to
use dd_watches_json and put the Json AST for Expression Language
(like log message template with segments).
Once the probe is executed, it interprets watches tags to evaluate the
expression then add it in the snapshot as special attributes watches.

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-2829

@pr-commenter
Copy link

pr-commenter bot commented Sep 6, 2024

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1725621599 1725621987
end_time 2024-09-06T11:21:12 2024-09-06T11:27:41
git_branch master jpbempel/poc-watches
git_commit_sha 70e0c21 96b623f
start_time 2024-09-06T11:20:00 2024-09-06T11:26:28
See matching parameters
Baseline Candidate
ci_job_id 630062901 630062901
ci_pipeline_id 43719131 43719131
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1725621025 1725621025

Summary

Found 5 performance improvements and 0 performance regressions! Performance is the same for 0 metrics, 10 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop better
[-472.711µs; -450.402µs] or [-4.370%; -4.164%]		 better
[-498.312µs; -445.218µs] or [-4.536%; -4.053%]		 better
[-506.673µs; -392.396µs] or [-4.579%; -3.547%]		 better
[-724.273µs; -234.313µs] or [-6.309%; -2.041%]		 better
[+2.846op/s; +4.767op/s] or [+3.159%; +5.291%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-28.149µs; +50.227µs] or [-10.959%; +19.555%]		 unstable
[-37.497µs; +68.758µs] or [-12.902%; +23.659%]		 unstable
[-49.409µs; +80.591µs] or [-16.186%; +26.401%]		 unstable
[-221.830µs; +569.594µs] or [-34.456%; +88.474%]		 unstable
[-175.745op/s; +101.698op/s] or [-6.415%; +3.712%]
scenario:basic unstable
[-9.209µs; +24.740µs] or [-3.470%; +9.322%]		 unstable
[-0.952µs; +45.830µs] or [-0.330%; +15.904%]		 unstable
[+1.132µs; +60.434µs] or [+0.381%; +20.313%]		 unstable
[-20.902µs; +115.441µs] or [-3.614%; +19.962%]		 unstable
[-271.180op/s; +136.228op/s] or [-10.305%; +5.177%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (290.624 µs) : 254, 328
.   : milestone, 291,
basic (288.161 µs) : 281, 295
.   : milestone, 288,
loop (10.985 ms) : 10962, 11008
.   : milestone, 10985,
section candidate
noprobe (306.255 µs) : 245, 368
.   : milestone, 306,
basic (310.599 µs) : 279, 342
.   : milestone, 311,
loop (10.513 ms) : 10487, 10540
.   : milestone, 10513,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 290.624 µs [253.711 µs, 327.537 µs]
basic 288.161 µs [280.997 µs, 295.325 µs]
loop 10.985 ms [10.962 ms, 11.008 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 306.255 µs [244.787 µs, 367.723 µs]
basic 310.599 µs [279.039 µs, 342.16 µs]
loop 10.513 ms [10.487 ms, 10.54 ms]

@pr-commenter
Copy link

pr-commenter bot commented Sep 6, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/poc-watches
git_commit_date 1726653524 1726671253
git_commit_sha 5765052 fc56979
release_version 1.40.0-SNAPSHOT~5765052e34 1.40.0-SNAPSHOT~fc569797c2
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726673765 1726673765
ci_job_id 643643506 643643506
ci_pipeline_id 44595842 44595842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 49 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.064 s) : 0, 1063772
Total [baseline] (10.384 s) : 0, 10384251
Agent [candidate] (1.069 s) : 0, 1068769
Total [candidate] (10.404 s) : 0, 10403739
section appsec
Agent [baseline] (1.199 s) : 0, 1198510
Total [baseline] (10.683 s) : 0, 10682815
Agent [candidate] (1.202 s) : 0, 1202014
Total [candidate] (10.664 s) : 0, 10663790
section iast
Agent [baseline] (1.186 s) : 0, 1185865
Total [baseline] (10.848 s) : 0, 10847837
Agent [candidate] (1.197 s) : 0, 1197263
Total [candidate] (10.855 s) : 0, 10854748
section profiling
Agent [baseline] (1.264 s) : 0, 1264276
Total [baseline] (10.596 s) : 0, 10595923
Agent [candidate] (1.263 s) : 0, 1263170
Total [candidate] (10.712 s) : 0, 10712192
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent appsec 1.199 s 134.739 ms (12.7%)
Agent iast 1.186 s 122.093 ms (11.5%)
Agent profiling 1.264 s 200.505 ms (18.8%)
Total tracing 10.384 s -
Total appsec 10.683 s 298.564 ms (2.9%)
Total iast 10.848 s 463.586 ms (4.5%)
Total profiling 10.596 s 211.672 ms (2.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.069 s -
Agent appsec 1.202 s 133.245 ms (12.5%)
Agent iast 1.197 s 128.494 ms (12.0%)
Agent profiling 1.263 s 194.401 ms (18.2%)
Total tracing 10.404 s -
Total appsec 10.664 s 260.051 ms (2.5%)
Total iast 10.855 s 451.009 ms (4.3%)
Total profiling 10.712 s 308.453 ms (3.0%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.965 ms) : 0, 678965
BytebuddyAgent [candidate] (682.678 ms) : 0, 682678
GlobalTracer [baseline] (310.503 ms) : 0, 310503
GlobalTracer [candidate] (311.832 ms) : 0, 311832
AppSec [baseline] (52.496 ms) : 0, 52496
AppSec [candidate] (52.364 ms) : 0, 52364
Remote Config [baseline] (686.461 µs) : 0, 686
Remote Config [candidate] (679.628 µs) : 0, 680
Telemetry [baseline] (7.476 ms) : 0, 7476
Telemetry [candidate] (7.543 ms) : 0, 7543
section appsec
BytebuddyAgent [baseline] (701.467 ms) : 0, 701467
BytebuddyAgent [candidate] (703.255 ms) : 0, 703255
GlobalTracer [baseline] (303.307 ms) : 0, 303307
GlobalTracer [candidate] (303.647 ms) : 0, 303647
AppSec [baseline] (160.441 ms) : 0, 160441
AppSec [candidate] (162.544 ms) : 0, 162544
Remote Config [baseline] (633.566 µs) : 0, 634
Remote Config [candidate] (625.945 µs) : 0, 626
Telemetry [baseline] (8.76 ms) : 0, 8760
Telemetry [candidate] (8.748 ms) : 0, 8748
IAST [baseline] (20.583 ms) : 0, 20583
IAST [candidate] (19.091 ms) : 0, 19091
section iast
BytebuddyAgent [baseline] (788.862 ms) : 0, 788862
BytebuddyAgent [candidate] (797.279 ms) : 0, 797279
GlobalTracer [baseline] (298.26 ms) : 0, 298260
GlobalTracer [candidate] (301.337 ms) : 0, 301337
AppSec [baseline] (51.908 ms) : 0, 51908
AppSec [candidate] (53.052 ms) : 0, 53052
Remote Config [baseline] (619.11 µs) : 0, 619
Remote Config [candidate] (636.539 µs) : 0, 637
Telemetry [baseline] (7.315 ms) : 0, 7315
Telemetry [candidate] (7.483 ms) : 0, 7483
IAST [baseline] (25.309 ms) : 0, 25309
IAST [candidate] (23.731 ms) : 0, 23731
section profiling
ProfilingAgent [baseline] (95.89 ms) : 0, 95890
ProfilingAgent [candidate] (96.672 ms) : 0, 96672
BytebuddyAgent [baseline] (673.604 ms) : 0, 673604
BytebuddyAgent [candidate] (672.283 ms) : 0, 672283
GlobalTracer [baseline] (395.561 ms) : 0, 395561
GlobalTracer [candidate] (395.098 ms) : 0, 395098
AppSec [baseline] (52.996 ms) : 0, 52996
AppSec [candidate] (53.059 ms) : 0, 53059
Remote Config [baseline] (672.616 µs) : 0, 673
Remote Config [candidate] (665.994 µs) : 0, 666
Telemetry [baseline] (7.418 ms) : 0, 7418
Telemetry [candidate] (7.392 ms) : 0, 7392
Profiling [baseline] (95.914 ms) : 0, 95914
Profiling [candidate] (96.696 ms) : 0, 96696
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1062910
Total [baseline] (8.556 s) : 0, 8556355
Agent [candidate] (1.064 s) : 0, 1063828
Total [candidate] (8.533 s) : 0, 8532834
section iast
Agent [baseline] (1.198 s) : 0, 1198211
Total [baseline] (9.042 s) : 0, 9042175
Agent [candidate] (1.19 s) : 0, 1190353
Total [candidate] (9.029 s) : 0, 9029480
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.197 s) : 0, 1197117
Total [baseline] (9.011 s) : 0, 9011181
Agent [candidate] (1.191 s) : 0, 1191251
Total [candidate] (8.997 s) : 0, 8997395
section iast_TELEMETRY_OFF
Agent [baseline] (1.188 s) : 0, 1188232
Total [baseline] (9.007 s) : 0, 9007247
Agent [candidate] (1.186 s) : 0, 1186251
Total [candidate] (9.04 s) : 0, 9040486
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.198 s 135.301 ms (12.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.197 s 134.207 ms (12.6%)
Agent iast_TELEMETRY_OFF 1.188 s 125.322 ms (11.8%)
Total tracing 8.556 s -
Total iast 9.042 s 485.82 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.011 s 454.826 ms (5.3%)
Total iast_TELEMETRY_OFF 9.007 s 450.892 ms (5.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.064 s -
Agent iast 1.19 s 126.525 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.191 s 127.423 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.186 s 122.423 ms (11.5%)
Total tracing 8.533 s -
Total iast 9.029 s 496.646 ms (5.8%)
Total iast_HARDCODED_SECRET_DISABLED 8.997 s 464.561 ms (5.4%)
Total iast_TELEMETRY_OFF 9.04 s 507.653 ms (5.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.938 ms) : 0, 678938
BytebuddyAgent [candidate] (679.004 ms) : 0, 679004
GlobalTracer [baseline] (309.679 ms) : 0, 309679
GlobalTracer [candidate] (310.674 ms) : 0, 310674
AppSec [baseline] (52.483 ms) : 0, 52483
AppSec [candidate] (52.384 ms) : 0, 52384
Remote Config [baseline] (670.768 µs) : 0, 671
Remote Config [candidate] (659.926 µs) : 0, 660
Telemetry [baseline] (7.506 ms) : 0, 7506
Telemetry [candidate] (7.446 ms) : 0, 7446
section iast
BytebuddyAgent [baseline] (797.891 ms) : 0, 797891
BytebuddyAgent [candidate] (792.363 ms) : 0, 792363
GlobalTracer [baseline] (301.489 ms) : 0, 301489
GlobalTracer [candidate] (299.16 ms) : 0, 299160
AppSec [baseline] (53.638 ms) : 0, 53638
AppSec [candidate] (53.755 ms) : 0, 53755
IAST [baseline] (23.489 ms) : 0, 23489
IAST [candidate] (23.468 ms) : 0, 23468
Remote Config [baseline] (634.749 µs) : 0, 635
Remote Config [candidate] (632.892 µs) : 0, 633
Telemetry [baseline] (7.325 ms) : 0, 7325
Telemetry [candidate] (7.286 ms) : 0, 7286
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (796.747 ms) : 0, 796747
BytebuddyAgent [candidate] (792.358 ms) : 0, 792358
GlobalTracer [baseline] (300.849 ms) : 0, 300849
GlobalTracer [candidate] (299.997 ms) : 0, 299997
AppSec [baseline] (53.298 ms) : 0, 53298
AppSec [candidate] (54.775 ms) : 0, 54775
IAST [baseline] (24.462 ms) : 0, 24462
IAST [candidate] (22.539 ms) : 0, 22539
Remote Config [baseline] (634.819 µs) : 0, 635
Remote Config [candidate] (620.287 µs) : 0, 620
Telemetry [baseline] (7.374 ms) : 0, 7374
Telemetry [candidate] (7.256 ms) : 0, 7256
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (789.78 ms) : 0, 789780
BytebuddyAgent [candidate] (787.927 ms) : 0, 787927
GlobalTracer [baseline] (300.017 ms) : 0, 300017
GlobalTracer [candidate] (299.535 ms) : 0, 299535
AppSec [baseline] (53.977 ms) : 0, 53977
AppSec [candidate] (54.816 ms) : 0, 54816
IAST [baseline] (22.187 ms) : 0, 22187
IAST [candidate] (22.451 ms) : 0, 22451
Remote Config [baseline] (604.605 µs) : 0, 605
Remote Config [candidate] (606.713 µs) : 0, 607
Telemetry [baseline] (8.012 ms) : 0, 8012
Telemetry [candidate] (7.279 ms) : 0, 7279
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-18T15:06:31 2024-09-18T15:13:24
git_branch master jpbempel/poc-watches
git_commit_date 1726653524 1726671253
git_commit_sha 5765052 fc56979
release_version 1.40.0-SNAPSHOT~5765052e34 1.40.0-SNAPSHOT~fc569797c2
start_time 2024-09-18T15:06:18 2024-09-18T15:13:10
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726672750 1726672750
ci_job_id 643643507 643643507
ci_pipeline_id 44595842 44595842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.349 ms) : 1330, 1369
.   : milestone, 1349,
appsec (1.711 ms) : 1686, 1736
.   : milestone, 1711,
appsec_no_iast (1.734 ms) : 1710, 1758
.   : milestone, 1734,
iast (1.49 ms) : 1467, 1513
.   : milestone, 1490,
profiling (1.552 ms) : 1527, 1576
.   : milestone, 1552,
tracing (1.47 ms) : 1445, 1495
.   : milestone, 1470,
section candidate
no_agent (1.342 ms) : 1322, 1361
.   : milestone, 1342,
appsec (1.706 ms) : 1682, 1731
.   : milestone, 1706,
appsec_no_iast (1.733 ms) : 1709, 1757
.   : milestone, 1733,
iast (1.485 ms) : 1462, 1508
.   : milestone, 1485,
profiling (1.511 ms) : 1485, 1537
.   : milestone, 1511,
tracing (1.472 ms) : 1448, 1496
.   : milestone, 1472,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.349 ms [1.33 ms, 1.369 ms] -
appsec 1.711 ms [1.686 ms, 1.736 ms] 361.285 µs (26.8%)
appsec_no_iast 1.734 ms [1.71 ms, 1.758 ms] 384.518 µs (28.5%)
iast 1.49 ms [1.467 ms, 1.513 ms] 140.364 µs (10.4%)
profiling 1.552 ms [1.527 ms, 1.576 ms] 202.416 µs (15.0%)
tracing 1.47 ms [1.445 ms, 1.495 ms] 120.463 µs (8.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.322 ms, 1.361 ms] -
appsec 1.706 ms [1.682 ms, 1.731 ms] 364.775 µs (27.2%)
appsec_no_iast 1.733 ms [1.709 ms, 1.757 ms] 391.578 µs (29.2%)
iast 1.485 ms [1.462 ms, 1.508 ms] 143.242 µs (10.7%)
profiling 1.511 ms [1.485 ms, 1.537 ms] 169.18 µs (12.6%)
tracing 1.472 ms [1.448 ms, 1.496 ms] 130.541 µs (9.7%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.484 µs) : 353, 392
.   : milestone, 372,
iast (491.901 µs) : 470, 514
.   : milestone, 492,
iast_FULL (556.386 µs) : 535, 578
.   : milestone, 556,
iast_GLOBAL (514.309 µs) : 492, 536
.   : milestone, 514,
iast_HARDCODED_SECRET_DISABLED (488.364 µs) : 467, 510
.   : milestone, 488,
iast_INACTIVE (451.523 µs) : 430, 473
.   : milestone, 452,
iast_TELEMETRY_OFF (479.232 µs) : 456, 502
.   : milestone, 479,
tracing (446.724 µs) : 426, 467
.   : milestone, 447,
section candidate
no_agent (372.736 µs) : 353, 393
.   : milestone, 373,
iast (484.396 µs) : 463, 505
.   : milestone, 484,
iast_FULL (559.401 µs) : 538, 581
.   : milestone, 559,
iast_GLOBAL (512.023 µs) : 490, 534
.   : milestone, 512,
iast_HARDCODED_SECRET_DISABLED (485.488 µs) : 464, 507
.   : milestone, 485,
iast_INACTIVE (451.634 µs) : 430, 473
.   : milestone, 452,
iast_TELEMETRY_OFF (479.667 µs) : 457, 502
.   : milestone, 480,
tracing (445.301 µs) : 425, 466
.   : milestone, 445,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.484 µs [352.585 µs, 392.384 µs] -
iast 491.901 µs [469.508 µs, 514.294 µs] 119.417 µs (32.1%)
iast_FULL 556.386 µs [535.049 µs, 577.722 µs] 183.901 µs (49.4%)
iast_GLOBAL 514.309 µs [492.469 µs, 536.15 µs] 141.825 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 488.364 µs [467.127 µs, 509.602 µs] 115.88 µs (31.1%)
iast_INACTIVE 451.523 µs [430.239 µs, 472.807 µs] 79.038 µs (21.2%)
iast_TELEMETRY_OFF 479.232 µs [456.131 µs, 502.332 µs] 106.748 µs (28.7%)
tracing 446.724 µs [426.345 µs, 467.103 µs] 74.24 µs (19.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.736 µs [352.756 µs, 392.715 µs] -
iast 484.396 µs [463.386 µs, 505.405 µs] 111.66 µs (30.0%)
iast_FULL 559.401 µs [538.15 µs, 580.652 µs] 186.665 µs (50.1%)
iast_GLOBAL 512.023 µs [489.962 µs, 534.084 µs] 139.287 µs (37.4%)
iast_HARDCODED_SECRET_DISABLED 485.488 µs [463.836 µs, 507.14 µs] 112.752 µs (30.2%)
iast_INACTIVE 451.634 µs [430.459 µs, 472.808 µs] 78.898 µs (21.2%)
iast_TELEMETRY_OFF 479.667 µs [457.226 µs, 502.108 µs] 106.931 µs (28.7%)
tracing 445.301 µs [424.629 µs, 465.972 µs] 72.565 µs (19.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/poc-watches
git_commit_date 1726653524 1726671253
git_commit_sha 5765052 fc56979
release_version 1.40.0-SNAPSHOT~5765052e34 1.40.0-SNAPSHOT~fc569797c2
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1726673252 1726673252
ci_job_id 643643508 643643508
ci_pipeline_id 44595842 44595842
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.469 ms) : 1457, 1480
.   : milestone, 1469,
appsec (2.285 ms) : 2245, 2325
.   : milestone, 2285,
iast (2.029 ms) : 1981, 2077
.   : milestone, 2029,
iast_GLOBAL (2.109 ms) : 2058, 2160
.   : milestone, 2109,
profiling (1.919 ms) : 1879, 1960
.   : milestone, 1919,
tracing (1.901 ms) : 1863, 1939
.   : milestone, 1901,
section candidate
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (2.3 ms) : 2259, 2341
.   : milestone, 2300,
iast (2.055 ms) : 2004, 2105
.   : milestone, 2055,
iast_GLOBAL (2.109 ms) : 2057, 2160
.   : milestone, 2109,
profiling (1.941 ms) : 1901, 1982
.   : milestone, 1941,
tracing (1.912 ms) : 1873, 1951
.   : milestone, 1912,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.469 ms [1.457 ms, 1.48 ms] -
appsec 2.285 ms [2.245 ms, 2.325 ms] 815.85 µs (55.5%)
iast 2.029 ms [1.981 ms, 2.077 ms] 560.482 µs (38.2%)
iast_GLOBAL 2.109 ms [2.058 ms, 2.16 ms] 640.21 µs (43.6%)
profiling 1.919 ms [1.879 ms, 1.96 ms] 450.618 µs (30.7%)
tracing 1.901 ms [1.863 ms, 1.939 ms] 432.435 µs (29.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 2.3 ms [2.259 ms, 2.341 ms] 825.818 µs (56.0%)
iast 2.055 ms [2.004 ms, 2.105 ms] 580.188 µs (39.4%)
iast_GLOBAL 2.109 ms [2.057 ms, 2.16 ms] 634.236 µs (43.0%)
profiling 1.941 ms [1.901 ms, 1.982 ms] 466.773 µs (31.7%)
tracing 1.912 ms [1.873 ms, 1.951 ms] 437.79 µs (29.7%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~fc569797c2, baseline=1.40.0-SNAPSHOT~5765052e34
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.779 s) : 15779000, 15779000
.   : milestone, 15779000,
appsec (15.165 s) : 15165000, 15165000
.   : milestone, 15165000,
iast (18.806 s) : 18806000, 18806000
.   : milestone, 18806000,
iast_GLOBAL (17.875 s) : 17875000, 17875000
.   : milestone, 17875000,
profiling (15.466 s) : 15466000, 15466000
.   : milestone, 15466000,
tracing (15.004 s) : 15004000, 15004000
.   : milestone, 15004000,
section candidate
no_agent (15.041 s) : 15041000, 15041000
.   : milestone, 15041000,
appsec (15.294 s) : 15294000, 15294000
.   : milestone, 15294000,
iast (18.872 s) : 18872000, 18872000
.   : milestone, 18872000,
iast_GLOBAL (17.808 s) : 17808000, 17808000
.   : milestone, 17808000,
profiling (15.17 s) : 15170000, 15170000
.   : milestone, 15170000,
tracing (15.189 s) : 15189000, 15189000
.   : milestone, 15189000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.779 s [15.779 s, 15.779 s] -
appsec 15.165 s [15.165 s, 15.165 s] -614.0 ms (-3.9%)
iast 18.806 s [18.806 s, 18.806 s] 3.027 s (19.2%)
iast_GLOBAL 17.875 s [17.875 s, 17.875 s] 2.096 s (13.3%)
profiling 15.466 s [15.466 s, 15.466 s] -313.0 ms (-2.0%)
tracing 15.004 s [15.004 s, 15.004 s] -775.0 ms (-4.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.041 s [15.041 s, 15.041 s] -
appsec 15.294 s [15.294 s, 15.294 s] 253.0 ms (1.7%)
iast 18.872 s [18.872 s, 18.872 s] 3.831 s (25.5%)
iast_GLOBAL 17.808 s [17.808 s, 17.808 s] 2.767 s (18.4%)
profiling 15.17 s [15.17 s, 15.17 s] 129.0 ms (0.9%)
tracing 15.189 s [15.189 s, 15.189 s] 148.0 ms (1.0%)

@jpbempel
Add watches support through probe tags
fc56979 
Watches collect specific (deep) fields to add them exclusively into
the snapshot instead of locals/arguments/statics/...
Watches are defined using probe definition predefined tags:
 - dd_watches_dsl
 - dd_watches_json
When you just want to reference deep fields just a.b.c you can use the
dsl syntax (like Expression Language). But if you need more advanced
(accessing map or list, using filter, any, ... functions) you need to
use dd_watches_json and put the Json AST for Expression Language
(like log message template with segments).
Once the probe is executed, it interprets watches tags to evaluate the
expression then add it in the snapshot as special attributes watches.
@jpbempel jpbempel changed the title DRAFT Watches Add watches support through probe tags Sep 18, 2024
@jpbempel jpbempel marked this pull request as ready for review September 18, 2024 14:58
@jpbempel jpbempel requested a review from a team as a code owner September 18, 2024 14:58
@jpbempel jpbempel requested review from ojung and removed request for a team September 18, 2024 14:58
@jpbempel jpbempel added comp: debugger Dynamic Instrumentation type: enhancement labels Sep 18, 2024
shatzi
shatzi approved these changes Sep 18, 2024
View reviewed changes
Copy link
Contributor

@shatzi shatzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/LogProbe.java
@@ -759,6 +836,7 @@ public static class Builder extends ProbeDefinition.Builder<Builder> {
private String template;
private List<Segment> segments;
private boolean captureSnapshot;
private List<ValueScript> watches;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think should be similar to EvaluationError format. to store "expr" and "value" for each watch. that way in the future the UI can help the user to understand which watch created witch value.

Maybe we should also add "error" if the watch evaluated as error - topics for the RFC I assume

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ValueScript already wraps string representation of the expr and the expression root AST. I am using the string DSL to add it to the snapshot that way we know which watch it is

dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/LogProbe.java
sb.append("...");
msg = sb.toString();
processMsgTemplate(context, logStatus);
processWatches(context, logStatus);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want watches get processed so normal logs probe as well? neat stuff.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no why?
log templates are already acting as watches

@jpbempel jpbempel merged commit b4718bd into master Sep 19, 2024
103 checks passed
@jpbempel jpbempel deleted the jpbempel/poc-watches branch September 19, 2024 12:40
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 19, 2024
jordan-wong pushed a commit that referenced this pull request Sep 23, 2024
@jpbempel @jordan-wong
Add watches support through probe tags (#7573)
4b3bbb4 
Watches collect specific (deep) fields to add them exclusively into
the snapshot instead of locals/arguments/statics/...
Watches are defined using probe definition predefined tags:
 - dd_watches_dsl
 - dd_watches_json
When you just want to reference deep fields just a.b.c you can use the
dsl syntax (like Expression Language). But if you need more advanced
(accessing map or list, using filter, any, ... functions) you need to
use dd_watches_json and put the Json AST for Expression Language
(like log message template with segments).
Once the probe is executed, it interprets watches tags to evaluate the
expression then add it in the snapshot as special attributes watches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shatzi shatzi shatzi approved these changes

@ojung ojung Awaiting requested review from ojung ojung is a code owner automatically assigned from DataDog/debugger-java

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation type: enhancement
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @shatzi