Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix stack trace inconsistency between excluded frames in vulnerability location and metastruct stack trace #7865

Merged
merged 12 commits into from
Nov 7, 2024
Merged

Fix stack trace inconsistency between excluded frames in vulnerability location and metastruct stack trace #7865

merged 12 commits into from
Nov 7, 2024

Conversation

jandro996
Copy link
Member

@jandro996 jandro996 commented Oct 31, 2024 •
edited
Loading

What Does This Do

  • Parametrize StackUtils#generateUserCodeStackTrace with a predicate
  • Add AbstractStackWalker#isNotDatadogTraceStackElement as default filter
  • Update isNotDatadogTraceStackElement to also filter com.datadog.appsec

Motivation

We are filtering different stack frames according to their class to calculate the vulnerability location and the vulnerability stack trace that is reported via meta struct

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jandro996
Fix stacktrace inconsistency between excluded frames in vulnerability…
a6b89e5 
… location and vulnerability stacktrace reported in metastruct
@jandro996
Copy link
Member Author

I'm not sure if we need more test to validate this change, keep in mind that it would be also tested in system tests as right now it would fail due to the vulnerability location is filtered from the reported stack trace as the location is in a com.datadog package

@pr-commenter
Copy link

pr-commenter bot commented Oct 31, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alejandro.gonzalez/fix-stack-inconsistency-iast
git_commit_date 1730827777 1730877317
git_commit_sha 21c0b2d 1d9c6df
release_version 1.43.0-SNAPSHOT~21c0b2df63 1.43.0-SNAPSHOT~1d9c6df96f
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1730879790 1730879790
ci_job_id 696791987 696791987
ci_pipeline_id 48320058 48320058
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 52 metrics, 11 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.089 s) : 0, 1089155
Total [baseline] (10.393 s) : 0, 10393371
Agent [candidate] (1.087 s) : 0, 1087424
Total [candidate] (10.538 s) : 0, 10537620
section appsec
Agent [baseline] (1.221 s) : 0, 1220705
Total [baseline] (10.694 s) : 0, 10693876
Agent [candidate] (1.224 s) : 0, 1223818
Total [candidate] (10.726 s) : 0, 10725792
section iast
Agent [baseline] (1.21 s) : 0, 1209844
Total [baseline] (10.909 s) : 0, 10909392
Agent [candidate] (1.209 s) : 0, 1209393
Total [candidate] (11.046 s) : 0, 11046367
section profiling
Agent [baseline] (1.295 s) : 0, 1294853
Total [baseline] (10.906 s) : 0, 10905820
Agent [candidate] (1.28 s) : 0, 1280040
Total [candidate] (10.801 s) : 0, 10800960
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.089 s -
Agent appsec 1.221 s 131.549 ms (12.1%)
Agent iast 1.21 s 120.689 ms (11.1%)
Agent profiling 1.295 s 205.697 ms (18.9%)
Total tracing 10.393 s -
Total appsec 10.694 s 300.505 ms (2.9%)
Total iast 10.909 s 516.022 ms (5.0%)
Total profiling 10.906 s 512.45 ms (4.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.087 s -
Agent appsec 1.224 s 136.394 ms (12.5%)
Agent iast 1.209 s 121.969 ms (11.2%)
Agent profiling 1.28 s 192.616 ms (17.7%)
Total tracing 10.538 s -
Total appsec 10.726 s 188.172 ms (1.8%)
Total iast 11.046 s 508.747 ms (4.8%)
Total profiling 10.801 s 263.34 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (692.357 ms) : 0, 692357
BytebuddyAgent [candidate] (689.593 ms) : 0, 689593
GlobalTracer [baseline] (318.437 ms) : 0, 318437
GlobalTracer [candidate] (317.766 ms) : 0, 317766
AppSec [baseline] (54.638 ms) : 0, 54638
AppSec [candidate] (54.269 ms) : 0, 54269
Remote Config [baseline] (679.621 µs) : 0, 680
Remote Config [candidate] (664.5 µs) : 0, 664
Telemetry [baseline] (9.196 ms) : 0, 9196
Telemetry [candidate] (11.374 ms) : 0, 11374
section appsec
BytebuddyAgent [baseline] (706.317 ms) : 0, 706317
BytebuddyAgent [candidate] (708.578 ms) : 0, 708578
GlobalTracer [baseline] (315.28 ms) : 0, 315280
GlobalTracer [candidate] (315.925 ms) : 0, 315925
AppSec [baseline] (167.247 ms) : 0, 167247
AppSec [candidate] (166.851 ms) : 0, 166851
Remote Config [baseline] (639.093 µs) : 0, 639
Remote Config [candidate] (646.049 µs) : 0, 646
Telemetry [baseline] (7.753 ms) : 0, 7753
Telemetry [candidate] (7.494 ms) : 0, 7494
IAST [baseline] (19.391 ms) : 0, 19391
IAST [candidate] (21.034 ms) : 0, 21034
section iast
BytebuddyAgent [baseline] (803.629 ms) : 0, 803629
BytebuddyAgent [candidate] (803.111 ms) : 0, 803111
GlobalTracer [baseline] (305.266 ms) : 0, 305266
GlobalTracer [candidate] (306.102 ms) : 0, 306102
AppSec [baseline] (57.332 ms) : 0, 57332
AppSec [candidate] (57.403 ms) : 0, 57403
Remote Config [baseline] (633.612 µs) : 0, 634
Remote Config [candidate] (622.178 µs) : 0, 622
Telemetry [baseline] (7.58 ms) : 0, 7580
Telemetry [candidate] (7.559 ms) : 0, 7559
IAST [baseline] (21.616 ms) : 0, 21616
IAST [candidate] (20.831 ms) : 0, 20831
section profiling
BytebuddyAgent [baseline] (688.008 ms) : 0, 688008
BytebuddyAgent [candidate] (680.688 ms) : 0, 680688
GlobalTracer [baseline] (404.73 ms) : 0, 404730
GlobalTracer [candidate] (400.604 ms) : 0, 400604
AppSec [baseline] (55.263 ms) : 0, 55263
AppSec [candidate] (54.472 ms) : 0, 54472
Remote Config [baseline] (693.129 µs) : 0, 693
Remote Config [candidate] (675.679 µs) : 0, 676
Telemetry [baseline] (10.168 ms) : 0, 10168
Telemetry [candidate] (12.864 ms) : 0, 12864
ProfilingAgent [baseline] (96.509 ms) : 0, 96509
ProfilingAgent [candidate] (91.724 ms) : 0, 91724
Profiling [baseline] (96.532 ms) : 0, 96532
Profiling [candidate] (91.748 ms) : 0, 91748
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1084651
Total [baseline] (8.589 s) : 0, 8589287
Agent [candidate] (1.082 s) : 0, 1081593
Total [candidate] (8.593 s) : 0, 8592657
section iast
Agent [baseline] (1.216 s) : 0, 1215746
Total [baseline] (9.159 s) : 0, 9159283
Agent [candidate] (1.208 s) : 0, 1208339
Total [candidate] (9.158 s) : 0, 9157669
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.225 s) : 0, 1225179
Total [baseline] (9.151 s) : 0, 9150910
Agent [candidate] (1.21 s) : 0, 1209537
Total [candidate] (9.138 s) : 0, 9138078
section iast_TELEMETRY_OFF
Agent [baseline] (1.205 s) : 0, 1205289
Total [baseline] (9.139 s) : 0, 9138553
Agent [candidate] (1.214 s) : 0, 1214121
Total [candidate] (9.154 s) : 0, 9153595
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent iast 1.216 s 131.095 ms (12.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.225 s 140.528 ms (13.0%)
Agent iast_TELEMETRY_OFF 1.205 s 120.638 ms (11.1%)
Total tracing 8.589 s -
Total iast 9.159 s 569.996 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.151 s 561.623 ms (6.5%)
Total iast_TELEMETRY_OFF 9.139 s 549.266 ms (6.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.082 s -
Agent iast 1.208 s 126.746 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.21 s 127.943 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.214 s 132.527 ms (12.3%)
Total tracing 8.593 s -
Total iast 9.158 s 565.012 ms (6.6%)
Total iast_HARDCODED_SECRET_DISABLED 9.138 s 545.421 ms (6.3%)
Total iast_TELEMETRY_OFF 9.154 s 560.938 ms (6.5%) 
gantt
    title insecure-bank - break down per module: candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.491 ms) : 0, 689491
BytebuddyAgent [candidate] (687.562 ms) : 0, 687562
GlobalTracer [baseline] (317.198 ms) : 0, 317198
GlobalTracer [candidate] (316.346 ms) : 0, 316346
AppSec [baseline] (54.248 ms) : 0, 54248
AppSec [candidate] (54.132 ms) : 0, 54132
Remote Config [baseline] (668.533 µs) : 0, 669
Remote Config [candidate] (687.695 µs) : 0, 688
Telemetry [baseline] (9.216 ms) : 0, 9216
Telemetry [candidate] (9.104 ms) : 0, 9104
section iast
BytebuddyAgent [baseline] (808.661 ms) : 0, 808661
BytebuddyAgent [candidate] (802.468 ms) : 0, 802468
GlobalTracer [baseline] (306.78 ms) : 0, 306780
GlobalTracer [candidate] (304.998 ms) : 0, 304998
AppSec [baseline] (57.048 ms) : 0, 57048
AppSec [candidate] (57.405 ms) : 0, 57405
Remote Config [baseline] (613.223 µs) : 0, 613
Remote Config [candidate] (609.835 µs) : 0, 610
Telemetry [baseline] (7.484 ms) : 0, 7484
Telemetry [candidate] (7.504 ms) : 0, 7504
IAST [baseline] (21.293 ms) : 0, 21293
IAST [candidate] (21.569 ms) : 0, 21569
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (815.636 ms) : 0, 815636
BytebuddyAgent [candidate] (803.675 ms) : 0, 803675
GlobalTracer [baseline] (307.639 ms) : 0, 307639
GlobalTracer [candidate] (306.048 ms) : 0, 306048
AppSec [baseline] (57.834 ms) : 0, 57834
AppSec [candidate] (57.545 ms) : 0, 57545
Remote Config [baseline] (642.653 µs) : 0, 643
Remote Config [candidate] (614.326 µs) : 0, 614
Telemetry [baseline] (7.663 ms) : 0, 7663
Telemetry [candidate] (7.455 ms) : 0, 7455
IAST [baseline] (21.758 ms) : 0, 21758
IAST [candidate] (20.383 ms) : 0, 20383
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (800.107 ms) : 0, 800107
BytebuddyAgent [candidate] (806.371 ms) : 0, 806371
GlobalTracer [baseline] (304.614 ms) : 0, 304614
GlobalTracer [candidate] (307.467 ms) : 0, 307467
AppSec [baseline] (56.571 ms) : 0, 56571
AppSec [candidate] (58.197 ms) : 0, 58197
Remote Config [baseline] (618.121 µs) : 0, 618
Remote Config [candidate] (626.171 µs) : 0, 626
Telemetry [baseline] (7.466 ms) : 0, 7466
Telemetry [candidate] (7.39 ms) : 0, 7390
IAST [baseline] (22.138 ms) : 0, 22138
IAST [candidate] (20.204 ms) : 0, 20204
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-11-06T07:26:41 2024-11-06T07:33:37
git_branch master alejandro.gonzalez/fix-stack-inconsistency-iast
git_commit_date 1730827777 1730877317
git_commit_sha 21c0b2d 1d9c6df
release_version 1.43.0-SNAPSHOT~21c0b2df63 1.43.0-SNAPSHOT~1d9c6df96f
start_time 2024-11-06T07:26:27 2024-11-06T07:33:24
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1730878768 1730878768
ci_job_id 696791988 696791988
ci_pipeline_id 48320058 48320058
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63
    dateFormat X
    axisFormat %s
section baseline
no_agent (375.539 µs) : 356, 395
.   : milestone, 376,
iast (506.147 µs) : 484, 529
.   : milestone, 506,
iast_FULL (651.718 µs) : 630, 673
.   : milestone, 652,
iast_GLOBAL (515.554 µs) : 494, 537
.   : milestone, 516,
iast_HARDCODED_SECRET_DISABLED (490.127 µs) : 469, 511
.   : milestone, 490,
iast_INACTIVE (452.273 µs) : 431, 473
.   : milestone, 452,
iast_TELEMETRY_OFF (484.869 µs) : 463, 507
.   : milestone, 485,
tracing (448.479 µs) : 428, 469
.   : milestone, 448,
section candidate
no_agent (376.682 µs) : 355, 398
.   : milestone, 377,
iast (491.853 µs) : 470, 513
.   : milestone, 492,
iast_FULL (649.515 µs) : 628, 671
.   : milestone, 650,
iast_GLOBAL (520.233 µs) : 499, 542
.   : milestone, 520,
iast_HARDCODED_SECRET_DISABLED (490.114 µs) : 469, 512
.   : milestone, 490,
iast_INACTIVE (456.04 µs) : 435, 477
.   : milestone, 456,
iast_TELEMETRY_OFF (477.647 µs) : 457, 498
.   : milestone, 478,
tracing (448.167 µs) : 428, 468
.   : milestone, 448,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 375.539 µs [356.146 µs, 394.932 µs] -
iast 506.147 µs [483.611 µs, 528.683 µs] 130.608 µs (34.8%)
iast_FULL 651.718 µs [630.296 µs, 673.14 µs] 276.179 µs (73.5%)
iast_GLOBAL 515.554 µs [493.693 µs, 537.415 µs] 140.015 µs (37.3%)
iast_HARDCODED_SECRET_DISABLED 490.127 µs [469.003 µs, 511.25 µs] 114.588 µs (30.5%)
iast_INACTIVE 452.273 µs [431.393 µs, 473.154 µs] 76.734 µs (20.4%)
iast_TELEMETRY_OFF 484.869 µs [463.049 µs, 506.689 µs] 109.33 µs (29.1%)
tracing 448.479 µs [427.741 µs, 469.217 µs] 72.94 µs (19.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 376.682 µs [355.214 µs, 398.149 µs] -
iast 491.853 µs [470.402 µs, 513.304 µs] 115.172 µs (30.6%)
iast_FULL 649.515 µs [627.96 µs, 671.071 µs] 272.834 µs (72.4%)
iast_GLOBAL 520.233 µs [498.581 µs, 541.886 µs] 143.552 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED 490.114 µs [468.501 µs, 511.727 µs] 113.433 µs (30.1%)
iast_INACTIVE 456.04 µs [434.696 µs, 477.385 µs] 79.359 µs (21.1%)
iast_TELEMETRY_OFF 477.647 µs [456.838 µs, 498.455 µs] 100.965 µs (26.8%)
tracing 448.167 µs [427.959 µs, 468.374 µs] 71.485 µs (19.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.347 ms) : 1328, 1366
.   : milestone, 1347,
appsec (1.732 ms) : 1709, 1754
.   : milestone, 1732,
appsec_no_iast (1.73 ms) : 1705, 1755
.   : milestone, 1730,
iast (1.486 ms) : 1462, 1509
.   : milestone, 1486,
profiling (1.49 ms) : 1467, 1513
.   : milestone, 1490,
tracing (1.475 ms) : 1450, 1501
.   : milestone, 1475,
section candidate
no_agent (1.351 ms) : 1332, 1371
.   : milestone, 1351,
appsec (1.74 ms) : 1715, 1766
.   : milestone, 1740,
appsec_no_iast (1.74 ms) : 1715, 1765
.   : milestone, 1740,
iast (1.501 ms) : 1478, 1524
.   : milestone, 1501,
profiling (1.485 ms) : 1462, 1508
.   : milestone, 1485,
tracing (1.467 ms) : 1442, 1491
.   : milestone, 1467,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.328 ms, 1.366 ms] -
appsec 1.732 ms [1.709 ms, 1.754 ms] 384.354 µs (28.5%)
appsec_no_iast 1.73 ms [1.705 ms, 1.755 ms] 382.539 µs (28.4%)
iast 1.486 ms [1.462 ms, 1.509 ms] 138.555 µs (10.3%)
profiling 1.49 ms [1.467 ms, 1.513 ms] 142.774 µs (10.6%)
tracing 1.475 ms [1.45 ms, 1.501 ms] 128.285 µs (9.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.351 ms [1.332 ms, 1.371 ms] -
appsec 1.74 ms [1.715 ms, 1.766 ms] 389.209 µs (28.8%)
appsec_no_iast 1.74 ms [1.715 ms, 1.765 ms] 388.669 µs (28.8%)
iast 1.501 ms [1.478 ms, 1.524 ms] 149.986 µs (11.1%)
profiling 1.485 ms [1.462 ms, 1.508 ms] 133.77 µs (9.9%)
tracing 1.467 ms [1.442 ms, 1.491 ms] 115.289 µs (8.5%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master alejandro.gonzalez/fix-stack-inconsistency-iast
git_commit_date 1730827777 1730877317
git_commit_sha 21c0b2d 1d9c6df
release_version 1.43.0-SNAPSHOT~21c0b2df63 1.43.0-SNAPSHOT~1d9c6df96f
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1730879275 1730879275
ci_job_id 696791989 696791989
ci_pipeline_id 48320058 48320058
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.185 s) : 15185000, 15185000
.   : milestone, 15185000,
appsec (15.093 s) : 15093000, 15093000
.   : milestone, 15093000,
iast (18.717 s) : 18717000, 18717000
.   : milestone, 18717000,
iast_GLOBAL (17.557 s) : 17557000, 17557000
.   : milestone, 17557000,
profiling (15.147 s) : 15147000, 15147000
.   : milestone, 15147000,
tracing (15.109 s) : 15109000, 15109000
.   : milestone, 15109000,
section candidate
no_agent (15.688 s) : 15688000, 15688000
.   : milestone, 15688000,
appsec (15.3 s) : 15300000, 15300000
.   : milestone, 15300000,
iast (18.453 s) : 18453000, 18453000
.   : milestone, 18453000,
iast_GLOBAL (18.154 s) : 18154000, 18154000
.   : milestone, 18154000,
profiling (14.955 s) : 14955000, 14955000
.   : milestone, 14955000,
tracing (15.155 s) : 15155000, 15155000
.   : milestone, 15155000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.185 s [15.185 s, 15.185 s] -
appsec 15.093 s [15.093 s, 15.093 s] -92.0 ms (-0.6%)
iast 18.717 s [18.717 s, 18.717 s] 3.532 s (23.3%)
iast_GLOBAL 17.557 s [17.557 s, 17.557 s] 2.372 s (15.6%)
profiling 15.147 s [15.147 s, 15.147 s] -38.0 ms (-0.3%)
tracing 15.109 s [15.109 s, 15.109 s] -76.0 ms (-0.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.688 s [15.688 s, 15.688 s] -
appsec 15.3 s [15.3 s, 15.3 s] -388.0 ms (-2.5%)
iast 18.453 s [18.453 s, 18.453 s] 2.765 s (17.6%)
iast_GLOBAL 18.154 s [18.154 s, 18.154 s] 2.466 s (15.7%)
profiling 14.955 s [14.955 s, 14.955 s] -733.0 ms (-4.7%)
tracing 15.155 s [15.155 s, 15.155 s] -533.0 ms (-3.4%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.43.0-SNAPSHOT~1d9c6df96f, baseline=1.43.0-SNAPSHOT~21c0b2df63
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.459 ms) : 1448, 1471
.   : milestone, 1459,
appsec (2.316 ms) : 2275, 2357
.   : milestone, 2316,
iast (2.071 ms) : 2019, 2124
.   : milestone, 2071,
iast_GLOBAL (2.124 ms) : 2071, 2177
.   : milestone, 2124,
profiling (1.939 ms) : 1897, 1980
.   : milestone, 1939,
tracing (1.912 ms) : 1872, 1952
.   : milestone, 1912,
section candidate
no_agent (1.465 ms) : 1454, 1477
.   : milestone, 1465,
appsec (2.321 ms) : 2280, 2361
.   : milestone, 2321,
iast (2.072 ms) : 2020, 2125
.   : milestone, 2072,
iast_GLOBAL (2.124 ms) : 2071, 2177
.   : milestone, 2124,
profiling (1.928 ms) : 1887, 1969
.   : milestone, 1928,
tracing (1.914 ms) : 1874, 1954
.   : milestone, 1914,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.459 ms [1.448 ms, 1.471 ms] -
appsec 2.316 ms [2.275 ms, 2.357 ms] 856.672 µs (58.7%)
iast 2.071 ms [2.019 ms, 2.124 ms] 611.839 µs (41.9%)
iast_GLOBAL 2.124 ms [2.071 ms, 2.177 ms] 664.891 µs (45.6%)
profiling 1.939 ms [1.897 ms, 1.98 ms] 479.156 µs (32.8%)
tracing 1.912 ms [1.872 ms, 1.952 ms] 452.505 µs (31.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.465 ms [1.454 ms, 1.477 ms] -
appsec 2.321 ms [2.28 ms, 2.361 ms] 855.217 µs (58.4%)
iast 2.072 ms [2.02 ms, 2.125 ms] 607.014 µs (41.4%)
iast_GLOBAL 2.124 ms [2.071 ms, 2.177 ms] 658.59 µs (44.9%)
profiling 1.928 ms [1.887 ms, 1.969 ms] 462.604 µs (31.6%)
tracing 1.914 ms [1.874 ms, 1.954 ms] 448.181 µs (30.6%)

@pr-commenter
Copy link

pr-commenter bot commented Nov 5, 2024 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1730816868 1730817248
end_time 2024-11-05T14:29:04 2024-11-05T14:35:24
git_branch master alejandro.gonzalez/fix-stack-inconsistency-iast
git_commit_sha fe6442b 596370a
start_time 2024-11-05T14:27:49 2024-11-05T14:34:09
See matching parameters
Baseline Candidate
ci_job_id 695703904 695703904
ci_pipeline_id 48258014 48258014
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1730816218 1730816218

Summary

Found 4 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 8 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop better
[-378.573µs; -227.241µs] or [-3.638%; -2.184%]		 better
[-403.022µs; -233.894µs] or [-3.811%; -2.212%]		 better
[-398.862µs; -198.660µs] or [-3.737%; -1.862%]		 unsure
[-761.326µs; -75.204µs] or [-6.828%; -0.675%]		 better
[+1.352op/s; +4.039op/s] or [+1.447%; +4.322%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-34.766µs; +51.554µs] or [-12.973%; +19.238%]		 unstable
[-48.444µs; +65.385µs] or [-15.725%; +21.224%]		 unstable
[-59.922µs; +80.703µs] or [-18.558%; +24.994%]		 unstable
[-264.960µs; +965.248µs] or [-42.626%; +155.284%]		 unstable
[-153.511op/s; +153.511op/s] or [-5.833%; +5.833%]
scenario:basic same same unstable
[-13.626µs; +19.855µs] or [-4.354%; +6.344%]		 unstable
[-44.169µs; +90.845µs] or [-8.138%; +16.738%]		 unstable
[-188.272op/s; +188.272op/s] or [-7.343%; +7.343%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (308.064 µs) : 266, 350
.   : milestone, 308,
basic (301.462 µs) : 293, 310
.   : milestone, 301,
loop (10.575 ms) : 10470, 10681
.   : milestone, 10575,
section candidate
noprobe (316.535 µs) : 252, 381
.   : milestone, 317,
basic (304.377 µs) : 289, 320
.   : milestone, 304,
loop (10.257 ms) : 10222, 10291
.   : milestone, 10257,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 308.064 µs [265.925 µs, 350.203 µs]
basic 301.462 µs [292.75 µs, 310.174 µs]
loop 10.575 ms [10.47 ms, 10.681 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 316.535 µs [252.39 µs, 380.68 µs]
basic 304.377 µs [288.563 µs, 320.191 µs]
loop 10.257 ms [10.222 ms, 10.291 ms]

@jandro996 jandro996 marked this pull request as ready for review November 5, 2024 13:02
@jandro996 jandro996 requested review from a team as code owners November 5, 2024 13:02
@jandro996 jandro996 requested review from evanchooly and removed request for a team November 5, 2024 13:02
smola
smola approved these changes Nov 5, 2024
View reviewed changes
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me for RASP and IAST, but needs a review by the DI team.

dd-java-agent/agent-debugger/build.gradle Outdated Show resolved Hide resolved
@smola
Copy link
Member

smola commented Nov 5, 2024

Also note the performance regression in debugger benchmarks.

@jandro996 jandro996 requested review from smola and jpbempel November 5, 2024 14:25
evanchooly
evanchooly reviewed Nov 5, 2024
View reviewed changes
dd-java-agent/agent-debugger/build.gradle Outdated Show resolved Hide resolved
dd-java-agent/agent-debugger/src/test/java/foo/bar/debugger/origin/CodeOriginTest.java Outdated Show resolved Hide resolved
internal-api/src/main/java/datadog/trace/util/stacktrace/AbstractStackWalker.java Outdated Show resolved Hide resolved
internal-api/src/test/java/datadog/trace/util/stacktrace/StackUtilsTest.java
@Test
public void test_generateUserCodeStackTrace() {
List<StackTraceFrame> userCodeStack = StackUtils.generateUserCodeStackTrace();
private static Stream<Arguments> test_generateUserCodeStackTrace_Params() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test only seems to care about the junit frames being 0 or not so I'm not sure the filtering change is even necessary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My intention with this test is cover the StackUtils#generateUserCodeStackTrace parametrization, more than the AbstractStackWalker#isNotDatadogTraceStackElement that is covered by other tests. But I can try to add more specific if you feel that is necessary

@jandro996 jandro996 requested a review from evanchooly November 6, 2024 09:08
smola
smola approved these changes Nov 7, 2024
View reviewed changes
Copy link
Member

@smola smola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now, broader impact is gone.

@jandro996 jandro996 merged commit 15e5861 into master Nov 7, 2024
97 of 100 checks passed
@jandro996 jandro996 deleted the alejandro.gonzalez/fix-stack-inconsistency-iast branch November 7, 2024 10:05
@github-actions github-actions bot added this to the 1.43.0 milestone Nov 7, 2024
jandro996 added a commit that referenced this pull request Nov 11, 2024
@jandro996
Fix stack trace inconsistency between excluded frames in vulnerabilit…
fec1c89 
…y location and metastruct stack trace (#7865)

What Does This Do
Parametrize StackUtils#generateUserCodeStackTrace with a predicate
Add AbstractStackWalker#isNotDatadogTraceStackElement as default filter
Update isNotDatadogTraceStackElement to also filter com.datadog.appsec

Motivation
We are filtering different stack frames according to their class to calculate the vulnerability location and the vulnerability stack trace that is reported via meta struct
@jandro996 jandro996 added the comp: asm iast Application Security Management (IAST) label Nov 19, 2024
@github-actions GitHub Actions
Copy link
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@manuel-alvarez-alvarez manuel-alvarez-alvarez manuel-alvarez-alvarez approved these changes

@smola smola smola approved these changes

@jpbempel jpbempel Awaiting requested review from jpbempel

@evanchooly evanchooly Awaiting requested review from evanchooly

Assignees
No one assigned
Labels
comp: asm iast Application Security Management (IAST)
Projects
None yet
Milestone
1.43.0
Development

Successfully merging this pull request may close these issues.
5 participants
@jandro996 @smola @evanchooly @jpbempel @manuel-alvarez-alvarez