Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix message for snapshots with evaluation errors #7653

Merged
merged 2 commits into from
Sep 24, 2024
Merged

Fix message for snapshots with evaluation errors #7653

merged 2 commits into from
Sep 24, 2024
+53 −18

Conversation

jpbempel
Copy link
Member

@jpbempel jpbempel commented Sep 20, 2024 •
edited by jira bot
Loading

What Does This Do

set the message in snapshot with the content of the first evaluation errors

Motivation

Additional Notes

Contributor Checklist

Jira ticket: DEBUG-2704

@jpbempel jpbempel added the comp: debugger Dynamic Instrumentation label Sep 20, 2024
@jpbempel jpbempel requested a review from a team as a code owner September 20, 2024 09:21
@jpbempel jpbempel requested review from ojung and cimi and removed request for a team September 20, 2024 09:21
@pr-commenter
Copy link

pr-commenter bot commented Sep 20, 2024 •
edited
Loading

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1727082083 1727082457
end_time 2024-09-23T09:02:38 2024-09-23T09:08:53
git_branch master jpbempel/fix-eval-error-msg
git_commit_sha 5ca0c5b 0820c43
start_time 2024-09-23T09:01:24 2024-09-23T09:07:38
See matching parameters
Baseline Candidate
ci_job_id 647781268 647781268
ci_pipeline_id 44888870 44888870
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1727081407 1727081407

Summary

Found 0 performance improvements and 5 performance regressions! Performance is the same for 1 metrics, 9 unstable metrics.

scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:loop worse
[+619.108µs; +717.101µs] or [+6.129%; +7.099%]		 worse
[+610.369µs; +736.586µs] or [+5.942%; +7.171%]		 worse
[+568.934µs; +749.912µs] or [+5.489%; +7.235%]		 worse
[+350.441µs; +875.096µs] or [+3.261%; +8.144%]		 worse
[-6.803op/s; -4.509op/s] or [-7.076%; -4.689%]
See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-34.154µs; +86.572µs] or [-13.034%; +33.037%]		 unstable
[-40.869µs; +93.861µs] or [-13.624%; +31.290%]		 unstable
[-48.729µs; +105.419µs] or [-15.541%; +33.620%]		 unstable
[-398.608µs; +1194.006µs] or [-65.983%; +197.649%]		 unstable
[-182.496op/s; +112.321op/s] or [-6.844%; +4.212%]
scenario:basic same unstable
[-9.812µs; +45.383µs] or [-3.278%; +15.160%]		 unstable
[-13.115µs; +53.164µs] or [-4.244%; +17.206%]		 unstable
[-261.344µs; +785.775µs] or [-48.357%; +145.393%]		 unstable
[-322.281op/s; +59.123op/s] or [-12.247%; +2.247%]
Request duration reports for reports 
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (299.973 µs) : 265, 335
.   : milestone, 300,
basic (299.357 µs) : 290, 308
.   : milestone, 299,
loop (10.272 ms) : 10230, 10315
.   : milestone, 10272,
section candidate
noprobe (326.469 µs) : 243, 410
.   : milestone, 326,
basic (317.142 µs) : 280, 354
.   : milestone, 317,
loop (10.946 ms) : 10875, 11017
.   : milestone, 10946,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 299.973 µs [264.954 µs, 334.992 µs]
basic 299.357 µs [290.398 µs, 308.315 µs]
loop 10.272 ms [10.23 ms, 10.315 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 326.469 µs [242.645 µs, 410.293 µs]
basic 317.142 µs [280.074 µs, 354.21 µs]
loop 10.946 ms [10.875 ms, 11.017 ms]

@pr-commenter
Copy link

pr-commenter bot commented Sep 20, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-eval-error-msg
git_commit_date 1727079626 1727081407
git_commit_sha 5ca0c5b 0820c43
release_version 1.40.0-SNAPSHOT~5ca0c5bab6 1.40.0-SNAPSHOT~0820c433b5
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1727083906 1727083906
ci_job_id 647781262 647781262
ci_pipeline_id 44888870 44888870
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 47 metrics, 15 unstable metrics.

scenario Δ mean execution_time candidate mean execution_time baseline mean execution_time
scenario:startup:insecure-bank:iast_TELEMETRY_OFF:Telemetry worse
[+231.999µs; +538.006µs] or [+3.341%; +7.747%]		 7.330ms 6.945ms
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.069 s) : 0, 1069353
Total [baseline] (8.545 s) : 0, 8545273
Agent [candidate] (1.066 s) : 0, 1066291
Total [candidate] (8.544 s) : 0, 8544403
section iast
Agent [baseline] (1.204 s) : 0, 1204313
Total [baseline] (9.071 s) : 0, 9070558
Agent [candidate] (1.192 s) : 0, 1191962
Total [candidate] (9.007 s) : 0, 9006789
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.202 s) : 0, 1201813
Total [baseline] (9.01 s) : 0, 9009532
Agent [candidate] (1.194 s) : 0, 1194458
Total [candidate] (8.993 s) : 0, 8992798
section iast_TELEMETRY_OFF
Agent [baseline] (1.19 s) : 0, 1189764
Total [baseline] (8.992 s) : 0, 8992463
Agent [candidate] (1.2 s) : 0, 1199605
Total [candidate] (9.028 s) : 0, 9028317
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.069 s -
Agent iast 1.204 s 134.959 ms (12.6%)
Agent iast_HARDCODED_SECRET_DISABLED 1.202 s 132.46 ms (12.4%)
Agent iast_TELEMETRY_OFF 1.19 s 120.411 ms (11.3%)
Total tracing 8.545 s -
Total iast 9.071 s 525.285 ms (6.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.01 s 464.259 ms (5.4%)
Total iast_TELEMETRY_OFF 8.992 s 447.19 ms (5.2%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.192 s 125.671 ms (11.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.194 s 128.168 ms (12.0%)
Agent iast_TELEMETRY_OFF 1.2 s 133.315 ms (12.5%)
Total tracing 8.544 s -
Total iast 9.007 s 462.386 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 8.993 s 448.395 ms (5.2%)
Total iast_TELEMETRY_OFF 9.028 s 483.914 ms (5.7%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.247 ms) : 0, 682247
BytebuddyAgent [candidate] (680.175 ms) : 0, 680175
GlobalTracer [baseline] (311.147 ms) : 0, 311147
GlobalTracer [candidate] (310.278 ms) : 0, 310278
AppSec [baseline] (53.867 ms) : 0, 53867
AppSec [candidate] (53.789 ms) : 0, 53789
Remote Config [baseline] (684.429 µs) : 0, 684
Remote Config [candidate] (667.371 µs) : 0, 667
Telemetry [baseline] (7.671 ms) : 0, 7671
Telemetry [candidate] (7.639 ms) : 0, 7639
section iast
BytebuddyAgent [baseline] (800.286 ms) : 0, 800286
BytebuddyAgent [candidate] (792.317 ms) : 0, 792317
GlobalTracer [baseline] (302.692 ms) : 0, 302692
GlobalTracer [candidate] (298.913 ms) : 0, 298913
AppSec [baseline] (56.385 ms) : 0, 56385
AppSec [candidate] (52.761 ms) : 0, 52761
IAST [baseline] (23.244 ms) : 0, 23244
IAST [candidate] (26.204 ms) : 0, 26204
Remote Config [baseline] (621.148 µs) : 0, 621
Remote Config [candidate] (603.626 µs) : 0, 604
Telemetry [baseline] (7.176 ms) : 0, 7176
Telemetry [candidate] (7.359 ms) : 0, 7359
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (799.72 ms) : 0, 799720
BytebuddyAgent [candidate] (794.201 ms) : 0, 794201
GlobalTracer [baseline] (301.326 ms) : 0, 301326
GlobalTracer [candidate] (299.566 ms) : 0, 299566
AppSec [baseline] (56.15 ms) : 0, 56150
AppSec [candidate] (55.339 ms) : 0, 55339
IAST [baseline] (22.177 ms) : 0, 22177
IAST [candidate] (23.558 ms) : 0, 23558
Remote Config [baseline] (629.832 µs) : 0, 630
Remote Config [candidate] (595.663 µs) : 0, 596
Telemetry [baseline] (7.923 ms) : 0, 7923
Telemetry [candidate] (7.419 ms) : 0, 7419
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (790.484 ms) : 0, 790484
BytebuddyAgent [candidate] (796.569 ms) : 0, 796569
GlobalTracer [baseline] (299.254 ms) : 0, 299254
GlobalTracer [candidate] (302.08 ms) : 0, 302080
AppSec [baseline] (54.679 ms) : 0, 54679
AppSec [candidate] (55.631 ms) : 0, 55631
IAST [baseline] (23.226 ms) : 0, 23226
IAST [candidate] (23.482 ms) : 0, 23482
Remote Config [baseline] (1.424 ms) : 0, 1424
Remote Config [candidate] (630.002 µs) : 0, 630
Telemetry [baseline] (6.945 ms) : 0, 6945
Telemetry [candidate] (7.33 ms) : 0, 7330
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.069 s) : 0, 1068758
Total [baseline] (10.44 s) : 0, 10439556
Agent [candidate] (1.065 s) : 0, 1065029
Total [candidate] (10.359 s) : 0, 10358621
section appsec
Agent [baseline] (1.206 s) : 0, 1205833
Total [baseline] (10.63 s) : 0, 10629567
Agent [candidate] (1.205 s) : 0, 1205171
Total [candidate] (10.635 s) : 0, 10635369
section iast
Agent [baseline] (1.201 s) : 0, 1200904
Total [baseline] (10.791 s) : 0, 10791207
Agent [candidate] (1.197 s) : 0, 1196785
Total [candidate] (10.8 s) : 0, 10799799
section profiling
Agent [baseline] (1.266 s) : 0, 1266473
Total [baseline] (10.593 s) : 0, 10592919
Agent [candidate] (1.267 s) : 0, 1267136
Total [candidate] (10.621 s) : 0, 10621479
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.069 s -
Agent appsec 1.206 s 137.075 ms (12.8%)
Agent iast 1.201 s 132.146 ms (12.4%)
Agent profiling 1.266 s 197.715 ms (18.5%)
Total tracing 10.44 s -
Total appsec 10.63 s 190.012 ms (1.8%)
Total iast 10.791 s 351.652 ms (3.4%)
Total profiling 10.593 s 153.364 ms (1.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent appsec 1.205 s 140.142 ms (13.2%)
Agent iast 1.197 s 131.756 ms (12.4%)
Agent profiling 1.267 s 202.108 ms (19.0%)
Total tracing 10.359 s -
Total appsec 10.635 s 276.748 ms (2.7%)
Total iast 10.8 s 441.177 ms (4.3%)
Total profiling 10.621 s 262.858 ms (2.5%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (682.303 ms) : 0, 682303
BytebuddyAgent [candidate] (679.287 ms) : 0, 679287
GlobalTracer [baseline] (310.616 ms) : 0, 310616
GlobalTracer [candidate] (310.161 ms) : 0, 310161
AppSec [baseline] (53.851 ms) : 0, 53851
AppSec [candidate] (53.649 ms) : 0, 53649
Remote Config [baseline] (663.612 µs) : 0, 664
Remote Config [candidate] (667.19 µs) : 0, 667
Telemetry [baseline] (7.63 ms) : 0, 7630
Telemetry [candidate] (7.584 ms) : 0, 7584
section appsec
BytebuddyAgent [baseline] (704.849 ms) : 0, 704849
BytebuddyAgent [candidate] (704.301 ms) : 0, 704301
GlobalTracer [baseline] (304.738 ms) : 0, 304738
GlobalTracer [candidate] (304.427 ms) : 0, 304427
AppSec [baseline] (164.114 ms) : 0, 164114
AppSec [candidate] (163.881 ms) : 0, 163881
IAST [baseline] (19.185 ms) : 0, 19185
IAST [candidate] (19.94 ms) : 0, 19940
Remote Config [baseline] (645.621 µs) : 0, 646
Remote Config [candidate] (644.921 µs) : 0, 645
Telemetry [baseline] (8.154 ms) : 0, 8154
Telemetry [candidate] (8.203 ms) : 0, 8203
section iast
BytebuddyAgent [baseline] (799.261 ms) : 0, 799261
BytebuddyAgent [candidate] (795.917 ms) : 0, 795917
GlobalTracer [baseline] (301.008 ms) : 0, 301008
GlobalTracer [candidate] (300.232 ms) : 0, 300232
AppSec [baseline] (55.112 ms) : 0, 55112
AppSec [candidate] (54.989 ms) : 0, 54989
IAST [baseline] (23.188 ms) : 0, 23188
IAST [candidate] (23.775 ms) : 0, 23775
Remote Config [baseline] (640.802 µs) : 0, 641
Remote Config [candidate] (628.875 µs) : 0, 629
Telemetry [baseline] (7.827 ms) : 0, 7827
Telemetry [candidate] (7.433 ms) : 0, 7433
section profiling
BytebuddyAgent [baseline] (673.659 ms) : 0, 673659
BytebuddyAgent [candidate] (674.125 ms) : 0, 674125
GlobalTracer [baseline] (394.642 ms) : 0, 394642
GlobalTracer [candidate] (396.603 ms) : 0, 396603
AppSec [baseline] (54.864 ms) : 0, 54864
AppSec [candidate] (54.305 ms) : 0, 54305
Remote Config [baseline] (652.878 µs) : 0, 653
Remote Config [candidate] (652.276 µs) : 0, 652
Telemetry [baseline] (7.503 ms) : 0, 7503
Telemetry [candidate] (7.416 ms) : 0, 7416
ProfilingAgent [baseline] (96.53 ms) : 0, 96530
ProfilingAgent [candidate] (95.833 ms) : 0, 95833
Profiling [baseline] (96.553 ms) : 0, 96553
Profiling [candidate] (95.857 ms) : 0, 95857
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-23T09:02:17 2024-09-23T09:09:07
git_branch master jpbempel/fix-eval-error-msg
git_commit_date 1727079626 1727081407
git_commit_sha 5ca0c5b 0820c43
release_version 1.40.0-SNAPSHOT~5ca0c5bab6 1.40.0-SNAPSHOT~0820c433b5
start_time 2024-09-23T09:02:03 2024-09-23T09:08:54
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1727082893 1727082893
ci_job_id 647781263 647781263
ci_pipeline_id 44888870 44888870
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 16 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.342 ms) : 1323, 1362
.   : milestone, 1342,
appsec (1.725 ms) : 1701, 1749
.   : milestone, 1725,
appsec_no_iast (1.706 ms) : 1681, 1731
.   : milestone, 1706,
iast (1.477 ms) : 1455, 1499
.   : milestone, 1477,
profiling (1.515 ms) : 1489, 1541
.   : milestone, 1515,
tracing (1.439 ms) : 1415, 1463
.   : milestone, 1439,
section candidate
no_agent (1.344 ms) : 1325, 1364
.   : milestone, 1344,
appsec (1.724 ms) : 1700, 1747
.   : milestone, 1724,
appsec_no_iast (1.726 ms) : 1702, 1749
.   : milestone, 1726,
iast (1.472 ms) : 1449, 1494
.   : milestone, 1472,
profiling (1.491 ms) : 1466, 1517
.   : milestone, 1491,
tracing (1.478 ms) : 1452, 1503
.   : milestone, 1478,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.342 ms [1.323 ms, 1.362 ms] -
appsec 1.725 ms [1.701 ms, 1.749 ms] 382.473 µs (28.5%)
appsec_no_iast 1.706 ms [1.681 ms, 1.731 ms] 363.759 µs (27.1%)
iast 1.477 ms [1.455 ms, 1.499 ms] 134.766 µs (10.0%)
profiling 1.515 ms [1.489 ms, 1.541 ms] 172.663 µs (12.9%)
tracing 1.439 ms [1.415 ms, 1.463 ms] 96.7 µs (7.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.344 ms [1.325 ms, 1.364 ms] -
appsec 1.724 ms [1.7 ms, 1.747 ms] 379.281 µs (28.2%)
appsec_no_iast 1.726 ms [1.702 ms, 1.749 ms] 381.328 µs (28.4%)
iast 1.472 ms [1.449 ms, 1.494 ms] 127.126 µs (9.5%)
profiling 1.491 ms [1.466 ms, 1.517 ms] 146.998 µs (10.9%)
tracing 1.478 ms [1.452 ms, 1.503 ms] 133.485 µs (9.9%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6
    dateFormat X
    axisFormat %s
section baseline
no_agent (369.801 µs) : 350, 390
.   : milestone, 370,
iast (484.694 µs) : 463, 506
.   : milestone, 485,
iast_FULL (555.428 µs) : 534, 577
.   : milestone, 555,
iast_GLOBAL (503.414 µs) : 482, 525
.   : milestone, 503,
iast_HARDCODED_SECRET_DISABLED (478.483 µs) : 457, 500
.   : milestone, 478,
iast_INACTIVE (444.653 µs) : 424, 465
.   : milestone, 445,
iast_TELEMETRY_OFF (470.91 µs) : 449, 493
.   : milestone, 471,
tracing (443.967 µs) : 424, 464
.   : milestone, 444,
section candidate
no_agent (370.347 µs) : 350, 391
.   : milestone, 370,
iast (488.044 µs) : 466, 510
.   : milestone, 488,
iast_FULL (549.706 µs) : 529, 571
.   : milestone, 550,
iast_GLOBAL (511.152 µs) : 488, 534
.   : milestone, 511,
iast_HARDCODED_SECRET_DISABLED (480.994 µs) : 460, 502
.   : milestone, 481,
iast_INACTIVE (443.526 µs) : 423, 464
.   : milestone, 444,
iast_TELEMETRY_OFF (472.832 µs) : 450, 496
.   : milestone, 473,
tracing (441.183 µs) : 421, 462
.   : milestone, 441,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 369.801 µs [350.086 µs, 389.515 µs] -
iast 484.694 µs [463.355 µs, 506.033 µs] 114.893 µs (31.1%)
iast_FULL 555.428 µs [534.033 µs, 576.823 µs] 185.627 µs (50.2%)
iast_GLOBAL 503.414 µs [482.293 µs, 524.536 µs] 133.613 µs (36.1%)
iast_HARDCODED_SECRET_DISABLED 478.483 µs [457.297 µs, 499.668 µs] 108.682 µs (29.4%)
iast_INACTIVE 444.653 µs [424.007 µs, 465.299 µs] 74.852 µs (20.2%)
iast_TELEMETRY_OFF 470.91 µs [448.588 µs, 493.231 µs] 101.109 µs (27.3%)
tracing 443.967 µs [423.604 µs, 464.331 µs] 74.167 µs (20.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 370.347 µs [350.169 µs, 390.526 µs] -
iast 488.044 µs [465.724 µs, 510.364 µs] 117.697 µs (31.8%)
iast_FULL 549.706 µs [528.501 µs, 570.912 µs] 179.359 µs (48.4%)
iast_GLOBAL 511.152 µs [487.895 µs, 534.409 µs] 140.804 µs (38.0%)
iast_HARDCODED_SECRET_DISABLED 480.994 µs [459.853 µs, 502.135 µs] 110.647 µs (29.9%)
iast_INACTIVE 443.526 µs [423.049 µs, 464.003 µs] 73.179 µs (19.8%)
iast_TELEMETRY_OFF 472.832 µs [449.802 µs, 495.862 µs] 102.484 µs (27.7%)
tracing 441.183 µs [420.715 µs, 461.652 µs] 70.836 µs (19.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master jpbempel/fix-eval-error-msg
git_commit_date 1727079626 1727081407
git_commit_sha 5ca0c5b 0820c43
release_version 1.40.0-SNAPSHOT~5ca0c5bab6 1.40.0-SNAPSHOT~0820c433b5
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1727083435 1727083435
ci_job_id 647781264 647781264
ci_pipeline_id 44888870 44888870
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.759 s) : 14759000, 14759000
.   : milestone, 14759000,
appsec (15.316 s) : 15316000, 15316000
.   : milestone, 15316000,
iast (18.796 s) : 18796000, 18796000
.   : milestone, 18796000,
iast_GLOBAL (18.162 s) : 18162000, 18162000
.   : milestone, 18162000,
profiling (15.365 s) : 15365000, 15365000
.   : milestone, 15365000,
tracing (15.403 s) : 15403000, 15403000
.   : milestone, 15403000,
section candidate
no_agent (15.403 s) : 15403000, 15403000
.   : milestone, 15403000,
appsec (15.385 s) : 15385000, 15385000
.   : milestone, 15385000,
iast (19.051 s) : 19051000, 19051000
.   : milestone, 19051000,
iast_GLOBAL (18.195 s) : 18195000, 18195000
.   : milestone, 18195000,
profiling (15.427 s) : 15427000, 15427000
.   : milestone, 15427000,
tracing (15.332 s) : 15332000, 15332000
.   : milestone, 15332000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.759 s [14.759 s, 14.759 s] -
appsec 15.316 s [15.316 s, 15.316 s] 557.0 ms (3.8%)
iast 18.796 s [18.796 s, 18.796 s] 4.037 s (27.4%)
iast_GLOBAL 18.162 s [18.162 s, 18.162 s] 3.403 s (23.1%)
profiling 15.365 s [15.365 s, 15.365 s] 606.0 ms (4.1%)
tracing 15.403 s [15.403 s, 15.403 s] 644.0 ms (4.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.403 s [15.403 s, 15.403 s] -
appsec 15.385 s [15.385 s, 15.385 s] -18.0 ms (-0.1%)
iast 19.051 s [19.051 s, 19.051 s] 3.648 s (23.7%)
iast_GLOBAL 18.195 s [18.195 s, 18.195 s] 2.792 s (18.1%)
profiling 15.427 s [15.427 s, 15.427 s] 24.0 ms (0.2%)
tracing 15.332 s [15.332 s, 15.332 s] -71.0 ms (-0.5%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~0820c433b5, baseline=1.40.0-SNAPSHOT~5ca0c5bab6
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.472 ms) : 1461, 1484
.   : milestone, 1472,
appsec (2.322 ms) : 2280, 2363
.   : milestone, 2322,
iast (2.064 ms) : 2013, 2115
.   : milestone, 2064,
iast_GLOBAL (2.097 ms) : 2046, 2148
.   : milestone, 2097,
profiling (1.951 ms) : 1909, 1993
.   : milestone, 1951,
tracing (1.906 ms) : 1867, 1945
.   : milestone, 1906,
section candidate
no_agent (1.462 ms) : 1451, 1473
.   : milestone, 1462,
appsec (2.337 ms) : 2295, 2379
.   : milestone, 2337,
iast (2.062 ms) : 2010, 2113
.   : milestone, 2062,
iast_GLOBAL (2.11 ms) : 2058, 2162
.   : milestone, 2110,
profiling (1.946 ms) : 1903, 1988
.   : milestone, 1946,
tracing (1.912 ms) : 1873, 1952
.   : milestone, 1912,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.472 ms [1.461 ms, 1.484 ms] -
appsec 2.322 ms [2.28 ms, 2.363 ms] 849.648 µs (57.7%)
iast 2.064 ms [2.013 ms, 2.115 ms] 591.607 µs (40.2%)
iast_GLOBAL 2.097 ms [2.046 ms, 2.148 ms] 624.662 µs (42.4%)
profiling 1.951 ms [1.909 ms, 1.993 ms] 478.712 µs (32.5%)
tracing 1.906 ms [1.867 ms, 1.945 ms] 434.102 µs (29.5%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.462 ms [1.451 ms, 1.473 ms] -
appsec 2.337 ms [2.295 ms, 2.379 ms] 874.971 µs (59.8%)
iast 2.062 ms [2.01 ms, 2.113 ms] 599.649 µs (41.0%)
iast_GLOBAL 2.11 ms [2.058 ms, 2.162 ms] 648.266 µs (44.3%)
profiling 1.946 ms [1.903 ms, 1.988 ms] 483.494 µs (33.1%)
tracing 1.912 ms [1.873 ms, 1.952 ms] 450.376 µs (30.8%)

shatzi
shatzi reviewed Sep 20, 2024
View reviewed changes
Copy link
Contributor

@shatzi shatzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is a bug here... please check that setEntry/setExit works

dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/LogProbe.java Outdated
CapturedContext context, Snapshot snapshot, LogStatus status) {
if (context.getCapturedThrowable() != null) {
// report also uncaught exception
snapshot.setEntry(context);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we always call snapshot.setEntry here... I think this is a bug?

Do we have a test to check this ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, fixed

dd-java-agent/agent-debugger/src/main/java/com/datadog/debugger/probe/LogProbe.java Outdated
shouldCommit = true;
}
return shouldCommit;
}

private static void fillEvaluationErrors(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggest rename to popuplateSnapshotFromContextAndErrors

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed, but shorter

@jpbempel
Fix message for snapshots with evaluation errors
eb6ec9c 
set the message in snapshot with the content of the first evaluation
errors
@jpbempel
fix setExit
0820c43
@jpbempel jpbempel force-pushed the jpbempel/fix-eval-error-msg branch from deb74f3 to 0820c43 Compare September 23, 2024 08:50
@jpbempel jpbempel merged commit 67342bb into master Sep 24, 2024
90 of 92 checks passed
@jpbempel jpbempel deleted the jpbempel/fix-eval-error-msg branch September 24, 2024 07:39
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 24, 2024
shatzi pushed a commit that referenced this pull request Sep 25, 2024
@jpbempel @shatzi
Fix message for snapshots with evaluation errors (#7653)
fa39382 
set the message in snapshot with the content of the first evaluation
errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shatzi shatzi shatzi approved these changes

@ojung ojung Awaiting requested review from ojung ojung is a code owner automatically assigned from DataDog/debugger-java

@cimi cimi Awaiting requested review from cimi

Assignees
No one assigned
Labels
comp: debugger Dynamic Instrumentation
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@jpbempel @shatzi