Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Karate instrumentation to handle parallel tests execution #6924

Merged
merged 2 commits into from
Apr 17, 2024
Merged

Fix Karate instrumentation to handle parallel tests execution #6924

merged 2 commits into from
Apr 17, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Apr 17, 2024 •
edited by jira bot
Loading

What Does This Do

Fixes Karate testing framework instrumentation to handle parallel tests execution.

Additional Notes

There is a check in the instrumentation code that determines whether a Karate feature has any executable (i.e. not filtered with the tags configuration) scenarios.
This check was using scenarios iterator available in Karate's FeatureRuntime object.
It worked incorrectly in case of parallel tests execution, because in that case by the time "before feature" event was handled all of the elements from the scenario iterator were consumed by the parallel execution scheduler (and because iterators can only be traversed once).

Jira ticket: CIVIS-9799

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Apr 17, 2024
@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review April 17, 2024 10:52
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner April 17, 2024 10:52
@pr-commenter
Copy link

pr-commenter bot commented Apr 17, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/karate-parallel-execution-fix
git_commit_date 1713338255 1713350011
git_commit_sha 34e7066 6a6a136
release_version 1.33.0-SNAPSHOT~34e706697c 1.33.0-SNAPSHOT~6a6a136faa
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1713352480 1713352480
ci_job_id 488993767 488993767
ci_pipeline_id 32293301 32293301
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 16 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.076 s) : 0, 1076218
Total [baseline] (10.357 s) : 0, 10356825
Agent [candidate] (1.075 s) : 0, 1074571
Total [candidate] (10.337 s) : 0, 10336837
section appsec
Agent [baseline] (1.193 s) : 0, 1193112
Total [baseline] (10.495 s) : 0, 10495483
Agent [candidate] (1.187 s) : 0, 1187109
Total [candidate] (10.424 s) : 0, 10424167
section iast
Agent [baseline] (1.198 s) : 0, 1198106
Total [baseline] (10.708 s) : 0, 10708194
Agent [candidate] (1.205 s) : 0, 1204951
Total [candidate] (10.715 s) : 0, 10715161
section profiling
Agent [baseline] (1.283 s) : 0, 1283325
Total [baseline] (10.607 s) : 0, 10607037
Agent [candidate] (1.272 s) : 0, 1272377
Total [candidate] (10.68 s) : 0, 10680206
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.076 s -
Agent appsec 1.193 s 116.894 ms (10.9%)
Agent iast 1.198 s 121.887 ms (11.3%)
Agent profiling 1.283 s 207.106 ms (19.2%)
Total tracing 10.357 s -
Total appsec 10.495 s 138.658 ms (1.3%)
Total iast 10.708 s 351.369 ms (3.4%)
Total profiling 10.607 s 250.212 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.075 s -
Agent appsec 1.187 s 112.538 ms (10.5%)
Agent iast 1.205 s 130.38 ms (12.1%)
Agent profiling 1.272 s 197.806 ms (18.4%)
Total tracing 10.337 s -
Total appsec 10.424 s 87.329 ms (0.8%)
Total iast 10.715 s 378.324 ms (3.7%)
Total profiling 10.68 s 343.369 ms (3.3%) 
gantt
    title petclinic - break down per module: candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (674.241 ms) : 0, 674241
BytebuddyAgent [candidate] (672.771 ms) : 0, 672771
GlobalTracer [baseline] (309.822 ms) : 0, 309822
GlobalTracer [candidate] (309.605 ms) : 0, 309605
AppSec [baseline] (49.529 ms) : 0, 49529
AppSec [candidate] (49.33 ms) : 0, 49330
Remote Config [baseline] (656.558 µs) : 0, 657
Remote Config [candidate] (664.566 µs) : 0, 665
Telemetry [baseline] (7.57 ms) : 0, 7570
Telemetry [candidate] (7.647 ms) : 0, 7647
section appsec
BytebuddyAgent [baseline] (697.463 ms) : 0, 697463
BytebuddyAgent [candidate] (695.027 ms) : 0, 695027
GlobalTracer [baseline] (292.637 ms) : 0, 292637
GlobalTracer [candidate] (291.379 ms) : 0, 291379
AppSec [baseline] (150.174 ms) : 0, 150174
AppSec [candidate] (149.298 ms) : 0, 149298
Remote Config [baseline] (624.561 µs) : 0, 625
Remote Config [candidate] (617.347 µs) : 0, 617
Telemetry [baseline] (8.102 ms) : 0, 8102
Telemetry [candidate] (6.879 ms) : 0, 6879
IAST [baseline] (19.035 ms) : 0, 19035
IAST [candidate] (18.968 ms) : 0, 18968
section iast
BytebuddyAgent [baseline] (794.267 ms) : 0, 794267
BytebuddyAgent [candidate] (798.943 ms) : 0, 798943
GlobalTracer [baseline] (287.867 ms) : 0, 287867
GlobalTracer [candidate] (289.537 ms) : 0, 289537
AppSec [baseline] (51.36 ms) : 0, 51360
AppSec [candidate] (51.567 ms) : 0, 51567
Remote Config [baseline] (563.564 µs) : 0, 564
Remote Config [candidate] (657.119 µs) : 0, 657
Telemetry [baseline] (8.129 ms) : 0, 8129
Telemetry [candidate] (6.525 ms) : 0, 6525
IAST [baseline] (21.516 ms) : 0, 21516
IAST [candidate] (23.1 ms) : 0, 23100
section profiling
BytebuddyAgent [baseline] (686.476 ms) : 0, 686476
BytebuddyAgent [candidate] (679.348 ms) : 0, 679348
GlobalTracer [baseline] (384.082 ms) : 0, 384082
GlobalTracer [candidate] (381.495 ms) : 0, 381495
AppSec [baseline] (50.673 ms) : 0, 50673
AppSec [candidate] (50.268 ms) : 0, 50268
Remote Config [baseline] (722.897 µs) : 0, 723
Remote Config [candidate] (712.572 µs) : 0, 713
Telemetry [baseline] (7.51 ms) : 0, 7510
Telemetry [candidate] (7.437 ms) : 0, 7437
ProfilingAgent [baseline] (96.552 ms) : 0, 96552
ProfilingAgent [candidate] (96.904 ms) : 0, 96904
Profiling [baseline] (96.575 ms) : 0, 96575
Profiling [candidate] (96.928 ms) : 0, 96928
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081036
Total [baseline] (8.58 s) : 0, 8579802
Agent [candidate] (1.074 s) : 0, 1074233
Total [candidate] (8.566 s) : 0, 8566260
section iast
Agent [baseline] (1.197 s) : 0, 1196952
Total [baseline] (9.037 s) : 0, 9037407
Agent [candidate] (1.198 s) : 0, 1197665
Total [candidate] (8.987 s) : 0, 8986534
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.207 s) : 0, 1206653
Total [baseline] (8.971 s) : 0, 8971051
Agent [candidate] (1.197 s) : 0, 1197188
Total [candidate] (9.0 s) : 0, 8999959
section iast_TELEMETRY_OFF
Agent [baseline] (1.202 s) : 0, 1202388
Total [baseline] (8.983 s) : 0, 8983493
Agent [candidate] (1.196 s) : 0, 1195615
Total [candidate] (9.032 s) : 0, 9032398
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.081 s -
Agent iast 1.197 s 115.916 ms (10.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.207 s 125.617 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.202 s 121.352 ms (11.2%)
Total tracing 8.58 s -
Total iast 9.037 s 457.605 ms (5.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.971 s 391.248 ms (4.6%)
Total iast_TELEMETRY_OFF 8.983 s 403.69 ms (4.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.074 s -
Agent iast 1.198 s 123.431 ms (11.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.197 s 122.955 ms (11.4%)
Agent iast_TELEMETRY_OFF 1.196 s 121.382 ms (11.3%)
Total tracing 8.566 s -
Total iast 8.987 s 420.274 ms (4.9%)
Total iast_HARDCODED_SECRET_DISABLED 9.0 s 433.698 ms (5.1%)
Total iast_TELEMETRY_OFF 9.032 s 466.138 ms (5.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (676.797 ms) : 0, 676797
BytebuddyAgent [candidate] (672.612 ms) : 0, 672612
GlobalTracer [baseline] (311.516 ms) : 0, 311516
GlobalTracer [candidate] (309.444 ms) : 0, 309444
AppSec [baseline] (49.736 ms) : 0, 49736
AppSec [candidate] (49.416 ms) : 0, 49416
Remote Config [baseline] (672.386 µs) : 0, 672
Remote Config [candidate] (662.202 µs) : 0, 662
Telemetry [baseline] (7.773 ms) : 0, 7773
Telemetry [candidate] (7.654 ms) : 0, 7654
section iast
BytebuddyAgent [baseline] (793.884 ms) : 0, 793884
BytebuddyAgent [candidate] (793.758 ms) : 0, 793758
GlobalTracer [baseline] (287.145 ms) : 0, 287145
GlobalTracer [candidate] (287.654 ms) : 0, 287654
AppSec [baseline] (50.088 ms) : 0, 50088
AppSec [candidate] (50.697 ms) : 0, 50697
Remote Config [baseline] (575.846 µs) : 0, 576
Remote Config [candidate] (579.262 µs) : 0, 579
Telemetry [baseline] (7.332 ms) : 0, 7332
Telemetry [candidate] (6.64 ms) : 0, 6640
IAST [baseline] (23.62 ms) : 0, 23620
IAST [candidate] (23.962 ms) : 0, 23962
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (799.423 ms) : 0, 799423
BytebuddyAgent [candidate] (792.885 ms) : 0, 792885
GlobalTracer [baseline] (290.168 ms) : 0, 290168
GlobalTracer [candidate] (288.15 ms) : 0, 288150
AppSec [baseline] (50.722 ms) : 0, 50722
AppSec [candidate] (49.382 ms) : 0, 49382
Remote Config [baseline] (574.486 µs) : 0, 574
Remote Config [candidate] (568.397 µs) : 0, 568
Telemetry [baseline] (8.063 ms) : 0, 8063
Telemetry [candidate] (8.14 ms) : 0, 8140
IAST [baseline] (23.024 ms) : 0, 23024
IAST [candidate] (23.672 ms) : 0, 23672
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (796.901 ms) : 0, 796901
BytebuddyAgent [candidate] (791.752 ms) : 0, 791752
GlobalTracer [baseline] (289.323 ms) : 0, 289323
GlobalTracer [candidate] (287.833 ms) : 0, 287833
AppSec [baseline] (48.86 ms) : 0, 48860
AppSec [candidate] (48.629 ms) : 0, 48629
Remote Config [baseline] (581.845 µs) : 0, 582
Remote Config [candidate] (590.593 µs) : 0, 591
Telemetry [baseline] (8.033 ms) : 0, 8033
Telemetry [candidate] (6.6 ms) : 0, 6600
IAST [baseline] (23.969 ms) : 0, 23969
IAST [candidate] (25.742 ms) : 0, 25742
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-04-17T10:43:36 2024-04-17T10:50:26
git_branch master nikita-tkachenko/karate-parallel-execution-fix
git_commit_date 1713338255 1713350011
git_commit_sha 34e7066 6a6a136
release_version 1.33.0-SNAPSHOT~34e706697c 1.33.0-SNAPSHOT~6a6a136faa
start_time 2024-04-17T10:43:23 2024-04-17T10:50:13
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1713351372 1713351372
ci_job_id 488993768 488993768
ci_pipeline_id 32293301 32293301
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 13 metrics, 15 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.347 ms) : 1328, 1367
.   : milestone, 1347,
appsec (1.735 ms) : 1712, 1759
.   : milestone, 1735,
appsec_no_iast (1.731 ms) : 1707, 1755
.   : milestone, 1731,
iast (1.498 ms) : 1475, 1521
.   : milestone, 1498,
profiling (1.543 ms) : 1517, 1568
.   : milestone, 1543,
tracing (1.471 ms) : 1447, 1495
.   : milestone, 1471,
section candidate
no_agent (1.353 ms) : 1333, 1373
.   : milestone, 1353,
appsec (1.723 ms) : 1699, 1747
.   : milestone, 1723,
appsec_no_iast (1.722 ms) : 1698, 1746
.   : milestone, 1722,
iast (1.503 ms) : 1479, 1526
.   : milestone, 1503,
profiling (1.535 ms) : 1509, 1561
.   : milestone, 1535,
tracing (1.491 ms) : 1467, 1516
.   : milestone, 1491,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.328 ms, 1.367 ms] -
appsec 1.735 ms [1.712 ms, 1.759 ms] 388.097 µs (28.8%)
appsec_no_iast 1.731 ms [1.707 ms, 1.755 ms] 383.51 µs (28.5%)
iast 1.498 ms [1.475 ms, 1.521 ms] 151.04 µs (11.2%)
profiling 1.543 ms [1.517 ms, 1.568 ms] 195.395 µs (14.5%)
tracing 1.471 ms [1.447 ms, 1.495 ms] 123.602 µs (9.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.353 ms [1.333 ms, 1.373 ms] -
appsec 1.723 ms [1.699 ms, 1.747 ms] 370.4 µs (27.4%)
appsec_no_iast 1.722 ms [1.698 ms, 1.746 ms] 368.833 µs (27.3%)
iast 1.503 ms [1.479 ms, 1.526 ms] 149.578 µs (11.1%)
profiling 1.535 ms [1.509 ms, 1.561 ms] 181.717 µs (13.4%)
tracing 1.491 ms [1.467 ms, 1.516 ms] 138.441 µs (10.2%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (366.432 µs) : 347, 386
.   : milestone, 366,
iast (478.221 µs) : 457, 499
.   : milestone, 478,
iast_FULL (543.852 µs) : 522, 566
.   : milestone, 544,
iast_GLOBAL (498.131 µs) : 478, 519
.   : milestone, 498,
iast_HARDCODED_SECRET_DISABLED (477.31 µs) : 456, 498
.   : milestone, 477,
iast_INACTIVE (446.107 µs) : 426, 467
.   : milestone, 446,
iast_TELEMETRY_OFF (474.659 µs) : 453, 496
.   : milestone, 475,
tracing (448.653 µs) : 427, 470
.   : milestone, 449,
section candidate
no_agent (374.955 µs) : 355, 395
.   : milestone, 375,
iast (473.949 µs) : 452, 496
.   : milestone, 474,
iast_FULL (547.338 µs) : 526, 568
.   : milestone, 547,
iast_GLOBAL (502.053 µs) : 480, 524
.   : milestone, 502,
iast_HARDCODED_SECRET_DISABLED (482.651 µs) : 462, 504
.   : milestone, 483,
iast_INACTIVE (450.633 µs) : 430, 471
.   : milestone, 451,
iast_TELEMETRY_OFF (477.579 µs) : 457, 499
.   : milestone, 478,
tracing (441.65 µs) : 421, 462
.   : milestone, 442,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 366.432 µs [346.689 µs, 386.174 µs] -
iast 478.221 µs [457.277 µs, 499.165 µs] 111.79 µs (30.5%)
iast_FULL 543.852 µs [522.045 µs, 565.658 µs] 177.42 µs (48.4%)
iast_GLOBAL 498.131 µs [477.762 µs, 518.5 µs] 131.699 µs (35.9%)
iast_HARDCODED_SECRET_DISABLED 477.31 µs [456.484 µs, 498.135 µs] 110.878 µs (30.3%)
iast_INACTIVE 446.107 µs [425.623 µs, 466.592 µs] 79.676 µs (21.7%)
iast_TELEMETRY_OFF 474.659 µs [453.467 µs, 495.852 µs] 108.227 µs (29.5%)
tracing 448.653 µs [427.378 µs, 469.928 µs] 82.221 µs (22.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 374.955 µs [355.055 µs, 394.855 µs] -
iast 473.949 µs [451.61 µs, 496.288 µs] 98.994 µs (26.4%)
iast_FULL 547.338 µs [526.356 µs, 568.319 µs] 172.383 µs (46.0%)
iast_GLOBAL 502.053 µs [480.102 µs, 524.004 µs] 127.098 µs (33.9%)
iast_HARDCODED_SECRET_DISABLED 482.651 µs [461.556 µs, 503.745 µs] 107.696 µs (28.7%)
iast_INACTIVE 450.633 µs [429.878 µs, 471.388 µs] 75.678 µs (20.2%)
iast_TELEMETRY_OFF 477.579 µs [456.587 µs, 498.572 µs] 102.624 µs (27.4%)
tracing 441.65 µs [421.376 µs, 461.924 µs] 66.695 µs (17.8%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/karate-parallel-execution-fix
git_commit_date 1713338255 1713350011
git_commit_sha 34e7066 6a6a136
release_version 1.33.0-SNAPSHOT~34e706697c 1.33.0-SNAPSHOT~6a6a136faa
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1713351976 1713351976
ci_job_id 488993769 488993769
ci_pipeline_id 32293301 32293301
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.455 s) : 15455000, 15455000
.   : milestone, 15455000,
appsec (15.022 s) : 15022000, 15022000
.   : milestone, 15022000,
iast (18.651 s) : 18651000, 18651000
.   : milestone, 18651000,
iast_GLOBAL (17.907 s) : 17907000, 17907000
.   : milestone, 17907000,
profiling (15.295 s) : 15295000, 15295000
.   : milestone, 15295000,
tracing (15.159 s) : 15159000, 15159000
.   : milestone, 15159000,
section candidate
no_agent (14.851 s) : 14851000, 14851000
.   : milestone, 14851000,
appsec (15.226 s) : 15226000, 15226000
.   : milestone, 15226000,
iast (18.634 s) : 18634000, 18634000
.   : milestone, 18634000,
iast_GLOBAL (18.306 s) : 18306000, 18306000
.   : milestone, 18306000,
profiling (15.414 s) : 15414000, 15414000
.   : milestone, 15414000,
tracing (15.14 s) : 15140000, 15140000
.   : milestone, 15140000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.455 s [15.455 s, 15.455 s] -
appsec 15.022 s [15.022 s, 15.022 s] -433.0 ms (-2.8%)
iast 18.651 s [18.651 s, 18.651 s] 3.196 s (20.7%)
iast_GLOBAL 17.907 s [17.907 s, 17.907 s] 2.452 s (15.9%)
profiling 15.295 s [15.295 s, 15.295 s] -160.0 ms (-1.0%)
tracing 15.159 s [15.159 s, 15.159 s] -296.0 ms (-1.9%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.851 s [14.851 s, 14.851 s] -
appsec 15.226 s [15.226 s, 15.226 s] 375.0 ms (2.5%)
iast 18.634 s [18.634 s, 18.634 s] 3.783 s (25.5%)
iast_GLOBAL 18.306 s [18.306 s, 18.306 s] 3.455 s (23.3%)
profiling 15.414 s [15.414 s, 15.414 s] 563.0 ms (3.8%)
tracing 15.14 s [15.14 s, 15.14 s] 289.0 ms (1.9%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.33.0-SNAPSHOT~6a6a136faa, baseline=1.33.0-SNAPSHOT~34e706697c
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.451 ms) : 1439, 1462
.   : milestone, 1451,
appsec (2.185 ms) : 2151, 2218
.   : milestone, 2185,
iast (1.874 ms) : 1839, 1909
.   : milestone, 1874,
iast_GLOBAL (1.917 ms) : 1881, 1953
.   : milestone, 1917,
profiling (1.837 ms) : 1803, 1871
.   : milestone, 1837,
tracing (1.816 ms) : 1784, 1847
.   : milestone, 1816,
section candidate
no_agent (1.454 ms) : 1442, 1465
.   : milestone, 1454,
appsec (2.173 ms) : 2140, 2206
.   : milestone, 2173,
iast (1.874 ms) : 1839, 1909
.   : milestone, 1874,
iast_GLOBAL (1.906 ms) : 1870, 1941
.   : milestone, 1906,
profiling (1.843 ms) : 1810, 1876
.   : milestone, 1843,
tracing (1.808 ms) : 1777, 1839
.   : milestone, 1808,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.451 ms [1.439 ms, 1.462 ms] -
appsec 2.185 ms [2.151 ms, 2.218 ms] 734.311 µs (50.6%)
iast 1.874 ms [1.839 ms, 1.909 ms] 423.406 µs (29.2%)
iast_GLOBAL 1.917 ms [1.881 ms, 1.953 ms] 466.615 µs (32.2%)
profiling 1.837 ms [1.803 ms, 1.871 ms] 386.071 µs (26.6%)
tracing 1.816 ms [1.784 ms, 1.847 ms] 365.171 µs (25.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.454 ms [1.442 ms, 1.465 ms] -
appsec 2.173 ms [2.14 ms, 2.206 ms] 719.43 µs (49.5%)
iast 1.874 ms [1.839 ms, 1.909 ms] 420.512 µs (28.9%)
iast_GLOBAL 1.906 ms [1.87 ms, 1.941 ms] 451.989 µs (31.1%)
profiling 1.843 ms [1.81 ms, 1.876 ms] 389.549 µs (26.8%)
tracing 1.808 ms [1.777 ms, 1.839 ms] 354.511 µs (24.4%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 91854ff into master Apr 17, 2024
80 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/karate-parallel-execution-fix branch April 17, 2024 12:55
@github-actions github-actions bot added this to the 1.33.0 milestone Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romainkomorndatadog romainkomorndatadog romainkomorndatadog approved these changes

@juan-fernandez juan-fernandez Awaiting requested review from juan-fernandez juan-fernandez is a code owner automatically assigned from DataDog/ci-app-libraries-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
1.33.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @romainkomorndatadog