Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix tracing skipped suites in MUnit 1.0.1 #7605

Merged
merged 1 commit into from
Sep 13, 2024
Merged

Fix tracing skipped suites in MUnit 1.0.1 #7605

merged 1 commit into from
Sep 13, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Sep 11, 2024 •
edited by jira bot
Loading

What Does This Do

Fixes tracing of skipped suites in MUnit 1.0.1

Additional Notes

In MUnit 1.0.1 release a different set of events is fired for skipped test suites.
Previous versions of the framework fixed start/ignore/finish events for such suites.
The new version of the framework only fires the ignore event.

This appears to be an accidental side-effect of an unrelated change, and an issue was reported in the framework repo.
It does not look, however, like this is getting fixed any time soon, so a workaround is introduced in the tracer.

Contributor Checklist

Jira ticket: SDTEST-843

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Sep 11, 2024
@pr-commenter
Copy link

pr-commenter bot commented Sep 11, 2024

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/fix-munit-1.0.1-instr
git_commit_date 1726062980 1726070768
git_commit_sha 4abf841 773ac49
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~773ac49647
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726073041 1726073041
ci_job_id 635846721 635846721
ci_pipeline_id 44098616 44098616
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066065
Total [baseline] (8.529 s) : 0, 8528886
Agent [candidate] (1.073 s) : 0, 1072891
Total [candidate] (8.529 s) : 0, 8529201
section iast
Agent [baseline] (1.191 s) : 0, 1190776
Total [baseline] (8.986 s) : 0, 8985936
Agent [candidate] (1.19 s) : 0, 1189717
Total [candidate] (8.993 s) : 0, 8993161
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.19 s) : 0, 1190020
Total [baseline] (8.966 s) : 0, 8965559
Agent [candidate] (1.199 s) : 0, 1199189
Total [candidate] (8.993 s) : 0, 8992607
section iast_TELEMETRY_OFF
Agent [baseline] (1.193 s) : 0, 1193221
Total [baseline] (9.028 s) : 0, 9027756
Agent [candidate] (1.189 s) : 0, 1188661
Total [candidate] (8.99 s) : 0, 8990097
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent iast 1.191 s 124.711 ms (11.7%)
Agent iast_HARDCODED_SECRET_DISABLED 1.19 s 123.955 ms (11.6%)
Agent iast_TELEMETRY_OFF 1.193 s 127.156 ms (11.9%)
Total tracing 8.529 s -
Total iast 8.986 s 457.05 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 8.966 s 436.673 ms (5.1%)
Total iast_TELEMETRY_OFF 9.028 s 498.871 ms (5.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.073 s -
Agent iast 1.19 s 116.826 ms (10.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.199 s 126.299 ms (11.8%)
Agent iast_TELEMETRY_OFF 1.189 s 115.77 ms (10.8%)
Total tracing 8.529 s -
Total iast 8.993 s 463.96 ms (5.4%)
Total iast_HARDCODED_SECRET_DISABLED 8.993 s 463.406 ms (5.4%)
Total iast_TELEMETRY_OFF 8.99 s 460.897 ms (5.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (680.097 ms) : 0, 680097
BytebuddyAgent [candidate] (684.967 ms) : 0, 684967
GlobalTracer [baseline] (312.109 ms) : 0, 312109
GlobalTracer [candidate] (313.735 ms) : 0, 313735
AppSec [baseline] (51.862 ms) : 0, 51862
AppSec [candidate] (52.05 ms) : 0, 52050
Remote Config [baseline] (710.753 µs) : 0, 711
Remote Config [candidate] (717.282 µs) : 0, 717
Telemetry [baseline] (7.558 ms) : 0, 7558
Telemetry [candidate] (7.597 ms) : 0, 7597
section iast
BytebuddyAgent [baseline] (792.01 ms) : 0, 792010
BytebuddyAgent [candidate] (791.256 ms) : 0, 791256
GlobalTracer [baseline] (300.248 ms) : 0, 300248
GlobalTracer [candidate] (300.069 ms) : 0, 300069
AppSec [baseline] (54.755 ms) : 0, 54755
AppSec [candidate] (52.102 ms) : 0, 52102
IAST [baseline] (21.955 ms) : 0, 21955
IAST [candidate] (23.772 ms) : 0, 23772
Remote Config [baseline] (639.398 µs) : 0, 639
Remote Config [candidate] (606.462 µs) : 0, 606
Telemetry [baseline] (7.412 ms) : 0, 7412
Telemetry [candidate] (8.129 ms) : 0, 8129
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (791.322 ms) : 0, 791322
BytebuddyAgent [candidate] (797.308 ms) : 0, 797308
GlobalTracer [baseline] (299.994 ms) : 0, 299994
GlobalTracer [candidate] (302.33 ms) : 0, 302330
AppSec [baseline] (55.831 ms) : 0, 55831
AppSec [candidate] (54.671 ms) : 0, 54671
IAST [baseline] (21.041 ms) : 0, 21041
IAST [candidate] (22.157 ms) : 0, 22157
Remote Config [baseline] (613.068 µs) : 0, 613
Remote Config [candidate] (613.872 µs) : 0, 614
Telemetry [baseline] (7.448 ms) : 0, 7448
Telemetry [candidate] (8.23 ms) : 0, 8230
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.469 ms) : 0, 792469
BytebuddyAgent [candidate] (789.015 ms) : 0, 789015
GlobalTracer [baseline] (301.719 ms) : 0, 301719
GlobalTracer [candidate] (301.366 ms) : 0, 301366
AppSec [baseline] (55.676 ms) : 0, 55676
AppSec [candidate] (54.159 ms) : 0, 54159
IAST [baseline] (20.603 ms) : 0, 20603
IAST [candidate] (22.392 ms) : 0, 22392
Remote Config [baseline] (621.894 µs) : 0, 622
Remote Config [candidate] (590.976 µs) : 0, 591
Telemetry [baseline] (8.275 ms) : 0, 8275
Telemetry [candidate] (7.293 ms) : 0, 7293
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067220
Total [baseline] (10.376 s) : 0, 10375646
Agent [candidate] (1.066 s) : 0, 1065595
Total [candidate] (10.339 s) : 0, 10338879
section appsec
Agent [baseline] (1.198 s) : 0, 1198383
Total [baseline] (10.607 s) : 0, 10607379
Agent [candidate] (1.2 s) : 0, 1200209
Total [candidate] (10.597 s) : 0, 10597016
section iast
Agent [baseline] (1.192 s) : 0, 1191626
Total [baseline] (10.91 s) : 0, 10909823
Agent [candidate] (1.19 s) : 0, 1190493
Total [candidate] (10.82 s) : 0, 10820104
section profiling
Agent [baseline] (1.267 s) : 0, 1267120
Total [baseline] (10.628 s) : 0, 10627717
Agent [candidate] (1.273 s) : 0, 1272772
Total [candidate] (10.613 s) : 0, 10612838
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.067 s -
Agent appsec 1.198 s 131.163 ms (12.3%)
Agent iast 1.192 s 124.407 ms (11.7%)
Agent profiling 1.267 s 199.901 ms (18.7%)
Total tracing 10.376 s -
Total appsec 10.607 s 231.732 ms (2.2%)
Total iast 10.91 s 534.177 ms (5.1%)
Total profiling 10.628 s 252.07 ms (2.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.066 s -
Agent appsec 1.2 s 134.614 ms (12.6%)
Agent iast 1.19 s 124.898 ms (11.7%)
Agent profiling 1.273 s 207.176 ms (19.4%)
Total tracing 10.339 s -
Total appsec 10.597 s 258.137 ms (2.5%)
Total iast 10.82 s 481.225 ms (4.7%)
Total profiling 10.613 s 273.959 ms (2.6%) 
gantt
    title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (680.89 ms) : 0, 680890
BytebuddyAgent [candidate] (679.858 ms) : 0, 679858
GlobalTracer [baseline] (312.411 ms) : 0, 312411
GlobalTracer [candidate] (311.852 ms) : 0, 311852
AppSec [baseline] (51.862 ms) : 0, 51862
AppSec [candidate] (51.903 ms) : 0, 51903
Remote Config [baseline] (717.744 µs) : 0, 718
Remote Config [candidate] (697.761 µs) : 0, 698
Telemetry [baseline] (7.6 ms) : 0, 7600
Telemetry [candidate] (7.54 ms) : 0, 7540
section appsec
BytebuddyAgent [baseline] (700.276 ms) : 0, 700276
BytebuddyAgent [candidate] (700.088 ms) : 0, 700088
GlobalTracer [baseline] (304.081 ms) : 0, 304081
GlobalTracer [candidate] (304.842 ms) : 0, 304842
AppSec [baseline] (160.935 ms) : 0, 160935
AppSec [candidate] (161.543 ms) : 0, 161543
IAST [baseline] (19.942 ms) : 0, 19942
IAST [candidate] (20.719 ms) : 0, 20719
Remote Config [baseline] (654.796 µs) : 0, 655
Remote Config [candidate] (646.51 µs) : 0, 647
Telemetry [baseline] (8.754 ms) : 0, 8754
Telemetry [candidate] (8.998 ms) : 0, 8998
section iast
BytebuddyAgent [baseline] (792.131 ms) : 0, 792131
BytebuddyAgent [candidate] (791.094 ms) : 0, 791094
GlobalTracer [baseline] (300.671 ms) : 0, 300671
GlobalTracer [candidate] (300.362 ms) : 0, 300362
AppSec [baseline] (54.034 ms) : 0, 54034
AppSec [candidate] (53.404 ms) : 0, 53404
IAST [baseline] (22.242 ms) : 0, 22242
IAST [candidate] (23.915 ms) : 0, 23915
Remote Config [baseline] (1.345 ms) : 0, 1345
Remote Config [candidate] (605.074 µs) : 0, 605
Telemetry [baseline] (7.464 ms) : 0, 7464
Telemetry [candidate] (7.362 ms) : 0, 7362
section profiling
BytebuddyAgent [baseline] (675.804 ms) : 0, 675804
BytebuddyAgent [candidate] (679.001 ms) : 0, 679001
GlobalTracer [baseline] (395.895 ms) : 0, 395895
GlobalTracer [candidate] (397.391 ms) : 0, 397391
AppSec [baseline] (52.554 ms) : 0, 52554
AppSec [candidate] (52.873 ms) : 0, 52873
Remote Config [baseline] (715.864 µs) : 0, 716
Remote Config [candidate] (709.512 µs) : 0, 710
Telemetry [baseline] (7.533 ms) : 0, 7533
Telemetry [candidate] (7.514 ms) : 0, 7514
ProfilingAgent [baseline] (96.372 ms) : 0, 96372
ProfilingAgent [candidate] (96.79 ms) : 0, 96790
Profiling [baseline] (96.396 ms) : 0, 96396
Profiling [candidate] (96.813 ms) : 0, 96813
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-09-11T16:16:41 2024-09-11T16:23:30
git_branch master nikita-tkachenko/fix-munit-1.0.1-instr
git_commit_date 1726062980 1726070768
git_commit_sha 4abf841 773ac49
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~773ac49647
start_time 2024-09-11T16:16:28 2024-09-11T16:23:17
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1726072155 1726072155
ci_job_id 635846723 635846723
ci_pipeline_id 44098616 44098616
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (372.559 µs) : 352, 393
.   : milestone, 373,
iast (481.9 µs) : 460, 504
.   : milestone, 482,
iast_FULL (549.941 µs) : 528, 571
.   : milestone, 550,
iast_GLOBAL (505.85 µs) : 485, 527
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (478.398 µs) : 457, 500
.   : milestone, 478,
iast_INACTIVE (440.754 µs) : 420, 461
.   : milestone, 441,
iast_TELEMETRY_OFF (472.699 µs) : 450, 495
.   : milestone, 473,
tracing (438.143 µs) : 417, 459
.   : milestone, 438,
section candidate
no_agent (361.004 µs) : 341, 381
.   : milestone, 361,
iast (478.222 µs) : 456, 500
.   : milestone, 478,
iast_FULL (545.348 µs) : 524, 567
.   : milestone, 545,
iast_GLOBAL (499.914 µs) : 479, 521
.   : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (483.213 µs) : 461, 506
.   : milestone, 483,
iast_INACTIVE (443.23 µs) : 422, 464
.   : milestone, 443,
iast_TELEMETRY_OFF (474.36 µs) : 451, 497
.   : milestone, 474,
tracing (444.0 µs) : 423, 465
.   : milestone, 444,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 372.559 µs [351.692 µs, 393.425 µs] -
iast 481.9 µs [459.673 µs, 504.127 µs] 109.341 µs (29.3%)
iast_FULL 549.941 µs [528.421 µs, 571.461 µs] 177.382 µs (47.6%)
iast_GLOBAL 505.85 µs [484.615 µs, 527.086 µs] 133.291 µs (35.8%)
iast_HARDCODED_SECRET_DISABLED 478.398 µs [456.627 µs, 500.169 µs] 105.84 µs (28.4%)
iast_INACTIVE 440.754 µs [420.086 µs, 461.423 µs] 68.196 µs (18.3%)
iast_TELEMETRY_OFF 472.699 µs [449.9 µs, 495.498 µs] 100.14 µs (26.9%)
tracing 438.143 µs [417.405 µs, 458.881 µs] 65.584 µs (17.6%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 361.004 µs [340.531 µs, 381.477 µs] -
iast 478.222 µs [456.05 µs, 500.394 µs] 117.218 µs (32.5%)
iast_FULL 545.348 µs [524.099 µs, 566.598 µs] 184.344 µs (51.1%)
iast_GLOBAL 499.914 µs [478.946 µs, 520.883 µs] 138.91 µs (38.5%)
iast_HARDCODED_SECRET_DISABLED 483.213 µs [460.917 µs, 505.509 µs] 122.209 µs (33.9%)
iast_INACTIVE 443.23 µs [422.367 µs, 464.094 µs] 82.226 µs (22.8%)
iast_TELEMETRY_OFF 474.36 µs [451.243 µs, 497.477 µs] 113.356 µs (31.4%)
tracing 444.0 µs [422.656 µs, 465.343 µs] 82.996 µs (23.0%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.318 ms) : 1299, 1337
.   : milestone, 1318,
appsec (1.725 ms) : 1701, 1749
.   : milestone, 1725,
appsec_no_iast (1.687 ms) : 1662, 1711
.   : milestone, 1687,
iast (1.471 ms) : 1449, 1493
.   : milestone, 1471,
profiling (1.517 ms) : 1492, 1543
.   : milestone, 1517,
tracing (1.47 ms) : 1446, 1493
.   : milestone, 1470,
section candidate
no_agent (1.35 ms) : 1329, 1370
.   : milestone, 1350,
appsec (1.699 ms) : 1674, 1724
.   : milestone, 1699,
appsec_no_iast (1.716 ms) : 1692, 1741
.   : milestone, 1716,
iast (1.462 ms) : 1439, 1485
.   : milestone, 1462,
profiling (1.471 ms) : 1448, 1494
.   : milestone, 1471,
tracing (1.466 ms) : 1442, 1490
.   : milestone, 1466,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.318 ms [1.299 ms, 1.337 ms] -
appsec 1.725 ms [1.701 ms, 1.749 ms] 406.943 µs (30.9%)
appsec_no_iast 1.687 ms [1.662 ms, 1.711 ms] 369.026 µs (28.0%)
iast 1.471 ms [1.449 ms, 1.493 ms] 153.195 µs (11.6%)
profiling 1.517 ms [1.492 ms, 1.543 ms] 199.249 µs (15.1%)
tracing 1.47 ms [1.446 ms, 1.493 ms] 151.704 µs (11.5%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.35 ms [1.329 ms, 1.37 ms] -
appsec 1.699 ms [1.674 ms, 1.724 ms] 349.22 µs (25.9%)
appsec_no_iast 1.716 ms [1.692 ms, 1.741 ms] 366.327 µs (27.1%)
iast 1.462 ms [1.439 ms, 1.485 ms] 112.148 µs (8.3%)
profiling 1.471 ms [1.448 ms, 1.494 ms] 120.949 µs (9.0%)
tracing 1.466 ms [1.442 ms, 1.49 ms] 116.39 µs (8.6%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/fix-munit-1.0.1-instr
git_commit_date 1726062980 1726070768
git_commit_sha 4abf841 773ac49
release_version 1.40.0-SNAPSHOT~4abf841368 1.40.0-SNAPSHOT~773ac49647
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1726072721 1726072721
ci_job_id 635846726 635846726
ci_pipeline_id 44098616 44098616
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.462 ms) : 1450, 1473
.   : milestone, 1462,
appsec (2.275 ms) : 2235, 2316
.   : milestone, 2275,
iast (2.051 ms) : 2002, 2100
.   : milestone, 2051,
iast_GLOBAL (2.084 ms) : 2033, 2134
.   : milestone, 2084,
profiling (1.917 ms) : 1876, 1958
.   : milestone, 1917,
tracing (1.896 ms) : 1857, 1934
.   : milestone, 1896,
section candidate
no_agent (1.456 ms) : 1445, 1468
.   : milestone, 1456,
appsec (2.263 ms) : 2223, 2303
.   : milestone, 2263,
iast (2.033 ms) : 1984, 2081
.   : milestone, 2033,
iast_GLOBAL (2.082 ms) : 2032, 2132
.   : milestone, 2082,
profiling (1.92 ms) : 1879, 1962
.   : milestone, 1920,
tracing (1.89 ms) : 1852, 1928
.   : milestone, 1890,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.462 ms [1.45 ms, 1.473 ms] -
appsec 2.275 ms [2.235 ms, 2.316 ms] 813.228 µs (55.6%)
iast 2.051 ms [2.002 ms, 2.1 ms] 589.102 µs (40.3%)
iast_GLOBAL 2.084 ms [2.033 ms, 2.134 ms] 621.67 µs (42.5%)
profiling 1.917 ms [1.876 ms, 1.958 ms] 455.259 µs (31.1%)
tracing 1.896 ms [1.857 ms, 1.934 ms] 433.915 µs (29.7%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.456 ms [1.445 ms, 1.468 ms] -
appsec 2.263 ms [2.223 ms, 2.303 ms] 806.819 µs (55.4%)
iast 2.033 ms [1.984 ms, 2.081 ms] 576.275 µs (39.6%)
iast_GLOBAL 2.082 ms [2.032 ms, 2.132 ms] 625.453 µs (42.9%)
profiling 1.92 ms [1.879 ms, 1.962 ms] 464.032 µs (31.9%)
tracing 1.89 ms [1.852 ms, 1.928 ms] 433.587 µs (29.8%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.343 s) : 15343000, 15343000
.   : milestone, 15343000,
appsec (15.088 s) : 15088000, 15088000
.   : milestone, 15088000,
iast (18.727 s) : 18727000, 18727000
.   : milestone, 18727000,
iast_GLOBAL (18.435 s) : 18435000, 18435000
.   : milestone, 18435000,
profiling (15.25 s) : 15250000, 15250000
.   : milestone, 15250000,
tracing (14.993 s) : 14993000, 14993000
.   : milestone, 14993000,
section candidate
no_agent (15.045 s) : 15045000, 15045000
.   : milestone, 15045000,
appsec (15.089 s) : 15089000, 15089000
.   : milestone, 15089000,
iast (18.817 s) : 18817000, 18817000
.   : milestone, 18817000,
iast_GLOBAL (18.24 s) : 18240000, 18240000
.   : milestone, 18240000,
profiling (15.831 s) : 15831000, 15831000
.   : milestone, 15831000,
tracing (15.077 s) : 15077000, 15077000
.   : milestone, 15077000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.343 s [15.343 s, 15.343 s] -
appsec 15.088 s [15.088 s, 15.088 s] -255.0 ms (-1.7%)
iast 18.727 s [18.727 s, 18.727 s] 3.384 s (22.1%)
iast_GLOBAL 18.435 s [18.435 s, 18.435 s] 3.092 s (20.2%)
profiling 15.25 s [15.25 s, 15.25 s] -93.0 ms (-0.6%)
tracing 14.993 s [14.993 s, 14.993 s] -350.0 ms (-2.3%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.045 s [15.045 s, 15.045 s] -
appsec 15.089 s [15.089 s, 15.089 s] 44.0 ms (0.3%)
iast 18.817 s [18.817 s, 18.817 s] 3.772 s (25.1%)
iast_GLOBAL 18.24 s [18.24 s, 18.24 s] 3.195 s (21.2%)
profiling 15.831 s [15.831 s, 15.831 s] 786.0 ms (5.2%)
tracing 15.077 s [15.077 s, 15.077 s] 32.0 ms (0.2%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review September 12, 2024 12:47
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 8f1e767 into master Sep 13, 2024
100 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/fix-munit-1.0.1-instr branch September 13, 2024 08:21
@github-actions github-actions bot added this to the 1.40.0 milestone Sep 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romainkomorndatadog romainkomorndatadog romainkomorndatadog approved these changes

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
1.40.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @romainkomorndatadog