-
Notifications
You must be signed in to change notification settings - Fork 291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix tracing skipped suites in MUnit 1.0.1 #7605
Merged
nikita-tkachenko-datadog
merged 1 commit into
master
from
nikita-tkachenko/fix-munit-1.0.1-instr
Sep 13, 2024
Merged
Fix tracing skipped suites in MUnit 1.0.1 #7605
nikita-tkachenko-datadog
merged 1 commit into
master
from
nikita-tkachenko/fix-munit-1.0.1-instr
Sep 13, 2024
+26
−8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nikita-tkachenko-datadog
added
type: bug
comp: ci visibility
Continuous Integration Visibility
labels
Sep 11, 2024
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066065
Total [baseline] (8.529 s) : 0, 8528886
Agent [candidate] (1.073 s) : 0, 1072891
Total [candidate] (8.529 s) : 0, 8529201
section iast
Agent [baseline] (1.191 s) : 0, 1190776
Total [baseline] (8.986 s) : 0, 8985936
Agent [candidate] (1.19 s) : 0, 1189717
Total [candidate] (8.993 s) : 0, 8993161
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.19 s) : 0, 1190020
Total [baseline] (8.966 s) : 0, 8965559
Agent [candidate] (1.199 s) : 0, 1199189
Total [candidate] (8.993 s) : 0, 8992607
section iast_TELEMETRY_OFF
Agent [baseline] (1.193 s) : 0, 1193221
Total [baseline] (9.028 s) : 0, 9027756
Agent [candidate] (1.189 s) : 0, 1188661
Total [candidate] (8.99 s) : 0, 8990097
gantt
title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (680.097 ms) : 0, 680097
BytebuddyAgent [candidate] (684.967 ms) : 0, 684967
GlobalTracer [baseline] (312.109 ms) : 0, 312109
GlobalTracer [candidate] (313.735 ms) : 0, 313735
AppSec [baseline] (51.862 ms) : 0, 51862
AppSec [candidate] (52.05 ms) : 0, 52050
Remote Config [baseline] (710.753 µs) : 0, 711
Remote Config [candidate] (717.282 µs) : 0, 717
Telemetry [baseline] (7.558 ms) : 0, 7558
Telemetry [candidate] (7.597 ms) : 0, 7597
section iast
BytebuddyAgent [baseline] (792.01 ms) : 0, 792010
BytebuddyAgent [candidate] (791.256 ms) : 0, 791256
GlobalTracer [baseline] (300.248 ms) : 0, 300248
GlobalTracer [candidate] (300.069 ms) : 0, 300069
AppSec [baseline] (54.755 ms) : 0, 54755
AppSec [candidate] (52.102 ms) : 0, 52102
IAST [baseline] (21.955 ms) : 0, 21955
IAST [candidate] (23.772 ms) : 0, 23772
Remote Config [baseline] (639.398 µs) : 0, 639
Remote Config [candidate] (606.462 µs) : 0, 606
Telemetry [baseline] (7.412 ms) : 0, 7412
Telemetry [candidate] (8.129 ms) : 0, 8129
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (791.322 ms) : 0, 791322
BytebuddyAgent [candidate] (797.308 ms) : 0, 797308
GlobalTracer [baseline] (299.994 ms) : 0, 299994
GlobalTracer [candidate] (302.33 ms) : 0, 302330
AppSec [baseline] (55.831 ms) : 0, 55831
AppSec [candidate] (54.671 ms) : 0, 54671
IAST [baseline] (21.041 ms) : 0, 21041
IAST [candidate] (22.157 ms) : 0, 22157
Remote Config [baseline] (613.068 µs) : 0, 613
Remote Config [candidate] (613.872 µs) : 0, 614
Telemetry [baseline] (7.448 ms) : 0, 7448
Telemetry [candidate] (8.23 ms) : 0, 8230
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (792.469 ms) : 0, 792469
BytebuddyAgent [candidate] (789.015 ms) : 0, 789015
GlobalTracer [baseline] (301.719 ms) : 0, 301719
GlobalTracer [candidate] (301.366 ms) : 0, 301366
AppSec [baseline] (55.676 ms) : 0, 55676
AppSec [candidate] (54.159 ms) : 0, 54159
IAST [baseline] (20.603 ms) : 0, 20603
IAST [candidate] (22.392 ms) : 0, 22392
Remote Config [baseline] (621.894 µs) : 0, 622
Remote Config [candidate] (590.976 µs) : 0, 591
Telemetry [baseline] (8.275 ms) : 0, 8275
Telemetry [candidate] (7.293 ms) : 0, 7293
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.067 s) : 0, 1067220
Total [baseline] (10.376 s) : 0, 10375646
Agent [candidate] (1.066 s) : 0, 1065595
Total [candidate] (10.339 s) : 0, 10338879
section appsec
Agent [baseline] (1.198 s) : 0, 1198383
Total [baseline] (10.607 s) : 0, 10607379
Agent [candidate] (1.2 s) : 0, 1200209
Total [candidate] (10.597 s) : 0, 10597016
section iast
Agent [baseline] (1.192 s) : 0, 1191626
Total [baseline] (10.91 s) : 0, 10909823
Agent [candidate] (1.19 s) : 0, 1190493
Total [candidate] (10.82 s) : 0, 10820104
section profiling
Agent [baseline] (1.267 s) : 0, 1267120
Total [baseline] (10.628 s) : 0, 10627717
Agent [candidate] (1.273 s) : 0, 1272772
Total [candidate] (10.613 s) : 0, 10612838
gantt
title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (680.89 ms) : 0, 680890
BytebuddyAgent [candidate] (679.858 ms) : 0, 679858
GlobalTracer [baseline] (312.411 ms) : 0, 312411
GlobalTracer [candidate] (311.852 ms) : 0, 311852
AppSec [baseline] (51.862 ms) : 0, 51862
AppSec [candidate] (51.903 ms) : 0, 51903
Remote Config [baseline] (717.744 µs) : 0, 718
Remote Config [candidate] (697.761 µs) : 0, 698
Telemetry [baseline] (7.6 ms) : 0, 7600
Telemetry [candidate] (7.54 ms) : 0, 7540
section appsec
BytebuddyAgent [baseline] (700.276 ms) : 0, 700276
BytebuddyAgent [candidate] (700.088 ms) : 0, 700088
GlobalTracer [baseline] (304.081 ms) : 0, 304081
GlobalTracer [candidate] (304.842 ms) : 0, 304842
AppSec [baseline] (160.935 ms) : 0, 160935
AppSec [candidate] (161.543 ms) : 0, 161543
IAST [baseline] (19.942 ms) : 0, 19942
IAST [candidate] (20.719 ms) : 0, 20719
Remote Config [baseline] (654.796 µs) : 0, 655
Remote Config [candidate] (646.51 µs) : 0, 647
Telemetry [baseline] (8.754 ms) : 0, 8754
Telemetry [candidate] (8.998 ms) : 0, 8998
section iast
BytebuddyAgent [baseline] (792.131 ms) : 0, 792131
BytebuddyAgent [candidate] (791.094 ms) : 0, 791094
GlobalTracer [baseline] (300.671 ms) : 0, 300671
GlobalTracer [candidate] (300.362 ms) : 0, 300362
AppSec [baseline] (54.034 ms) : 0, 54034
AppSec [candidate] (53.404 ms) : 0, 53404
IAST [baseline] (22.242 ms) : 0, 22242
IAST [candidate] (23.915 ms) : 0, 23915
Remote Config [baseline] (1.345 ms) : 0, 1345
Remote Config [candidate] (605.074 µs) : 0, 605
Telemetry [baseline] (7.464 ms) : 0, 7464
Telemetry [candidate] (7.362 ms) : 0, 7362
section profiling
BytebuddyAgent [baseline] (675.804 ms) : 0, 675804
BytebuddyAgent [candidate] (679.001 ms) : 0, 679001
GlobalTracer [baseline] (395.895 ms) : 0, 395895
GlobalTracer [candidate] (397.391 ms) : 0, 397391
AppSec [baseline] (52.554 ms) : 0, 52554
AppSec [candidate] (52.873 ms) : 0, 52873
Remote Config [baseline] (715.864 µs) : 0, 716
Remote Config [candidate] (709.512 µs) : 0, 710
Telemetry [baseline] (7.533 ms) : 0, 7533
Telemetry [candidate] (7.514 ms) : 0, 7514
ProfilingAgent [baseline] (96.372 ms) : 0, 96372
ProfilingAgent [candidate] (96.79 ms) : 0, 96790
Profiling [baseline] (96.396 ms) : 0, 96396
Profiling [candidate] (96.813 ms) : 0, 96813
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 18 unstable metrics. Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section baseline
no_agent (372.559 µs) : 352, 393
. : milestone, 373,
iast (481.9 µs) : 460, 504
. : milestone, 482,
iast_FULL (549.941 µs) : 528, 571
. : milestone, 550,
iast_GLOBAL (505.85 µs) : 485, 527
. : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (478.398 µs) : 457, 500
. : milestone, 478,
iast_INACTIVE (440.754 µs) : 420, 461
. : milestone, 441,
iast_TELEMETRY_OFF (472.699 µs) : 450, 495
. : milestone, 473,
tracing (438.143 µs) : 417, 459
. : milestone, 438,
section candidate
no_agent (361.004 µs) : 341, 381
. : milestone, 361,
iast (478.222 µs) : 456, 500
. : milestone, 478,
iast_FULL (545.348 µs) : 524, 567
. : milestone, 545,
iast_GLOBAL (499.914 µs) : 479, 521
. : milestone, 500,
iast_HARDCODED_SECRET_DISABLED (483.213 µs) : 461, 506
. : milestone, 483,
iast_INACTIVE (443.23 µs) : 422, 464
. : milestone, 443,
iast_TELEMETRY_OFF (474.36 µs) : 451, 497
. : milestone, 474,
tracing (444.0 µs) : 423, 465
. : milestone, 444,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section baseline
no_agent (1.318 ms) : 1299, 1337
. : milestone, 1318,
appsec (1.725 ms) : 1701, 1749
. : milestone, 1725,
appsec_no_iast (1.687 ms) : 1662, 1711
. : milestone, 1687,
iast (1.471 ms) : 1449, 1493
. : milestone, 1471,
profiling (1.517 ms) : 1492, 1543
. : milestone, 1517,
tracing (1.47 ms) : 1446, 1493
. : milestone, 1470,
section candidate
no_agent (1.35 ms) : 1329, 1370
. : milestone, 1350,
appsec (1.699 ms) : 1674, 1724
. : milestone, 1699,
appsec_no_iast (1.716 ms) : 1692, 1741
. : milestone, 1716,
iast (1.462 ms) : 1439, 1485
. : milestone, 1462,
profiling (1.471 ms) : 1448, 1494
. : milestone, 1471,
tracing (1.466 ms) : 1442, 1490
. : milestone, 1466,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section baseline
no_agent (1.462 ms) : 1450, 1473
. : milestone, 1462,
appsec (2.275 ms) : 2235, 2316
. : milestone, 2275,
iast (2.051 ms) : 2002, 2100
. : milestone, 2051,
iast_GLOBAL (2.084 ms) : 2033, 2134
. : milestone, 2084,
profiling (1.917 ms) : 1876, 1958
. : milestone, 1917,
tracing (1.896 ms) : 1857, 1934
. : milestone, 1896,
section candidate
no_agent (1.456 ms) : 1445, 1468
. : milestone, 1456,
appsec (2.263 ms) : 2223, 2303
. : milestone, 2263,
iast (2.033 ms) : 1984, 2081
. : milestone, 2033,
iast_GLOBAL (2.082 ms) : 2032, 2132
. : milestone, 2082,
profiling (1.92 ms) : 1879, 1962
. : milestone, 1920,
tracing (1.89 ms) : 1852, 1928
. : milestone, 1890,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~773ac49647, baseline=1.40.0-SNAPSHOT~4abf841368
dateFormat X
axisFormat %s
section baseline
no_agent (15.343 s) : 15343000, 15343000
. : milestone, 15343000,
appsec (15.088 s) : 15088000, 15088000
. : milestone, 15088000,
iast (18.727 s) : 18727000, 18727000
. : milestone, 18727000,
iast_GLOBAL (18.435 s) : 18435000, 18435000
. : milestone, 18435000,
profiling (15.25 s) : 15250000, 15250000
. : milestone, 15250000,
tracing (14.993 s) : 14993000, 14993000
. : milestone, 14993000,
section candidate
no_agent (15.045 s) : 15045000, 15045000
. : milestone, 15045000,
appsec (15.089 s) : 15089000, 15089000
. : milestone, 15089000,
iast (18.817 s) : 18817000, 18817000
. : milestone, 18817000,
iast_GLOBAL (18.24 s) : 18240000, 18240000
. : milestone, 18240000,
profiling (15.831 s) : 15831000, 15831000
. : milestone, 15831000,
tracing (15.077 s) : 15077000, 15077000
. : milestone, 15077000,
|
romainkomorndatadog
approved these changes
Sep 13, 2024
nikita-tkachenko-datadog
deleted the
nikita-tkachenko/fix-munit-1.0.1-instr
branch
September 13, 2024 08:21
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Fixes tracing of skipped suites in MUnit 1.0.1
Additional Notes
In MUnit 1.0.1 release a different set of events is fired for skipped test suites.
Previous versions of the framework fixed start/ignore/finish events for such suites.
The new version of the framework only fires the ignore event.
This appears to be an accidental side-effect of an unrelated change, and an issue was reported in the framework repo.
It does not look, however, like this is getting fixed any time soon, so a workaround is introduced in the tracer.
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: SDTEST-843