-
Notifications
You must be signed in to change notification settings - Fork 292
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable telemetry logs by default #7631
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
smola
force-pushed
the
smola/enable-telemetry-logs-globally
branch
from
September 17, 2024 15:14
c40213a
to
b645139
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 47 metrics, 15 unstable metrics.
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.062 s) : 0, 1062069
Total [baseline] (10.355 s) : 0, 10355354
Agent [candidate] (1.064 s) : 0, 1063891
Total [candidate] (10.415 s) : 0, 10414830
section appsec
Agent [baseline] (1.205 s) : 0, 1204575
Total [baseline] (10.614 s) : 0, 10613814
Agent [candidate] (1.199 s) : 0, 1198658
Total [candidate] (10.601 s) : 0, 10601394
section iast
Agent [baseline] (1.189 s) : 0, 1189304
Total [baseline] (10.82 s) : 0, 10819702
Agent [candidate] (1.195 s) : 0, 1194749
Total [candidate] (10.866 s) : 0, 10866309
section profiling
Agent [baseline] (1.262 s) : 0, 1262418
Total [baseline] (10.617 s) : 0, 10616972
Agent [candidate] (1.271 s) : 0, 1270964
Total [candidate] (10.548 s) : 0, 10547663
gantt
title petclinic - break down per module: candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (678.342 ms) : 0, 678342
BytebuddyAgent [candidate] (679.742 ms) : 0, 679742
GlobalTracer [baseline] (309.153 ms) : 0, 309153
GlobalTracer [candidate] (309.566 ms) : 0, 309566
AppSec [baseline] (52.633 ms) : 0, 52633
AppSec [candidate] (52.665 ms) : 0, 52665
Remote Config [baseline] (660.062 µs) : 0, 660
Remote Config [candidate] (661.241 µs) : 0, 661
Telemetry [baseline] (7.568 ms) : 0, 7568
Telemetry [candidate] (7.595 ms) : 0, 7595
section appsec
BytebuddyAgent [baseline] (705.618 ms) : 0, 705618
BytebuddyAgent [candidate] (702.078 ms) : 0, 702078
GlobalTracer [baseline] (303.816 ms) : 0, 303816
GlobalTracer [candidate] (303.486 ms) : 0, 303486
AppSec [baseline] (161.398 ms) : 0, 161398
AppSec [candidate] (161.098 ms) : 0, 161098
Remote Config [baseline] (625.061 µs) : 0, 625
Remote Config [candidate] (624.391 µs) : 0, 624
Telemetry [baseline] (9.021 ms) : 0, 9021
Telemetry [candidate] (8.133 ms) : 0, 8133
IAST [baseline] (20.688 ms) : 0, 20688
IAST [candidate] (19.078 ms) : 0, 19078
section iast
BytebuddyAgent [baseline] (791.95 ms) : 0, 791950
BytebuddyAgent [candidate] (796.067 ms) : 0, 796067
GlobalTracer [baseline] (298.541 ms) : 0, 298541
GlobalTracer [candidate] (299.624 ms) : 0, 299624
AppSec [baseline] (56.064 ms) : 0, 56064
AppSec [candidate] (54.412 ms) : 0, 54412
Remote Config [baseline] (619.358 µs) : 0, 619
Remote Config [candidate] (619.772 µs) : 0, 620
Telemetry [baseline] (7.396 ms) : 0, 7396
Telemetry [candidate] (7.173 ms) : 0, 7173
IAST [baseline] (21.016 ms) : 0, 21016
IAST [candidate] (23.047 ms) : 0, 23047
section profiling
ProfilingAgent [baseline] (96.489 ms) : 0, 96489
ProfilingAgent [candidate] (96.819 ms) : 0, 96819
BytebuddyAgent [baseline] (673.828 ms) : 0, 673828
BytebuddyAgent [candidate] (678.72 ms) : 0, 678720
GlobalTracer [baseline] (392.569 ms) : 0, 392569
GlobalTracer [candidate] (394.196 ms) : 0, 394196
AppSec [baseline] (53.172 ms) : 0, 53172
AppSec [candidate] (53.948 ms) : 0, 53948
Remote Config [baseline] (650.787 µs) : 0, 651
Remote Config [candidate] (655.621 µs) : 0, 656
Telemetry [baseline] (7.435 ms) : 0, 7435
Telemetry [candidate] (7.519 ms) : 0, 7519
Profiling [baseline] (96.513 ms) : 0, 96513
Profiling [candidate] (96.843 ms) : 0, 96843
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.078 s) : 0, 1078106
Total [baseline] (8.557 s) : 0, 8556998
Agent [candidate] (1.077 s) : 0, 1076913
Total [candidate] (8.601 s) : 0, 8600669
section iast
Agent [baseline] (1.187 s) : 0, 1187347
Total [baseline] (8.986 s) : 0, 8985513
Agent [candidate] (1.196 s) : 0, 1195708
Total [candidate] (8.996 s) : 0, 8995764
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.191 s) : 0, 1190698
Total [baseline] (9.009 s) : 0, 9009162
Agent [candidate] (1.19 s) : 0, 1190093
Total [candidate] (9.024 s) : 0, 9023793
section iast_TELEMETRY_OFF
Agent [baseline] (1.188 s) : 0, 1188096
Total [baseline] (8.981 s) : 0, 8980906
Agent [candidate] (1.205 s) : 0, 1204527
Total [candidate] (9.021 s) : 0, 9020993
gantt
title insecure-bank - break down per module: candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (689.448 ms) : 0, 689448
BytebuddyAgent [candidate] (688.845 ms) : 0, 688845
GlobalTracer [baseline] (313.081 ms) : 0, 313081
GlobalTracer [candidate] (312.623 ms) : 0, 312623
AppSec [baseline] (53.305 ms) : 0, 53305
AppSec [candidate] (53.254 ms) : 0, 53254
Remote Config [baseline] (678.839 µs) : 0, 679
Remote Config [candidate] (658.952 µs) : 0, 659
Telemetry [baseline] (7.695 ms) : 0, 7695
Telemetry [candidate] (7.655 ms) : 0, 7655
section iast
BytebuddyAgent [baseline] (790.336 ms) : 0, 790336
BytebuddyAgent [candidate] (796.908 ms) : 0, 796908
GlobalTracer [baseline] (298.108 ms) : 0, 298108
GlobalTracer [candidate] (299.731 ms) : 0, 299731
AppSec [baseline] (54.88 ms) : 0, 54880
AppSec [candidate] (55.636 ms) : 0, 55636
IAST [baseline] (22.179 ms) : 0, 22179
IAST [candidate] (21.947 ms) : 0, 21947
Remote Config [baseline] (634.043 µs) : 0, 634
Remote Config [candidate] (600.16 µs) : 0, 600
Telemetry [baseline] (7.461 ms) : 0, 7461
Telemetry [candidate] (7.049 ms) : 0, 7049
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (792.123 ms) : 0, 792123
BytebuddyAgent [candidate] (792.401 ms) : 0, 792401
GlobalTracer [baseline] (298.691 ms) : 0, 298691
GlobalTracer [candidate] (298.589 ms) : 0, 298589
AppSec [baseline] (53.429 ms) : 0, 53429
AppSec [candidate] (53.933 ms) : 0, 53933
IAST [baseline] (24.638 ms) : 0, 24638
IAST [candidate] (22.953 ms) : 0, 22953
Remote Config [baseline] (639.689 µs) : 0, 640
Remote Config [candidate] (626.61 µs) : 0, 627
Telemetry [baseline] (7.42 ms) : 0, 7420
Telemetry [candidate] (7.849 ms) : 0, 7849
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (789.427 ms) : 0, 789427
BytebuddyAgent [candidate] (801.603 ms) : 0, 801603
GlobalTracer [baseline] (299.35 ms) : 0, 299350
GlobalTracer [candidate] (303.145 ms) : 0, 303145
AppSec [baseline] (51.883 ms) : 0, 51883
AppSec [candidate] (55.43 ms) : 0, 55430
IAST [baseline] (25.005 ms) : 0, 25005
IAST [candidate] (22.682 ms) : 0, 22682
Remote Config [baseline] (588.231 µs) : 0, 588
Remote Config [candidate] (599.651 µs) : 0, 600
Telemetry [baseline] (8.081 ms) : 0, 8081
Telemetry [candidate] (7.103 ms) : 0, 7103
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 9 metrics, 19 unstable metrics. Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section baseline
no_agent (1.319 ms) : 1300, 1338
. : milestone, 1319,
appsec (1.702 ms) : 1678, 1727
. : milestone, 1702,
appsec_no_iast (1.697 ms) : 1674, 1721
. : milestone, 1697,
iast (1.456 ms) : 1433, 1478
. : milestone, 1456,
profiling (1.461 ms) : 1438, 1483
. : milestone, 1461,
tracing (1.456 ms) : 1431, 1481
. : milestone, 1456,
section candidate
no_agent (1.316 ms) : 1296, 1336
. : milestone, 1316,
appsec (1.723 ms) : 1700, 1746
. : milestone, 1723,
appsec_no_iast (1.695 ms) : 1671, 1720
. : milestone, 1695,
iast (1.462 ms) : 1439, 1485
. : milestone, 1462,
profiling (1.462 ms) : 1439, 1485
. : milestone, 1462,
tracing (1.451 ms) : 1428, 1475
. : milestone, 1451,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section baseline
no_agent (353.421 µs) : 333, 374
. : milestone, 353,
iast (467.375 µs) : 446, 489
. : milestone, 467,
iast_FULL (530.785 µs) : 510, 552
. : milestone, 531,
iast_GLOBAL (484.934 µs) : 464, 506
. : milestone, 485,
iast_HARDCODED_SECRET_DISABLED (461.781 µs) : 440, 483
. : milestone, 462,
iast_INACTIVE (426.434 µs) : 405, 447
. : milestone, 426,
iast_TELEMETRY_OFF (455.273 µs) : 433, 478
. : milestone, 455,
tracing (426.435 µs) : 406, 447
. : milestone, 426,
section candidate
no_agent (347.47 µs) : 326, 369
. : milestone, 347,
iast (461.064 µs) : 440, 482
. : milestone, 461,
iast_FULL (538.598 µs) : 517, 560
. : milestone, 539,
iast_GLOBAL (493.832 µs) : 472, 516
. : milestone, 494,
iast_HARDCODED_SECRET_DISABLED (468.095 µs) : 446, 490
. : milestone, 468,
iast_INACTIVE (427.773 µs) : 407, 449
. : milestone, 428,
iast_TELEMETRY_OFF (457.623 µs) : 435, 480
. : milestone, 458,
tracing (421.911 µs) : 401, 443
. : milestone, 422,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section baseline
no_agent (1.444 ms) : 1432, 1455
. : milestone, 1444,
appsec (2.269 ms) : 2228, 2310
. : milestone, 2269,
iast (2.03 ms) : 1979, 2081
. : milestone, 2030,
iast_GLOBAL (2.071 ms) : 2020, 2122
. : milestone, 2071,
profiling (1.895 ms) : 1855, 1935
. : milestone, 1895,
tracing (1.88 ms) : 1842, 1919
. : milestone, 1880,
section candidate
no_agent (1.442 ms) : 1430, 1453
. : milestone, 1442,
appsec (2.259 ms) : 2218, 2300
. : milestone, 2259,
iast (2.016 ms) : 1966, 2066
. : milestone, 2016,
iast_GLOBAL (2.056 ms) : 2006, 2106
. : milestone, 2056,
profiling (1.887 ms) : 1848, 1927
. : milestone, 1887,
tracing (1.865 ms) : 1827, 1903
. : milestone, 1865,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.40.0-SNAPSHOT~b645139a6d, baseline=1.40.0-SNAPSHOT~2d2c02962a
dateFormat X
axisFormat %s
section baseline
no_agent (15.037 s) : 15037000, 15037000
. : milestone, 15037000,
appsec (15.082 s) : 15082000, 15082000
. : milestone, 15082000,
iast (18.638 s) : 18638000, 18638000
. : milestone, 18638000,
iast_GLOBAL (18.157 s) : 18157000, 18157000
. : milestone, 18157000,
profiling (15.604 s) : 15604000, 15604000
. : milestone, 15604000,
tracing (15.07 s) : 15070000, 15070000
. : milestone, 15070000,
section candidate
no_agent (15.698 s) : 15698000, 15698000
. : milestone, 15698000,
appsec (15.079 s) : 15079000, 15079000
. : milestone, 15079000,
iast (18.897 s) : 18897000, 18897000
. : milestone, 18897000,
iast_GLOBAL (18.362 s) : 18362000, 18362000
. : milestone, 18362000,
profiling (15.278 s) : 15278000, 15278000
. : milestone, 15278000,
tracing (15.202 s) : 15202000, 15202000
. : milestone, 15202000,
|
PerfectSlayer
approved these changes
Sep 23, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Enable telemetry logs by default. These were already enabled by default for services using JDK 8, Code Security (IAST), Dynamic Instrumentation, or CI Visibility. They will be enabled for all services from now on.
Users who want to opt out can set
-Ddd.telemetry.log-collection.enabled=false
(system property) orDD_TELEMETRY_LOG_COLLECTION_ENABLED=false
(environment variable).Note that these are not application logs, but internal dd-trace-java logs which we use for troubleshooting. They are heavily redacted by sending only the log formats and not log parameters (e.g. when we do
log.error("An error happened at url: {}", url)
), we will send just"An error happened at url: {}"
, without any placeholder substitution. We also send error stacktraces, including exception type (but not exception message), and any stackframes from dd-trace-java itself or the standard library.Motivation
We use these logs to proactively detect and fix dd-trace-java bugs.
Additional Notes
Previous activation milestones:
Jira: APPSEC-53432
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issue[ ] Update the public documentation in case of new configuration flag or behavior