-
Notifications
You must be signed in to change notification settings - Fork 292
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support reactor context span propagation #7864
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 9 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.086 s) : 0, 1085558
Total [baseline] (8.547 s) : 0, 8547223
Agent [candidate] (1.088 s) : 0, 1087673
Total [candidate] (8.587 s) : 0, 8587012
section iast
Agent [baseline] (1.203 s) : 0, 1203037
Total [baseline] (9.096 s) : 0, 9096258
Agent [candidate] (1.204 s) : 0, 1203899
Total [candidate] (9.149 s) : 0, 9148769
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.202 s) : 0, 1202494
Total [baseline] (9.088 s) : 0, 9087742
Agent [candidate] (1.204 s) : 0, 1204493
Total [candidate] (9.083 s) : 0, 9083069
section iast_TELEMETRY_OFF
Agent [baseline] (1.199 s) : 0, 1199422
Total [baseline] (9.113 s) : 0, 9113331
Agent [candidate] (1.201 s) : 0, 1200734
Total [candidate] (9.101 s) : 0, 9100886
gantt
title insecure-bank - break down per module: candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (691.028 ms) : 0, 691028
BytebuddyAgent [candidate] (692.405 ms) : 0, 692405
GlobalTracer [baseline] (316.182 ms) : 0, 316182
GlobalTracer [candidate] (314.928 ms) : 0, 314928
AppSec [baseline] (54.119 ms) : 0, 54119
AppSec [candidate] (54.139 ms) : 0, 54139
Remote Config [baseline] (666.793 µs) : 0, 667
Remote Config [candidate] (659.822 µs) : 0, 660
Telemetry [baseline] (9.8 ms) : 0, 9800
Telemetry [candidate] (11.786 ms) : 0, 11786
section iast
BytebuddyAgent [baseline] (801.397 ms) : 0, 801397
BytebuddyAgent [candidate] (802.605 ms) : 0, 802605
GlobalTracer [baseline] (303.035 ms) : 0, 303035
GlobalTracer [candidate] (303.144 ms) : 0, 303144
AppSec [baseline] (55.681 ms) : 0, 55681
AppSec [candidate] (56.04 ms) : 0, 56040
Remote Config [baseline] (591.017 µs) : 0, 591
Remote Config [candidate] (593.365 µs) : 0, 593
Telemetry [baseline] (8.121 ms) : 0, 8121
Telemetry [candidate] (8.123 ms) : 0, 8123
IAST [baseline] (20.562 ms) : 0, 20562
IAST [candidate] (19.697 ms) : 0, 19697
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (800.597 ms) : 0, 800597
BytebuddyAgent [candidate] (802.247 ms) : 0, 802247
GlobalTracer [baseline] (303.024 ms) : 0, 303024
GlobalTracer [candidate] (302.98 ms) : 0, 302980
AppSec [baseline] (55.913 ms) : 0, 55913
AppSec [candidate] (56.568 ms) : 0, 56568
Remote Config [baseline] (594.666 µs) : 0, 595
Remote Config [candidate] (605.58 µs) : 0, 606
Telemetry [baseline] (7.338 ms) : 0, 7338
Telemetry [candidate] (7.422 ms) : 0, 7422
IAST [baseline] (21.365 ms) : 0, 21365
IAST [candidate] (21.008 ms) : 0, 21008
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (797.449 ms) : 0, 797449
BytebuddyAgent [candidate] (798.746 ms) : 0, 798746
GlobalTracer [baseline] (303.029 ms) : 0, 303029
GlobalTracer [candidate] (302.732 ms) : 0, 302732
AppSec [baseline] (56.715 ms) : 0, 56715
AppSec [candidate] (57.751 ms) : 0, 57751
Remote Config [baseline] (601.042 µs) : 0, 601
Remote Config [candidate] (607.375 µs) : 0, 607
Telemetry [baseline] (7.376 ms) : 0, 7376
Telemetry [candidate] (7.407 ms) : 0, 7407
IAST [baseline] (20.557 ms) : 0, 20557
IAST [candidate] (19.894 ms) : 0, 19894
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.074 s) : 0, 1074151
Total [baseline] (10.419 s) : 0, 10418623
Agent [candidate] (1.081 s) : 0, 1080824
Total [candidate] (10.41 s) : 0, 10409737
section appsec
Agent [baseline] (1.222 s) : 0, 1222226
Total [baseline] (10.689 s) : 0, 10688556
Agent [candidate] (1.211 s) : 0, 1210525
Total [candidate] (10.654 s) : 0, 10654023
section iast
Agent [baseline] (1.205 s) : 0, 1205414
Total [baseline] (10.85 s) : 0, 10849787
Agent [candidate] (1.214 s) : 0, 1213749
Total [candidate] (10.859 s) : 0, 10859269
section profiling
Agent [baseline] (1.288 s) : 0, 1287937
Total [baseline] (10.748 s) : 0, 10747557
Agent [candidate] (1.29 s) : 0, 1289946
Total [candidate] (10.778 s) : 0, 10777515
gantt
title petclinic - break down per module: candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section tracing
BytebuddyAgent [baseline] (684.732 ms) : 0, 684732
BytebuddyAgent [candidate] (687.412 ms) : 0, 687412
GlobalTracer [baseline] (313.582 ms) : 0, 313582
GlobalTracer [candidate] (314.965 ms) : 0, 314965
AppSec [baseline] (53.894 ms) : 0, 53894
AppSec [candidate] (54.232 ms) : 0, 54232
Remote Config [baseline] (659.297 µs) : 0, 659
Remote Config [candidate] (666.278 µs) : 0, 666
Telemetry [baseline] (7.661 ms) : 0, 7661
Telemetry [candidate] (9.895 ms) : 0, 9895
section appsec
BytebuddyAgent [baseline] (709.657 ms) : 0, 709657
BytebuddyAgent [candidate] (702.58 ms) : 0, 702580
GlobalTracer [baseline] (314.42 ms) : 0, 314420
GlobalTracer [candidate] (311.397 ms) : 0, 311397
AppSec [baseline] (166.185 ms) : 0, 166185
AppSec [candidate] (163.205 ms) : 0, 163205
Remote Config [baseline] (648.534 µs) : 0, 649
Remote Config [candidate] (628.931 µs) : 0, 629
Telemetry [baseline] (7.433 ms) : 0, 7433
Telemetry [candidate] (8.448 ms) : 0, 8448
IAST [baseline] (20.208 ms) : 0, 20208
IAST [candidate] (21.302 ms) : 0, 21302
section iast
BytebuddyAgent [baseline] (803.087 ms) : 0, 803087
BytebuddyAgent [candidate] (807.718 ms) : 0, 807718
GlobalTracer [baseline] (303.178 ms) : 0, 303178
GlobalTracer [candidate] (305.864 ms) : 0, 305864
AppSec [baseline] (57.343 ms) : 0, 57343
AppSec [candidate] (58.341 ms) : 0, 58341
Remote Config [baseline] (596.19 µs) : 0, 596
Remote Config [candidate] (608.077 µs) : 0, 608
Telemetry [baseline] (7.487 ms) : 0, 7487
Telemetry [candidate] (7.477 ms) : 0, 7477
IAST [baseline] (20.035 ms) : 0, 20035
IAST [candidate] (20.021 ms) : 0, 20021
section profiling
BytebuddyAgent [baseline] (686.536 ms) : 0, 686536
BytebuddyAgent [candidate] (689.197 ms) : 0, 689197
GlobalTracer [baseline] (401.701 ms) : 0, 401701
GlobalTracer [candidate] (401.672 ms) : 0, 401672
AppSec [baseline] (54.782 ms) : 0, 54782
AppSec [candidate] (55.0 ms) : 0, 55000
Remote Config [baseline] (679.986 µs) : 0, 680
Remote Config [candidate] (680.721 µs) : 0, 681
Telemetry [baseline] (12.072 ms) : 0, 12072
Telemetry [candidate] (12.059 ms) : 0, 12059
ProfilingAgent [baseline] (93.029 ms) : 0, 93029
ProfilingAgent [candidate] (92.184 ms) : 0, 92184
Profiling [baseline] (93.052 ms) : 0, 93052
Profiling [candidate] (92.207 ms) : 0, 92207
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics. Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section baseline
no_agent (366.492 µs) : 347, 386
. : milestone, 366,
iast (487.43 µs) : 466, 509
. : milestone, 487,
iast_FULL (638.184 µs) : 617, 659
. : milestone, 638,
iast_GLOBAL (510.634 µs) : 489, 532
. : milestone, 511,
iast_HARDCODED_SECRET_DISABLED (490.938 µs) : 470, 512
. : milestone, 491,
iast_INACTIVE (451.402 µs) : 430, 473
. : milestone, 451,
iast_TELEMETRY_OFF (475.489 µs) : 454, 497
. : milestone, 475,
tracing (440.278 µs) : 420, 461
. : milestone, 440,
section candidate
no_agent (364.651 µs) : 344, 385
. : milestone, 365,
iast (487.806 µs) : 466, 509
. : milestone, 488,
iast_FULL (646.734 µs) : 625, 668
. : milestone, 647,
iast_GLOBAL (524.066 µs) : 501, 547
. : milestone, 524,
iast_HARDCODED_SECRET_DISABLED (486.679 µs) : 465, 508
. : milestone, 487,
iast_INACTIVE (443.776 µs) : 423, 464
. : milestone, 444,
iast_TELEMETRY_OFF (479.089 µs) : 458, 500
. : milestone, 479,
tracing (442.873 µs) : 422, 464
. : milestone, 443,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section baseline
no_agent (1.328 ms) : 1309, 1347
. : milestone, 1328,
appsec (1.724 ms) : 1701, 1747
. : milestone, 1724,
appsec_no_iast (1.742 ms) : 1719, 1766
. : milestone, 1742,
iast (1.483 ms) : 1460, 1505
. : milestone, 1483,
profiling (1.494 ms) : 1472, 1516
. : milestone, 1494,
tracing (1.483 ms) : 1458, 1507
. : milestone, 1483,
section candidate
no_agent (1.339 ms) : 1320, 1358
. : milestone, 1339,
appsec (1.74 ms) : 1717, 1763
. : milestone, 1740,
appsec_no_iast (1.735 ms) : 1711, 1759
. : milestone, 1735,
iast (1.466 ms) : 1443, 1488
. : milestone, 1466,
profiling (1.549 ms) : 1514, 1584
. : milestone, 1549,
tracing (1.463 ms) : 1438, 1488
. : milestone, 1463,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section baseline
no_agent (1.477 ms) : 1466, 1489
. : milestone, 1477,
appsec (2.341 ms) : 2300, 2382
. : milestone, 2341,
iast (2.08 ms) : 2028, 2131
. : milestone, 2080,
iast_GLOBAL (2.13 ms) : 2077, 2182
. : milestone, 2130,
profiling (1.939 ms) : 1897, 1980
. : milestone, 1939,
tracing (1.917 ms) : 1878, 1957
. : milestone, 1917,
section candidate
no_agent (1.482 ms) : 1470, 1494
. : milestone, 1482,
appsec (2.359 ms) : 2318, 2401
. : milestone, 2359,
iast (2.088 ms) : 2036, 2140
. : milestone, 2088,
iast_GLOBAL (2.137 ms) : 2085, 2189
. : milestone, 2137,
profiling (1.958 ms) : 1916, 2000
. : milestone, 1958,
tracing (1.932 ms) : 1892, 1972
. : milestone, 1932,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.42.0-SNAPSHOT~a10e1d2ca0, baseline=1.42.0-SNAPSHOT~03ad4845ab
dateFormat X
axisFormat %s
section baseline
no_agent (15.515 s) : 15515000, 15515000
. : milestone, 15515000,
appsec (15.239 s) : 15239000, 15239000
. : milestone, 15239000,
iast (18.981 s) : 18981000, 18981000
. : milestone, 18981000,
iast_GLOBAL (17.815 s) : 17815000, 17815000
. : milestone, 17815000,
profiling (15.785 s) : 15785000, 15785000
. : milestone, 15785000,
tracing (15.514 s) : 15514000, 15514000
. : milestone, 15514000,
section candidate
no_agent (15.063 s) : 15063000, 15063000
. : milestone, 15063000,
appsec (15.135 s) : 15135000, 15135000
. : milestone, 15135000,
iast (18.703 s) : 18703000, 18703000
. : milestone, 18703000,
iast_GLOBAL (18.035 s) : 18035000, 18035000
. : milestone, 18035000,
profiling (15.719 s) : 15719000, 15719000
. : milestone, 15719000,
tracing (15.081 s) : 15081000, 15081000
. : milestone, 15081000,
|
...r-core-3.1/src/main/java/datadog/trace/instrumentation/reactor/core/SpanExtractorHelper.java
Outdated
Show resolved
Hide resolved
.../src/main/java/datadog/trace/instrumentation/reactor/core/CoreSubscriberInstrumentation.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice refactoring, just a minor comment about re-using the value of self.currentContext()
@@ -0,0 +1,5 @@ | |||
package datadog.trace.bootstrap.instrumentation.api; | |||
|
|||
public interface WithAgentSpan { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note for future improvements: we might expose the behaviour (i.e. activate()
) instead the internal AgentSpan object
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As future design idea, I would like to have the OT/OTel span capabilities exposed rather than their internals (delegate span).
For example, having an ActivableSpan
interface that exposes the activate()
method rather than WithAgentSpan
that exposes the delegate
internal.
if (context.hasKey("dd.span")) {
Object maybeSpan = context.get("dd.span");
if (maybeSpan instanceof WithAgentSpan) {
AgentSpan span = ((WithAgentSpan) maybeSpan).asAgentSpan();
if (span != null) {
return activateSpan(span);
}
}
}
will become:
if (context.hasKey("dd.span")) {
Object maybeSpan = context.get("dd.span");
if (maybeSpan instanceof ActivableSpan) {
return ((ActivableSpan) maybeSpan).activate();
}
}
Even though OT/OTel spans should rely on AgentSpan
for a long time, it feels safer to expose capability rather than internal. That's not blocking though :)
What Does This Do
This PR add the possibility to manually add the current span in the reactor subscriber context using the key
dd.span
and allows addressing top-down context propagation.Then, when the publisher has a span in its context will activate this span when it will signal downstream (
onNext
,onError
,onComplete
). That will make downstream operators using this span as active as well.Here above few examples (see also unit tests on this PR for more):
This is the basic example where context propagation happens only bottom up
The scope of
parent
is propagated upstream whenblock()
is called. The scope forchild
is not propagated upstream .The produced trace looks like
[parent[child][last]
We now change slightly the previous example to let
child
propagating downstream when the flatmap emits.When we indicate that we want to emit
child
as active span, the map operator will see now that span and not theparent
span captured whensubscribe
was calledHence the trace structure is now
[parent[child[last]]]
Previously, forcing an operator to emit a different active span caused all the downstream chain to see that span as active.
We might want to change the active span on the way down. The example will be now:
Each time we use
contextWrite
settingdd.span
it will propagate upstream that information so we should carefully set and reset that information in the right placesThe trace structure should look like
[parent[child][last]]
Motivation
Additional Notes
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]