Tracing.Builder.alwaysReportSpans() which sends locally sampled data to the normal reporter #958
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
brave/src/main/java/brave/propagation/ExtraFieldPropagation.java
Outdated
Show resolved
Hide resolved
| @@ -330,6 +332,42 @@ | |||
| .isEqualTo("12345"); | |||
| } | |||
|
|
|||
| /** Example that sets local sampled when in an experiment. */ | |||
| final static class LocalSampleCustomizer extends Customizer { | |||
There was a problem hiding this comment.
here's an example but a more thorough one will come later
|
PS here's the before and after, porting redaction to the new api. At first glance, I'm happy with it and don't think anyone will be able to notice a difference: before: after: |
| * contain data the backend needs to route to appropriate data stores, you are done. If it needs a | ||
| * hint, you can add a {@link FinishedSpanHandler} to add a tag which ensures the backend consistently | ||
| * and statelessly honors the sampling intent. | ||
| */ |
There was a problem hiding this comment.
haven't changed the example code, yet, but this is the design. I still need to run it through @narayaruna to ensure I am following things properly
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
ef05688 to
bf41390
Compare
|
chatting with nara again, seems eventually they will need edge (root but ttl=1) sampling so probably best to make the example sophisticated enough to show how to do this. |
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
bf41390 to
dfb132b
Compare
| * small subset of the architecture starting at an arbitrary place. As such, triage is more advanced | ||
| * that the other two systems. | ||
| * | ||
| * <h2>Trace data forwarder</h2> |
There was a problem hiding this comment.
@cwensel @drolando @worldtiki @openzipkin/core this example will introduce a familiar concept to you, trace data forwarder. In this case, it is different than "firehose handler" as the app is not responsible for deciding to send to two places. Instead, we push that responsibility into the zipkin-compatible endpoint at the cost of adding a tag to help that endpoint know where to divert data to (in some cases sending the same json to multiple places)
|
I think the ExtraFieldPropagation.Customizer should be self-contained which would allow it to define the fields it needs also. probably this hints at a name Plugin instead. Basically, this will allow the state management code from edge and triage to be in different files, allowing folks to progressively add new systems in a decoupled fashion. |
jcchavezs
reviewed
Aug 12, 2019
| * will be called for each field added to {@link FactoryBuilder}, in the same order they were, | ||
| * added and regardless of whether the field is present or not. | ||
| */ | ||
| public interface FieldCustomizer { |
There was a problem hiding this comment.
I am curious about the contracts on this interface. I see this method doing two things (setting a value or dropping a value) and I would like to understand the reason for not having two methods.
There was a problem hiding this comment.
because it is easier this way, also you can implement single-method types in a lambda.
There was a problem hiding this comment.
by easier I mean there's a very direct relationship without ordering concerns about what we are doing here. it is also easier on the code to not have to enter two methods for the same outcome (ex the callsite is a loop of only one method call vs two calls one to change one to drop)
codefromthecrypt
changed the title
|
need to rethink this if we are intentionally allowing duplicate span handlers. My hope was that someone can drop in a jar into something like spring boot that implements like this: @Configuration
class AddContextPlugins {
@Bean Plugin triageSampler(){...}
@Bean Plugin customCorrelationField(){...}
@Bean Plugin w3cTraceContext(){...}
}The implementation would implicitly register a cc @anuraaga |
codefromthecrypt
force-pushed
the
extra-customizer
branch
2 times, most recently
from
44e4bb7 to
b5847eb
Compare
devinsba
reviewed
Aug 14, 2019
brave/src/main/java/brave/propagation/ExtraFieldPropagation.java
Outdated
Show resolved
Hide resolved
| Factory(Propagation.Factory delegate, String[] fieldNames, String[] keyNames, BitSet redacted) { | ||
| this(delegate, fieldNames, keyNames, keyToField(keyNames), redacted); | ||
| @SuppressWarnings("ThreadLocalUsage") // intentional: instances may have different plugin counts | ||
| final ThreadLocal<Object[]> FIELD_UPDATERS = new ThreadLocal<>(); |
There was a problem hiding this comment.
Hmm, can you explain this further (even in just a reply here). I don't follow why a thread local is necessary here, I must be missing some context
There was a problem hiding this comment.
otherwise we allocate one throw-away array for each incoming request and each outgoing one, on top of the other overhead added by this feature. This is an attempt to reduce the garbage impact.
There was a problem hiding this comment.
Anyway good question. basically I used to only put in features with one thing like one Reporter<Span>, which you wouldn't need to loop over. However in plugins, this was designed for multiple and we can be a bit more efficient than end users are likely to be when trying to compose them.
I will cleanup this anyway, for example only using it when there are in fact multiple plugins present. that will help isolate the benchmarking concern, too, and protect the unsuspecting reviewer from seeing too much in one file, too!
brave/src/test/java/brave/features/propagation/SecondarySamplingTest.java
Outdated
Show resolved
Hide resolved
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
c1e519f to
0266a4c
Compare
|
ok I rebased and polished up some refactorings.. current known TODOs:
|
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
0266a4c to
bd5de7a
Compare
brave/src/test/java/brave/features/propagation/SecondarySamplingTest.java
Outdated
Show resolved
Hide resolved
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
5192ba0 to
e95de8e
Compare
|
owe some changes here, notably the tag used for sampling should be split into separate routing tags. ex "sampled.zipkin" ->"", "sampled.edge" ->"" vs a shared tag "sampled" -> "zipkin,edge". this decouples more and is easier on the back end cc @narayaruna |
|
@openzipkin/core For those interested, I had to make a traffic simulator to verify things. Mainly, it is hard to understand where sampling starts and stops if you only have 2 services. You really need at least 10 I think to "get" the value of subtree sampling. https://github.com/openzipkin/brave/blob/extra-customizer/brave/src/test/java/brave/features/propagation/TracedNode.java creates a service graph based on a made-up movie service with a static route through 2 paths intentionally overlapping some services. Map<String, String> initialContext = new LinkedHashMap<>();
initialContext.put("b3", "1");
gateway.execute("/recommend", initialContext);
gateway.execute("/play", initialContext);Unit tests make graph assertions, but if you were to look at the json, they are somewhat realistic, even if simplified. You can load it into lens for example:
Here are the json if you want to just look for yourself /recommend [
{
"traceId": "a468337851abb70b",
"parentId": "31ba4774a2efd405",
"id": "df3db528d93870ba",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713691469,
"duration": 16,
"localEndpoint": {
"serviceName": "authdb",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "31ba4774a2efd405",
"id": "df3db528d93870ba",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713691442,
"duration": 11749,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"parentId": "2adc9b0a7b3f60ce",
"id": "31ba4774a2efd405",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713691428,
"duration": 12161,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "2adc9b0a7b3f60ce",
"id": "31ba4774a2efd405",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713691402,
"duration": 12349,
"localEndpoint": {
"serviceName": "auth",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"parentId": "a7d2afa4a0361d6e",
"id": "2adc9b0a7b3f60ce",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713691385,
"duration": 12610,
"localEndpoint": {
"serviceName": "auth",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "a7d2afa4a0361d6e",
"id": "2adc9b0a7b3f60ce",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713691352,
"duration": 12759,
"localEndpoint": {
"serviceName": "api",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"parentId": "7a2719293544afc3",
"id": "092412b2a141d7cf",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713704567,
"duration": 4,
"localEndpoint": {
"serviceName": "recodb",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "7a2719293544afc3",
"id": "092412b2a141d7cf",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713704537,
"duration": 1175,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"parentId": "76e442e0b621230e",
"id": "7a2719293544afc3",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713704520,
"duration": 1478,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "76e442e0b621230e",
"id": "7a2719293544afc3",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713704491,
"duration": 1596,
"localEndpoint": {
"serviceName": "recommendations",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"parentId": "a7d2afa4a0361d6e",
"id": "76e442e0b621230e",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713704470,
"duration": 1808,
"localEndpoint": {
"serviceName": "recommendations",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "a7d2afa4a0361d6e",
"id": "76e442e0b621230e",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713704423,
"duration": 1939,
"localEndpoint": {
"serviceName": "api",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"parentId": "a468337851abb70b",
"id": "a7d2afa4a0361d6e",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713691320,
"duration": 15237,
"localEndpoint": {
"serviceName": "api",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "a468337851abb70b",
"parentId": "a468337851abb70b",
"id": "a7d2afa4a0361d6e",
"kind": "CLIENT",
"name": "/recommend",
"timestamp": 1567212713689598,
"duration": 16945,
"localEndpoint": {
"serviceName": "gateway",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "a468337851abb70b",
"id": "a468337851abb70b",
"kind": "SERVER",
"name": "/recommend",
"timestamp": 1567212713688911,
"duration": 17716,
"localEndpoint": {
"serviceName": "gateway",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
}
]/play [
{
"traceId": "df6e654722df5dc3",
"parentId": "6d9412fcae7ca484",
"id": "ae953dc1896a55f4",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812427713,
"duration": 17,
"localEndpoint": {
"serviceName": "authdb",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "6d9412fcae7ca484",
"id": "ae953dc1896a55f4",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812427690,
"duration": 14235,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "0d44c4974c51f286",
"id": "6d9412fcae7ca484",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812427676,
"duration": 14670,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "0d44c4974c51f286",
"id": "6d9412fcae7ca484",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812427649,
"duration": 14785,
"localEndpoint": {
"serviceName": "auth",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "69f5a557048760da",
"id": "0d44c4974c51f286",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812427633,
"duration": 14993,
"localEndpoint": {
"serviceName": "auth",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "69f5a557048760da",
"id": "0d44c4974c51f286",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812427606,
"duration": 15099,
"localEndpoint": {
"serviceName": "api",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "ac9175fb1a4720d7",
"id": "3f16f8d5292690e4",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812444761,
"duration": 9,
"localEndpoint": {
"serviceName": "licensedb",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "ac9175fb1a4720d7",
"id": "3f16f8d5292690e4",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812444675,
"duration": 275,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "fa11333db32d3089",
"id": "ac9175fb1a4720d7",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812443161,
"duration": 2109,
"localEndpoint": {
"serviceName": "cache",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "fa11333db32d3089",
"id": "ac9175fb1a4720d7",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812443119,
"duration": 2269,
"localEndpoint": {
"serviceName": "license",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "c44e913507e718fc",
"id": "fa11333db32d3089",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812443096,
"duration": 2569,
"localEndpoint": {
"serviceName": "license",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "c44e913507e718fc",
"id": "fa11333db32d3089",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812443059,
"duration": 2724,
"localEndpoint": {
"serviceName": "playback",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "c44e913507e718fc",
"id": "15467d016028a262",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812446190,
"duration": 7,
"localEndpoint": {
"serviceName": "moviemetadata",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "c44e913507e718fc",
"id": "15467d016028a262",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812446144,
"duration": 208,
"localEndpoint": {
"serviceName": "playback",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "c44e913507e718fc",
"id": "50812c6770bde0af",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812446793,
"duration": 7,
"localEndpoint": {
"serviceName": "streams",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "c44e913507e718fc",
"id": "50812c6770bde0af",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812446742,
"duration": 198,
"localEndpoint": {
"serviceName": "playback",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "69f5a557048760da",
"id": "c44e913507e718fc",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812443034,
"duration": 4228,
"localEndpoint": {
"serviceName": "playback",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "69f5a557048760da",
"id": "c44e913507e718fc",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812442981,
"duration": 4409,
"localEndpoint": {
"serviceName": "api",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"parentId": "df6e654722df5dc3",
"id": "69f5a557048760da",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812427578,
"duration": 20095,
"localEndpoint": {
"serviceName": "api",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
},
"shared": true
},
{
"traceId": "df6e654722df5dc3",
"parentId": "df6e654722df5dc3",
"id": "69f5a557048760da",
"kind": "CLIENT",
"name": "/play",
"timestamp": 1567212812426373,
"duration": 21359,
"localEndpoint": {
"serviceName": "gateway",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
},
{
"traceId": "df6e654722df5dc3",
"id": "df6e654722df5dc3",
"kind": "SERVER",
"name": "/play",
"timestamp": 1567212812425805,
"duration": 22057,
"localEndpoint": {
"serviceName": "gateway",
"ipv4": "192.168.0.101"
},
"tags": {
"systems": "zipkin"
}
}
] |
jeqo
reviewed
Sep 1, 2019
| * <p>Secondary fields and state management about them is the responsibility of {@link | ||
| * ExtraFieldPropagation}. Here, you can add an extra field that corresponds to a header sent out of | ||
| * process. You can also add an {@link ExtraFieldPlugin} to ensure a relevant tag gets to the trace | ||
| * forwarder. |
There was a problem hiding this comment.
This is awesome! Probably worth it to have a copy on Zipkin's wiki
There was a problem hiding this comment.
good idea I didn't even consider that. this is a massive javadoc :)
|
I will leave docs here until more stable (tomorrow?) then move to wiki. One thing I notice is poor choice of jargon:
anyway I think the above help clarify a bit. |
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
b9eb0aa to
bbd55c9
Compare
|
took out using a different propagated field per sampler as it makes integration unnecessarily complicated. This buys back time and energy to focus on more important things, such as http request parsing. |
codefromthecrypt
pushed a commit
that referenced
this pull request
Sep 2, 2019
Before, we allowed the "carrier" of trace context to be different than the request object. This is more helpful on the inject side than the extract side. If we have only one parameter extract side, we can do more for example, secondary sampling which considers the http path using `brave.http.HttpServerParser` See #958
|
the solution to the http-layer sampling problem is to introduce a new type HttpServerRequest/Response which is similar to zipkin-go. This allows us to inspect the type of the carrier and then consider other parts than just http headers when deciding a secondary decision. I have a little refactoring homework before this can complete (especially keeping api compatible) |
codefromthecrypt
pushed a commit
that referenced
this pull request
Sep 3, 2019
Initially, we had a partial function of (adapter, request) to parse attributes like the HTTP path from a request. This made sense in the http abstraction, as all types that needed these attributes, such as parsers and samplers, were in the same package and library. It also saved an object allocation otherwise needed to wrap a native type, and the indirection of unwrapping as needed. This worked, but especially on the response path this broke down. We found that attributes in the request, such as the method and matched route, are needed at response time, but aren't themselves a part of the response. Over time, we accumulated response wrappers or thread locals to pass these attributes to the response as many response objects lacked a property bag. Then, we noticed people wanting to do more during extraction than look at headers. For example, Netflix wanted to inspect the http path to make a secondary sampling decision. Without a known http request type, this couldn't be accomplished portably. Finally, we noticed zipkin-go side-step this problem by defining a top-level type for http requests and responses. This allows the same object to be used regardless of purpose, whether that is primary or secondary sampling, or tag parsing. All of this led to the introduction of `HttpServerRequest` and `HttpServerResponse` types in Brave 5.7, retrofitted into the original `HttpServerAdapter` so as to not break API. See #958
codefromthecrypt
commented
Sep 6, 2019
| * @see ExtraFieldPropagation.FactoryBuilder#addPlugin(ExtraFieldPlugin) | ||
| * @since 5.7 | ||
| */ | ||
| public Builder alwaysReportSpans() { |
There was a problem hiding this comment.
ps to implement this in brave 5.7 we can use a finished span handler that wraps normal reporter and then sets tracing.builder.spanReporter to noop
|
After various discussions, it seems like the actual use case is not required to have multiple components collaborating. For example, the intent is to use the same propagated field for everything mildly renamed to something like This means the original design is closer to the mark. I'm going to bring that back as this code wandered a lot based on some false assumption that the customer support style sampling would be firewalled off (ex using different state header) than the edge sampling |
codefromthecrypt
force-pushed
the
extra-customizer
branch
2 times, most recently
from
178ed5c to
4a132a1
Compare
codefromthecrypt
changed the title
| * This is a <a href="https://github.com/openzipkin/openzipkin.github.io/wiki/secondary-sampling">Secondary | ||
| * Sampling</a> proof of concept. | ||
| */ | ||
| public class SecondarySamplingIntegratedTest { |
There was a problem hiding this comment.
this is now converged, one scenario (skipped hops in gatewayplay) doesn't work yet
|
everything works now as far as I can tell. I'll retrofit the wiki next. |
codefromthecrypt
force-pushed
the
extra-customizer
branch
from
75d28a2 to
0c33588
Compare
added 2 commits
September 12, 2019 18:05
This code might eventually migrate out of the test tree to a separate module or repository. See https://github.com/openzipkin/openzipkin.github.io/wiki/secondary-sampling#the-trace-forwarder
|
closed via 17fe478 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently, we have a test
SecondarySamplingwhich is complex toimplement as it assumes conditional decisions will result in use of
different reporting libraries. Ex normal zipkin for sampled data, and
a different format for secondary sinks.
After a chat with Nara, we realized that data routing logic is cheapest
to address outside the process. In other words, if a secondary sampling
decision says "record" but only for a part of the network, this data
for that subtree will still end up posting to the same place.
This adds
Tracing.Builder.alwaysReportSpans()which allows the normal span reporter to get everything recorded.Fixes #955