Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v1.1.4] Not able to drop a trace after root span context is accessed with default RuleSampler #136

Open
tomthomson opened this issue May 11, 2020 · 1 comment

Comments

@tomthomson
Copy link

tomthomson commented May 11, 2020

With the new rules sampler in v1.1.4 I'm having problems to manually drop/keep a trace after a child span was created (and the root context has been accessed).
I'm using the default tracer with a default RuleSampler+PrioritySampler.

I created the following test to demonstrate my problem.
Currently the test fails because I cannot set the sampling priority after the root span context was accessed.

TEST_CASE(
    "sampling behaviour without mock sampler: sampling priority can be set until the root trace "
    "finishes") {
  datadog::opentracing::TracerOptions tracer_options{"localhost", 8126, "engine", "custom",
                                                     "develop"};
  tracer_options.sampling_rules = R"([])";
  tracer_options.sample_rate = std::nan("");
  tracer_options.priority_sampling = false;
  ot::Tracer::InitGlobal(datadog::opentracing::makeTracer(tracer_options));

  ot::StartSpanOptions start_options;
  auto root_span = ot::Tracer::Global()->StartSpanWithOptions("request", start_options);
  auto child_span =
      ot::Tracer::Global()->StartSpan("operation", {opentracing::ChildOf(&root_span->context())});
  child_span->Finish();

  root_span->SetTag("sampling.priority", static_cast<int>(SamplingPriority::UserDrop));
  auto p = static_cast<Span*>(root_span.get())->getSamplingPriority();
  REQUIRE(p);
  REQUIRE(*p == SamplingPriority::UserDrop);

  root_span->Finish();
}

The test section "sampling priority can be set until the root trace finishes" in propagation_test shows that this is possible if there is no default sampling priority set.

I wonder how I can achieve the same with the default RulesSampler.

After debugging a bit it I see the following:
When root span context is retrieved to create a child span, the sampling priority is locked in WritingSpanBuffer::setSamplingPriorityImpl, because sampling with the default PrioritySampler results in a default_sample_rate_ of 1.0 (i.e. SamplerKeep)

@cgilmour
Copy link
Contributor

Hi @tomthomson thanks for reporting this!

It seems that test with the mock sampler is wrong, and I'll need to fix it.

In general, it should be possible to update the sampling priority of a root trace until either

  • the trace is propagated or
  • the trace finishes
    Those events force a sampling decision to be made. However, you're correct - it's happening when the context is accessed, not when it's propagated/"injected".

I'll see what can be done to change it.

@tomthomson tomthomson changed the title [v1.1.4] Not able to drop a trace after root span context is propagated with default RuleSampler [v1.1.4] Not able to drop a trace after root span context is accessed with default RuleSampler May 12, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants