Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Errors when x-datadog-sampling-priority header is present but x-datadog-tags is not #241

Closed
hiddewie opened this issue Sep 6, 2022 · 5 comments

Comments

@hiddewie
Copy link

hiddewie commented Sep 6, 2022

Nginx version: 1.23.1
Datadog opentracing version: 1.3.4 (issue is also present in 1.3.3, but not in 1.3.2)
Nginx opentracing version: 0.26.0

Nginx server config

server {
  listen 9000 default_server;

  opentracing on;
  opentracing_trace_locations off;
  opentracing_tag http_user_agent $http_user_agent;
  opentracing_operation_name nginx.request;
  opentracing_propagate_context;

  location / {
    # proxy ...
  }
}

When the request contains the headers

x-datadog-parent-id: 628590525536830299
x-datadog-sampling-priority: 0
x-datadog-trace-id: 8761565436669190370

the following error is emitted

2022/09/06 08:33:46 [error] 102#102: *17 no opentracing context value found for span context key x_datadog_tags for request 00007FF9B25D2450, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", host: "localhost:9000"

If I add the header

x-datadog-tags: a=b

the nginx error is not emitted anymore.

I suspect the pull request #230

@sanglt
Copy link

sanglt commented Sep 7, 2022

Yes we have the same issue and need to rollback to v1.3.2

@dgoffredo
Copy link
Contributor

Thanks for reporting this.

I probably need a branch like this for x-datadog-tags:

} else if (nginx_opentracing_compatibility_hack_) {
// See the comment in the header file on nginx_opentracing_compatibility_hack_.
result = writer.Set(headers_impl.sampling_priority_header, "1");
if (!result) {
return result;
}
}
. nginx-opentracing needs to know in advance which HTTP headers we might add to the upstream request. It does this by pretending to serialize some trace context, and then inspects the headers written. The tracer is expected to detect this pretend serialization and write all tags that it ever might. The issue probably is that I didn't add x-datadog-tags there.

Our nginx integration test didn't catch this. I'll play with it and propose a fix.

@dgoffredo
Copy link
Contributor

#242

@hiddewie
Copy link
Author

hiddewie commented Sep 9, 2022

Thanks for looking into this quickly!

@dgoffredo
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants