Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OTEL_ZIPKIN_ENDPOINT not supported in Zipkin exporter #4926

Closed
giraone opened this issue Nov 8, 2022 · 9 comments · Fixed by #4995
Closed

OTEL_ZIPKIN_ENDPOINT not supported in Zipkin exporter #4926

giraone opened this issue Nov 8, 2022 · 9 comments · Fixed by #4995
Labels
Bug Something isn't working documentation

Comments

@giraone
Copy link

giraone commented Nov 8, 2022

Describe the bug
The documentation states, that one can set the OTEL_ZIPKIN_ENDPOINT environment variable, but

  • it does not seem to work
  • I have not found this name in the code at all

When I set the environment variable, I always get

i.o.exporter.zipkin.ZipkinSpanExporter   : Failed to export spans
org.springframework.web.reactive.function.client.WebClientRequestException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:9411

when the receiving service run on a different host. Also vice versa, when I set OTEL_ZIPKIN_ENDPOINT="http://unknown:9411/api/v2/spans", tracing works.

For me it looks like it uses always the localhost default endpoint.

Steps to reproduce

Use

    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-exporter-zipkin</artifactId>
     <version>1.19.0</version>
    </dependency>

in a Spring Boot 3 Service and set OTEL_ZIPKIN_ENDPOINT.

Then call an endpoint of the app.

Environment
Compiler: openjdk 17.0.4
OS: Ubuntu 20.04

io.opentelemetry:opentelemetry-exporter-zipkin:jar:1.19.0

@giraone giraone added the Bug Something isn't working label Nov 8, 2022
@jkwatson
Copy link
Contributor

jkwatson commented Nov 8, 2022

Unfortunately, the documentation is out of date. The env vars are only supported via the autoconfiguration module. The docs should be updated to reflect that.

@giraone
Copy link
Author

giraone commented Nov 9, 2022

Is there a working example with Spring Boot 3?

Now I have

    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-sdk-extension-autoconfigure</artifactId>
      <version>1.19.0-alpha</version>
    </dependency>
    <dependency>
      <groupId>io.opentelemetry</groupId>
      <artifactId>opentelemetry-exporter-zipkin</artifactId>
    </dependency>

in the pom.xml. If only the first dependency is present, nothing happens. If both are present, the export still want to access localhost:9411.

My docker environment is

   environment:
      - OTEL_TRACES_EXPORTER=zipkin
      - OTEL_EXPORTER_ZIPKIN_ENDPOINT=http://tempo-sb3:9411/api/v2/spans
      - OTEL_PROPAGATORS=b3

@jkwatson
Copy link
Contributor

jkwatson commented Nov 9, 2022

How are you initializing your OpenTelemetry instance? If you want autoconfigure, you'll need to use the autoconfigure classes to initialize it.

@jack-berg
Copy link
Member

FYI, docs on autoconfigure are available here.

@breedx-splk
Copy link
Contributor

@giraone Please provide some additional information about your setup, specifically around how you are initializing your OpenTelemetry instance.

This autoconfigure example might also be of some use to you.

But really, it sounds like you were expecting to just wire up a dependency in maven and have telemetry pop out. That's not really how this works. I know it can be confusing because of "auto" in the name...but "auto" here is referring to configuration. It explicitly does NOT include any auto instrumentation.

If I had to guess, I think you might be better off using the java agent (opentelemetry-java-instrumentation), which includes out-of-the-box support for autoconfiguration and includes all the auto-instrumentation as well.

@giraone
Copy link
Author

giraone commented Nov 12, 2022

In the meanwhile, the setup basically works. Traces are correctly exported via io.micrometer:micrometer-tracing-bridge-otel and io.opentelemetry:opentelemetry-exporter-zipkin from two Spring Boot 3 applications to the Zipkin ingest port of Grafana Tempo. Right now my "setup1" is one without the "Java instrumentation agent". For initialization I am using

OpenTelemetrySdk openTelemetrySdk = AutoConfiguredOpenTelemetrySdk.initialize()
            .getOpenTelemetrySdk();

What is not working is propagation of trace ids from client to server. Within one process the spans are correct, but they are not propagated in client-server REST calls. But I have no clue, whether this is a Spring, Micrometer or OpenTelemetry bug or missconfiguration of my setup.

The complete code is here: https://github.com/giraone/spring-boot3-demo

@jkwatson
Copy link
Contributor

You'll need to make sure you have configured compatible trace propagators on both the client and server for propagation to work correctly.

@giraone
Copy link
Author

giraone commented Nov 14, 2022

Client and Server share the the same code base. They also share the OTEL_PROPAGATORS environment variable. I have used "tracecontext,baggage", "b3", "b3multi" and all 4 together without any success.

@jkwatson
Copy link
Contributor

Have you debugged to see what's happening when the propagators are supposed to be extracting the context to put on the wire and take it off the wire on the server side? Have you examined the requests to see if any tracing headers are present?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working documentation
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants