Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AddJaegerExporter makes services crash on startup run if host is unknown. #4210

Closed
phamtri2996 opened this issue Feb 21, 2023 · 1 comment
Closed
Labels
bug Something isn't working

Comments

@phamtri2996
Copy link

Bug Report

opentelemetry 1.4.0-rc.3
opentelemetry.exporter.jaeger 1.3.2
opentelemetry.extensions.hosting 1.4.0-rc.3
opentelemetry.instrumentation.aspnetcore 1.0.0-rc9.12
opentelemetry.instrumentation.http 1.0.0-rc9.12

Runtime version: netcoreapp3.1

Symptom

public void Connect() => this.client.Connect(this.host, this.port);

On runtime, if jaeger hostname was unmapped, this method in constructor make the service crash.

What is the expected behavior?

JaegerExporter ignore whether the host name was mapped or not, allow the service to start up and handle requests.

Reproduce

Here's the simplified version of our OpenTelemetry implementation:

services.AddOpenTelemetry()
.WithTracing(
    tracerBuilder => 
    {
        tracerBuilder.Build(Configuration);
    })
.StartWithHost();
public static class OpenTelemetryExtension
{
    /// <summary>
    /// 
    /// </summary>
    /// <param name="builder"></param>
    public static void Build(this TracerProviderBuilder builder, IConfiguration Configuration)
    {
        builder
        .SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(
            Configuration.GetValue<string>("ServiceName")
            , serviceVersion:  Configuration.GetValue<string>("ServiceVersion")))
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddSqlClientInstrumentation();

        switch (Configuration.GetValue<string>("Exporter").ToLower())
        {
            case "jaeger":
                builder.AddJaegerExporter(o =>
                 {
                     o.AgentHost = Configuration.GetValue<string>("AgentHost");
                     o.AgentPort = Configuration.GetValue<int>("AgentPort");
                 });
                break;
            default:
                builder.AddConsoleExporter();
                break;
        }
    }
}

This is a library project that shared across our services.
The problem here is:

  • Not all of our services need monitoring.
  • Our system architecture is managed by another team
    I can manually disable OpenTelemetry in settings for services that need monitoring but if the host mapping is down due to system maintenance or something else or when k8s build more pods, we can't start our services unless we manually disable it in settings.

We will close this issue if:

  • The repro project you share with us is complex. We can't investigate custom
    projects, so don't point us to such, please.
  • If we can not reproduce the behavior you're reporting.

Additional Context

Add any other context about the problem here.

@phamtri2996 phamtri2996 added the bug Something isn't working label Feb 21, 2023
@alanwest
Copy link
Member

alanwest commented Feb 21, 2023

This was a recent change from #4006. We now fail on startup if a component cannot be initialized. Previously, this error was swallowed when using the OpenTelemetry.Extensions.Hosting package. Using the regular SDK.CreateTracerProviderBuilder() always threw on startup in this case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants