[OTEL 0.22] shutdown SdkMeterProvider when built from opentelemetry-otlp?

[Oliboy50]

Context

We built a simple lib in my company which abstract away most of the OpenTelemetry complexity (using tracing-opentelemetry internally) so my colleagues and I just have to use our simple lib and tracing in all our company projects.

Question

My question is about how to properly shutdown (if even needed?), the SdkMeterProvider used internally in this lib.

Because the SdkMeterProvider is built using the opentelemetry-otlp crate, and it disappears immediately in our lib internal API, such as:

pub fn build_metrics_layer() -> tracing_opentelemetry::MetricsLayer<S>
    where
        S: tracing::Subscriber + for<'span> tracing_subscriber::registry::LookupSpan<'span>,
{
    let meter_provider = opentelemetry_otlp::new_pipeline()
        .metrics(opentelemetry_sdk::runtime::Tokio)
        .with_exporter(some_exporter())
        .build()
        .expect("Failed to build OTLP meter provider");

    MetricsLayer::new(meter_provider)

    // meter_provider is gone here, 
    // so we cannot call `meter_provider.shutdown()` when closing the application :'(
}

Then in our company projects, we use tracing such as:

let registry = tracing_subscriber::registry()
    .with(tracing_subscriber::filter::EnvFilter::from_default_env())
    .with(build_metrics_layer());
tracing::subscriber::set_global_default(registry).expect("Failed to set global tracing subscribers");

// do our stuff, such as run an HTTP API server
// ...

// here we want to shutdown the HTTP API server
// but we can't since there is no more `opentelemetry::global::shutdown_meter_provider()` :'(

So how are we supposed to shutdown the SdkMeterProvider used internally, without using opentelemetry::global::shutdown_meter_provider()? 🙏

I tried to see if opentelemetry-otlp or tracing-opentelemetry was doing the shutdown stuff for us, but I wasn't able to find something there :'(

[Oliboy50]

Currently, I found a workaround which is to keep my own static instance of the SdkMeterProvider so that I can .shutdown it later.

use std::sync::OnceLock;
use opentelemetry_sdk::metrics::SdkMeterProvider;

static METER_PROVIDER: OnceLock<SdkMeterProvider> = OnceLock::new();

pub fn build_metrics_layer() -> tracing_opentelemetry::MetricsLayer<S>
    where
        S: tracing::Subscriber + for<'span> tracing_subscriber::registry::LookupSpan<'span>,
{
    let meter_provider = opentelemetry_otlp::new_pipeline()
        .metrics(opentelemetry_sdk::runtime::Tokio)
        .with_exporter(some_exporter())
        .build()
        .expect("Failed to build OTLP meter provider");

    METER_PROVIDER.set(meter_provider.clone()).expect(
        "build_metrics_layer() is not meant to be used more than once in the same process",
    );

    MetricsLayer::new(meter_provider)
}

pub fn shutdown_meter_provider() {
    if let Some(meter_provider) = METER_PROVIDER.get() {
        meter_provider
            .shutdown()
            .expect("Failed to shutdown meter provider");
    }
}

Then in our company projects, we should be able to do:

let registry = tracing_subscriber::registry()
    .with(tracing_subscriber::filter::EnvFilter::from_default_env())
    .with(build_metrics_layer());
tracing::subscriber::set_global_default(registry).expect("Failed to set global tracing subscribers");

// do our stuff, such as run an HTTP API server
// ...

shutdown_meter_provider();

Do you think this is a valid replacement? Is it ugly? Is it useless?

[lalitb]

I believe OTLP Metrics pipeline creation will also set the global meter provider. So in your application, you should be able to simply do something like:

 global::meter_provider().shutdown()

[Oliboy50]

This was the first thing I tried, unfortunately, global::meter_provider() returns an instance of GlobalMeterProvider (instead of SdkMeterProvider), which does not expose the .shutdown() method.

[lalitb]

Oh yeah, my bad. The shutdown() method is for the application, not the instrumented library, so it won't be exposed by GlobalMeterProvider.
I think your workaround should work (though good to validate it), but the solution should be for the MetricsLayer layer to take either instance of Meter, or the non-mutable reference to MeterProvider as an argument, as done for traces and logs. This will allow the MeterProvider instance to be used by the application for shutdown and flush calls.

[TommyCpp]

FYI we have removed the global shutdown method for meter provider in #1743

The reason being after discussion we still don't think API should provide a shutdown method. We ask application owner to own the lifecycle of the meter provider, as demostrated in this example.

I think this concluded this discussion but feel free to reopen it if you have additional questions