Using OpenTelemetry Sdk with Azure Functions

[utpilla]

For using OpenTelemetry Sdk with Azure Functions, AddOpenTelemetryTracing extension method cannot be used. This is because Azure Functions has restrictions on running background hosted services. The right way to configure it for Functions is below:

var openTelemetry = Sdk.CreateTracerProviderBuilder().
                AddSource("MyActivitySourceName")
                .SetSampler(new AlwaysOnSampler())
                .AddConsoleExporter()
                .Build();
            builder.Services.AddSingleton(openTelemetry);

There is also the issue (Azure/azure-functions-host#7135) of Azure Functions not supporting DiagnosticListener callbacks when using System.Diagnostics.DiagnosticSource (> 4.7.0). Therefore, OpenTelemetry instrumentations (like HttpClient, SqlClient etc.) will not work in Azure Functions as the instrumentations use System.Diagnostics.DiagnosticSource (= 5.0.1)

[oising]

Now that we have out of process net5 support, will this start working (since s.d.ds is part of net5?) -- @cijothomas

[cijothomas]

Now that we have out of process net5 support, will this start working (since s.d.ds is part of net5?) -- @cijothomas

I am not sure if I understood your question. Could you elaborate?

[oising]

Now that we have out of process net5 support, will this start working (since s.d.ds is part of net5?) -- @cijothomas

I am not sure if I understood your question. Could you elaborate?

I guess it's more about this: Azure/azure-functions-host#7135 -- but since you're here :)

The functions v2 runtime is in-process, and as such, we're stuck with using system.diagnostics.diagnosticsource 4.7, ergo OTEL instrumentation libraries for HttpClient, SqlClient etc do not work. Now that net 5.0 is out of process, and we provide the runtime ourselves (right?) does this mean that we can use system.diagnostics.diagnosticsource 5.x and OTEL will work correctly in a functions v3 context?

[cijothomas]

I cannot comment on whether Functions V3 (or V2) addresses this. The owners of Azure Functions has to make that clarification. I'll still check with Functions team and post back in this thread if I hear anything.

[josh-endries]

Just a note that using OpenTelemetry.Trace.TracerProviderBuilderExtensions.AddHttpClientInstrumentation with a .NET 5 isolated runtime Azure Function does seem to work, at least somewhat. If I set the trace ID in an ActivityContext and start my own Activity, then call HttpClient.GetAsync within the activity scope, that outgoing request contains the traceparent header that contains the trace ID.

[valdisiljuconoks]

Seems to be working. I'm using the following setup in functions (v4 runtime + in-process):

[assembly: FunctionsStartup(typeof(Startup))]

namespace FunctionApp1;

internal class Startup : FunctionsStartup
{
    public override void Configure(IFunctionsHostBuilder builder)
    {
        builder.Services.AddOpenTelemetryTracing(b =>
        {
            b.AddHttpClientInstrumentation();
            b.AddSource("AzureFunctionsOpenTelemetry");
            b.AddOtlpExporter(options => options.Endpoint = new Uri("http://localhost:4317"));
        });

        builder.AddOpenTelemetry(b =>
        {
            b.IncludeFormattedMessage = true;
            b.IncludeScopes = true;
            b.ParseStateValues = true;
            b.AddOtlpExporter(options => options.Endpoint = new Uri("http://localhost:4317"));
        });
    }
}

public static class OpenTelemetryLoggingExtensions
{
    public static IFunctionsHostBuilder AddOpenTelemetry(
        this IFunctionsHostBuilder builder,
        Action<OpenTelemetryLoggerOptions> configure = null)
    {
        if (builder == null)
        {
            throw new ArgumentNullException(nameof(builder));
        }

        builder.Services.TryAddEnumerable(ServiceDescriptor.Singleton<ILoggerProvider, OpenTelemetryLoggerProvider>());

        if (configure != null)
        {
            builder.Services.Configure(configure);
        }

        return builder;
    }
}

[smmaster]

can someone please provide a full working example of an azure function ?

[valdisiljuconoks]

can someone please provide a full working example of an azure function ?

This is repo with my experiments so far. Tried to model some "distributed" system in field of mine: e-commerce. https://github.com/valdisiljuconoks/otel-experiment/tree/master/application/Commerce.Batch

[smmaster]

can someone please provide a full working example of an azure function ?

This is repo with my experiments so far. Tried to model some "distributed" system in field of mine: e-commerce. https://github.com/valdisiljuconoks/otel-experiment/tree/master/application/Commerce.Batch

Can't access the repo, is it public?

[cijothomas]

https://github.com/cijothomas/FunctionsOpenTelemetry/blob/master/FunctionsOpenTelemetry/Startup.cs - Note that this is not approved or reviewed by Functions experts, so not sure if any Functions specific things are required. It does work in my testing.

[agr]

Is it supposed to work locally with function runner from Azure function core tools? Mine fails unable to find System.Diagnostics.DiagnosticSource assembly whenever the code touches any OpenTelemetry object.
Disregard. I'm having some build issues that prevent the DLL to end up in proper location.