Update context dynamically from attributes

[jriguera]

Component(s)

No response

Is your feature request related to a problem? Please describe.

We would need to have a way to define tenants from attributes. The headerssetterextension can automatically set tenants (or other headers) from the context, but there is not way to modify/update the context. See #27901 for more details.

Describe the solution you'd like

We are not sure about the solution. Potential solutions are:

• Create a dedicated component which is able to modify the context.
• Extend groupbyattrprocessor (or other processor) to group resources with a new context.
• Extend the OTTL to include a new function.

The problem is the context it needs to be changed is the connection context, "one level up" of the resource context.

Describe alternatives you've considered

Right now we are using a custom processor to overcome this limitation. This processor was not developed to be committed to Otel repos, it started as a PoC and now we are using it internally. If agreed, we are happy to integrated in Otel Contrib.

Additional context

No response

[jriguera]

Just my 2 cents.

I do not think that is possible make a new OTTL function (for the transformprocessor) because it will need to create a new Context and emit (or somehow batch) the telemetry being currently processed. See this piece of code (from here):

        rms := md.ResourceMetrics()
        for i := 0; i < rms.Len() && err == nil; i++ {
                rm := rms.At(i)
                attrs := rm.Resource().Attributes()
                newCtx := ctxt.actionsRunner.Apply(ctx, attrs)
                newMd := pmetric.NewMetrics()
                rm.CopyTo(newMd.ResourceMetrics().AppendEmpty())
                err = ctxt.nextConsumer.ConsumeMetrics(newCtx, newMd)

I think the best way is having a dedicated processor, which makes people aware of its purpose with a clear documentation and use cases (dealing with the Context). Extending the groupbyattrprocessor would be another option, but it will require extra configuration parameters to manage Context.

Time ago I discussed this topic with @jpkrohling

[newly12]

+1 for this need, we faced similar situation where we need to

1. extract attributes and set them in context, which will be honored by downstream otel collector's batch processor
2. copy attributes and set them in context, which will be used by downstream otel collector's batch processor to group data by certain attributes k/vs for other purpose

[atoulme]

Please consider attending a SIG meeting to discuss this.

[LilWatson]

+1 I'm having the same problem currently.

Is there anything against integrating https://github.com/springernature/o11y-otel-contextprocessor/tree/main/otelcol-dev/contextprocessor to contrib?

[jriguera]

I have updated the example with a similar configuration we are using in production as a gateway role.

You can always create your own distribution of the otelcol binary with this processor.