multi-container-pods can cause problems with useLabelsForResourceAttributes

[zeitlinger]

Component(s)

auto-instrumentation

Related slack

https://cloud-native.slack.com/archives/C041APFBYQP/p1732635742710849

Is your feature request related to a problem? Please describe.

If you have a multi-container pod that use OTel in combination with useLabelsForResourceAttributes, then all of those applications (on all containers) end up sharing the same service.instance.id and service.name.

Why is this a problem for service.instance.id?

The value must be unique (https://opentelemetry.io/docs/specs/semconv/attributes-registry/service/) - but all apps (in all containers) would get the same value.

Describe the solution you'd like

Given that the default (namespace name + pod name + container name) is good as it is, we could remove the logic for service.instance.id.

Is this also a problem for service.name?

It's not great, but you still have service.instance.id to differentiate.

Is this a breaking change?

Yes - but I think we should accept this and prevent uses from accidental mistakes that can be hard to understand.

[diurnalist]

Do you have an example of the user pain/confusion that can arise from the way it works now? In our environment, we made the choice where all containers w/in a pod logically constitute a service so it's good that they are consistently tagged w/ one service.instance.id (which is usually the pod name/uid).

I can understand how that might be a problem if for some reason team A is on call for container A and team B for container B, or other interesting organization topologies :)

[zeitlinger]

Do you have an example of the user pain/confusion

You can have multiple containers that both report memory or CPU usage with the same metric name - this is the motivation for the semantic convention.

Adhering to semantic conventions is almost always a good idea - because other tools are, or will be, based on them.

[diurnalist]

We have used k8s.container.name as the differentiating attribute for that use-case. It's a different container, same service instance ID.