Reasoning for cardinality limit

[akats7]

Hi all,

I wanted to inquire about the preset cardinality limit of 2000. I'm curious why this is a hard cap and set in the sdk instead of simply being enforced by the monitoring system.

Is there a reason its not configurable and set to this number specifically?

In our use case we're dealing with kafka metrics where topic is a critical attribute and is high cardinality by nature so we've been reaching the cardinality limit on certain nodes where the number of topics is higher. Removing topic as an attribute is not really an option for us.

Wanted to bring this up and get the thoughts of the team, thanks!

[jkwatson]

We are using the recommended default from the specification here: https://github.com/open-telemetry/opentelemetry-specification/blob/31e988297d9c3f1f89c3915aec13df4be1348d94/specification/metrics/sdk.md?plain=1#L719

[akats7]

We are using the recommended default from the specification here: https://github.com/open-telemetry/opentelemetry-specification/blob/31e988297d9c3f1f89c3915aec13df4be1348d94/specification/metrics/sdk.md?plain=1#L719

@jkwatson Got it, so I understand making that the default but would it be possible to make that a parameter that can be overridden, I imagine our use case is not exclusive to us and for us to lower the cardinality we would have to break other best practices with semantic conventions. And even then we would still have to sacrifice attributes and omit entire metrics that are critical

[jack-berg]

It will be configurable in the 1.28.0 release, which will be published tomorrow. We originally added it to protect against bad instrumentation producing high cardinality and causing apps to crash. When the java metrics SDK went stable, there wasn't any advice in the specification about protecting against high memory usage so we took a conservative approach.

[akats7]

@jack-berg That's great to hear, thank you for providing that context.

[jack-berg]

Oops I said something incorrect. Configurability of cardinality limits was added in #5494, which was first published in 1.27.0. 1.28.0 added #5560, which captures measurements above the cardinality limit in an overflow series.

Closing this issue.

[akats7]

@jack-berg Would it be reasonable to be able to update the global max_cardinality through a config prop to avoid setting it per view.