Add key_field option to tag_cardinality_limit

[nzxwang]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request

• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

I would like to use the tag_cardinality_limit component to limit the cardinality of tags on metric events to protect against accidental high cardinality usage. However, because my observability pipeline services a large number of customers, I would like to bucket events into separate limiters based on some key_field parameter before limiting the events to avoid the noisy neighbor problem.

Currently, without some key_field parameter, we would have to first route events to different routes based on some key_field and create one tag_cardinality_limit transform for each route. This would incur 2 VRL components per key_field that we want to support which would greatly increase vector's start-up time and is not feasible in our auto-scaling deployment environments.

Proposal

Implement some key_field parameter like that of throttle's.

This request is similar to #15743

Version

0.39.0