feat: intial draft sending medatada in prometheus remote write

[jmichalek132]

Description:

The initial draft of creating Openmetrics metadata and sending them as part of Prometheus remote write request.

As of now the prometheus remote write is not sending metadata described in the linked issue as it should per per the otel spec. This is draft of a PR to address it.
This is not the final form, among other things it's missing tests, but I opened it to discuss from which part of the codebase should we send the metadata and whether introducing this should be behind a feature flag or configuration option.

• Openmetrics metadata can be sent as part of the same remote write request
  • Than we have to deal with batching (ideally sending metadata only relevant to the batched metrics and maintaining the max batch size)
• We could also send it as as separate request
  • Than it's a question of where in the codebase is a good place to send it from
  • Should we wait for the request to finish before moving onto the next item?

Personally I lean towards just sending it as part of the same request and putting it behind a feature flag given the spec says the metadata should be sent I would eventually set is as default behavior.

TODO:

• Write a design document with these options
  • Sent metadata with every request (prometheus doesn't do this to avoid significantly increase network egress costs)
  • Sent metadata with ever X-th request (risking not sending metadata for every metric)
  • Keep a map in memory of every metric name exported with the last time it was sent and send it every 5 min in a separate request (would increase memory usage)
  • Replicate Prometheus behavior
• Add tests
• Verify the translation to medatata is correct / follows spec
• Check if it generates empty metadata objects
• Find out why certain metrics are still missing the description

Link to tracking Issue: #13849

Testing: Locally only for now, will add tests and do more testing after getting feedback.

Documentation:

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.