Scrapers: separate resource attributes in metadata.yaml

[dmitryax]

Problem

Currently attributes fields in metadata.yaml sometimes are used for both metric and resource attributes. Some scrapers don't specify resource attributes in metadata.yaml, but use resource attribute "hardcoded" in scraper code instead. We need to establish an approach and probably separate the resource attributes from other metric attributes.

Proposal

We can introduce another field in metadata.yaml to specify resource attributes. mdatagen should generate them into a new struct variable ResourceAttributes with alias RA keeping existing Attributes with A alias for metric attributes.

The field can be added to the metadata.yaml in the following format:

resource_attributes:
  <attribute_name>:
    # Optional description
    description: ...
    # Actual attribute string, if not provided `<attribute_name>` used instead, optional.
    value: state
    # Use a variable from `semconv` package instead of defining new attribute name,
    # e.g. "v1.8.0/AttributeCloudAccountID", optional.
    semconv: 

Besides generating ResourceAttributes struct that should be used in scraper code, the updated the new resource_attributes field should be used to generate additional documentation for the scraper. Potentially, if resource_attributes::semconv is set in metadata.yaml, we can also parse the semconv docstrings and auto-generate metric documentation from auto-generated semconv code :)

Action items:

• Introduce resource attributes in metadata.yaml and generate helpers from it
• [Metrics builder] Add sem conv support in metadata.yaml #8840
• [Metrics builder] Define resource schema in metadata.yaml and automatically set it in InstrumentationLibrary #8841
• [metrics builder] Add ability to enable/disable resource attributes #16373
• [cmd/mdatagen] Add warnings in resource_attributes section #19174

[dmitryax]

@djaglowski @bogdandrutu please let me know what do you think

[djaglowski]

This all makes sense to me.

One nit - I would support description being required unless semconv is specified.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• cmd/mdatagen: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• cmd/mdatagen: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• cmd/mdatagen: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• cmd/mdatagen: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[djaglowski]

The remaining checklist items (#8840, #19174) seem to be related but it's not clear to me how this issue is dependent on them. Should we close this issue?

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• cmd/mdatagen: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dmitryax]

The remaining checklist items (#8840, #19174) seem to be related but it's not clear to me how this issue is dependent on them. Should we close this issue?

Yes, they are just improvements. We can close this issue