diff --git a/cmd/mdatagen/README.md b/cmd/mdatagen/README.md index 49f6bd39b61d..0ca79053de8d 100644 --- a/cmd/mdatagen/README.md +++ b/cmd/mdatagen/README.md @@ -1,42 +1,56 @@ # Metadata Generator -Components must contain a `metadata.yaml` file that documents various aspects of the component including: +Every component's documentation should include a brief description of the component and guidance on how to use it. +There is also some information about the component (or metadata) that should be included to help end-users understand the current state of the component and whether it is right for their use case. +Examples of this metadata about a component are: * its stability level * the distributions containing it * the types of pipelines it supports * metrics emitted in the case of a scraping receiver -Current examples: +The metadata generator defines a schema for specifying this information to ensure it is complete and well-formed. +The metadata generator is then able to ingest the metadata, validate it against the schema and produce documentation in a standardized format. +An example of how this generated documentation looks can be found in [documentation.md](./documentation.md). -* hostmetricsreceiver scrapers like the [cpuscraper](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/metadata.yaml) +## Using the Metadata Generator -See [metric-metadata.yaml](metric-metadata.yaml) for file format documentation. +In order for a component to benefit from the metadata generator (`mdatagen`) these requirements need to be met: +1. A `metadata.yaml` file containing the metadata needs to be included in the component +2. The component should declare a `go:generate mdatagen` directive which tells `mdatagen` what to generate -If adding a new receiver a `doc.go` file should also be added to trigger the generation. See below for details. +As an example, here is a minimal `metadata.yaml` for the [OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver): +```yaml +type: otlp +status: + class: receiver + stability: + beta: [logs] + stable: [metrics, traces] +``` -## Generating +Detailed information about the schema of `metadata.yaml` can be found in [metadata-schema.yaml](./metadata-schema.yaml). -`make generate` triggers the following actions: +The `go:generate mdatagen` directive is usually defined in a `doc.go` file in the same package as the component, for example: +```go +//go:generate mdatagen metadata.yaml -1. `cd cmd/mdatagen && $(GOCMD) install .` to install `mdatagen` tool in`GOBIN` +package main +``` -2. All `go:generate mdatagen` directives that are usually defined in `doc.go` files of the metric scrapers will be - executed. For example, - [/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/doc.go](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/doc.go) - runs `mdatagen` for the [metadata.yaml](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/metadata.yaml) - which generates the metrics builder code in - [internal/metadata](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/internal/metadata) - and [documentation.md](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/internal/metadata) with - generated documentation about emitted metrics. +Below are some more examples that can be used for reference: -## Development +* The ElasticSearch receiver has an extensive [metadata.yaml](../../receiver/elasticsearchreceiver/metadata.yaml) +* The host metrics receiver has internal subcomponents, each with their own `metadata.yaml` and `doc.go`. See [cpuscraper](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper) for example. -In order to introduce support of a new functionality in metadata.yaml: +You can run `cd cmd/mdatagen && $(GOCMD) install .` to install the `mdatagen` tool in `GOBIN` and then run `mdatagen metadata.yaml` to generate documentation for a specific component or you can run `make generate` to generate documentation for all components. -1. Make code changes in the (generating code)[./loader.test] and (templates)[./templates/]. -2. Add usage of the new functionality in (metadata.yaml)[./metadata.yaml]. -3. Run `make mdatagen-test`. -4. Make sure all tests are passing including (generated tests)[./internal/metadata/generated_metrics_test.go]. -5. Run `make generate`. +## Contributing to the Metadata Generator +The code for generating the documentation can be found in [loader.go](./loader.go) and the templates for rendering the documentation can be found in [templates](./templates). +When making updates to the metadata generator or introducing support for new functionality: + +1. Ensure the [metadata-schema.yaml](./metadata-schema.yaml) and [./metadata.yaml](metadata.yaml) files reflect the changes. +2. Run `make mdatagen-test`. +3. Make sure all tests are passing including [generated tests](./internal/metadata/generated_metrics_test.go). +4. Run `make generate`. diff --git a/cmd/mdatagen/documentation.md b/cmd/mdatagen/documentation.md index 358512be86f2..4c649fea8320 100644 --- a/cmd/mdatagen/documentation.md +++ b/cmd/mdatagen/documentation.md @@ -1,6 +1,6 @@ [comment]: <> (Code generated by mdatagen. DO NOT EDIT.) -# test +# file ## Default Metrics diff --git a/cmd/mdatagen/internal/metadata/generated_config.go b/cmd/mdatagen/internal/metadata/generated_config.go index ff93bcb4ce55..973c02069f4a 100644 --- a/cmd/mdatagen/internal/metadata/generated_config.go +++ b/cmd/mdatagen/internal/metadata/generated_config.go @@ -23,7 +23,7 @@ func (ms *MetricConfig) Unmarshal(parser *confmap.Conf) error { return nil } -// MetricsConfig provides config for test metrics. +// MetricsConfig provides config for file metrics. type MetricsConfig struct { DefaultMetric MetricConfig `mapstructure:"default.metric"` DefaultMetricToBeRemoved MetricConfig `mapstructure:"default.metric.to_be_removed"` @@ -49,7 +49,7 @@ type ResourceAttributeConfig struct { Enabled bool `mapstructure:"enabled"` } -// ResourceAttributesConfig provides config for test resource attributes. +// ResourceAttributesConfig provides config for file resource attributes. type ResourceAttributesConfig struct { MapResourceAttr ResourceAttributeConfig `mapstructure:"map.resource.attr"` OptionalResourceAttr ResourceAttributeConfig `mapstructure:"optional.resource.attr"` @@ -78,7 +78,7 @@ func DefaultResourceAttributesConfig() ResourceAttributesConfig { } } -// MetricsBuilderConfig is a configuration for test metrics builder. +// MetricsBuilderConfig is a configuration for file metrics builder. type MetricsBuilderConfig struct { Metrics MetricsConfig `mapstructure:"metrics"` ResourceAttributes ResourceAttributesConfig `mapstructure:"resource_attributes"` diff --git a/cmd/mdatagen/internal/metadata/generated_status.go b/cmd/mdatagen/internal/metadata/generated_status.go index 4d81725d0f92..e9ab937c5055 100644 --- a/cmd/mdatagen/internal/metadata/generated_status.go +++ b/cmd/mdatagen/internal/metadata/generated_status.go @@ -7,6 +7,8 @@ import ( ) const ( - Type = "test" - MetricsStability = component.StabilityLevelDevelopment + Type = "file" + TracesStability = component.StabilityLevelBeta + LogsStability = component.StabilityLevelDevelopment + MetricsStability = component.StabilityLevelStable ) diff --git a/cmd/mdatagen/loader_test.go b/cmd/mdatagen/loader_test.go index 247259bf7b60..8fca2f8e432e 100644 --- a/cmd/mdatagen/loader_test.go +++ b/cmd/mdatagen/loader_test.go @@ -20,8 +20,18 @@ func Test_loadMetadata(t *testing.T) { { name: "metadata.yaml", want: metadata{ - Type: "test", + Type: "file", SemConvVersion: "1.9.0", + Status: &Status{ + Class: "receiver", + Stability: map[string][]string{ + "development": {"logs"}, + "beta": {"traces"}, + "stable": {"metrics"}, + }, + Distributions: []string{"contrib"}, + Warnings: []string{"Any additional information that should be brought to the consumer's attention"}, + }, ResourceAttributes: map[attributeName]attribute{ "string.resource.attr": { Description: "Resource attribute with any string value.", @@ -146,12 +156,6 @@ func Test_loadMetadata(t *testing.T) { }, }, ScopeName: "otelcol", - Status: &Status{ - Class: "receiver", - Stability: map[string][]string{ - "development": {"metrics"}, - }, - }, }, }, { diff --git a/cmd/mdatagen/metric-metadata.yaml b/cmd/mdatagen/metadata-schema.yaml similarity index 62% rename from cmd/mdatagen/metric-metadata.yaml rename to cmd/mdatagen/metadata-schema.yaml index 08f2f62ffe2c..b3c5444ab6ad 100644 --- a/cmd/mdatagen/metric-metadata.yaml +++ b/cmd/mdatagen/metadata-schema.yaml @@ -1,6 +1,23 @@ -# Required: type of the receiver. +# Required: The type of the component - Usually the name. The type and class combined uniquely identify the component (eg. receiver/otlp) or subcomponent (eg. receiver/hostmetricsreceiver/cpu) type: +# Required for components (Optional for subcomponents): A high-level view of the development status and use of this component +status: + # Required: The class of the component (For example receiver) + class: + # Required: The stability of the component - See https://github.com/open-telemetry/opentelemetry-collector#stability-levels + stability: + development: [] + alpha: [] + beta: [] + stable: [] + deprecated: [] + unmaintained: [] + # Optional: The distributions that this component is bundled with (For example core or contrib). See statusdata.go for a list of common distros. + distributions: [string] + # Optional: A list of warnings that should be brought to the attention of users looking to use this component + warnings: [string] + # Optional: OTel Semantic Conventions version that will be associated with the scraped metrics. # This attribute should be set for metrics compliant with OTel Semantic Conventions. sem_conv_version: 1.9.0 @@ -13,7 +30,7 @@ resource_attributes: # Required: description of the attribute. description: # Optional: array of attribute values if they are static values (currently, only string type is supported). - enum: + enum: [string] # Required: attribute value type. type: @@ -27,16 +44,16 @@ attributes: # Required: description of the attribute. description: # Optional: array of attribute values if they are static values (currently, only string type is supported). - enum: + enum: [string] # Required: attribute value type. type: -# Required: map of metric names with the key being the metric name and value +# Optional: map of metric names with the key being the metric name and value # being described below. metrics: : # Required: whether the metric is collected by default. - enabled: # true | false + enabled: bool # Required: metric description. description: # Optional: extended documentation of the metric. @@ -57,11 +74,11 @@ metrics: # Required: metric type with its settings. : # Required for sum and gauge metrics: type of number data point values. - value_type: # int | double + value_type: # Required for sum metric: whether the metric is monotonic (no negative delta values). - monotonic: # true | false + monotonic: bool # Required for sum metric: whether reported values incorporate previous measurements # (cumulative) or not (delta). - aggregation: # delta | cumulative + aggregation: # Optional: array of attributes that were defined in the attributes section that are emitted by this metric. - attributes: + attributes: [string] diff --git a/cmd/mdatagen/metadata.yaml b/cmd/mdatagen/metadata.yaml index d4dfe7cd3461..054ca3824143 100644 --- a/cmd/mdatagen/metadata.yaml +++ b/cmd/mdatagen/metadata.yaml @@ -1,13 +1,18 @@ # Sample metric metadata file with all available configurations. -type: test +type: file sem_conv_version: 1.9.0 status: class: receiver stability: - development: [metrics] + development: [logs] + beta: [traces] + stable: [metrics] + distributions: [contrib] + warnings: + - Any additional information that should be brought to the consumer's attention resource_attributes: string.resource.attr: