From 137ab317fb47de0bb3c2d02105a1b380b9f89ae5 Mon Sep 17 00:00:00 2001 From: Dewald de Jager Date: Fri, 2 Jun 2023 12:35:48 +0200 Subject: [PATCH] Document how to use mdatagen and the schema of metadata.yaml --- cmd/mdatagen/README.md | 60 +++++++++++++++++++------------ cmd/mdatagen/metadata-schema.yaml | 30 +++++++++------- 2 files changed, 55 insertions(+), 35 deletions(-) diff --git a/cmd/mdatagen/README.md b/cmd/mdatagen/README.md index 635e8c49b856..0ca79053de8d 100644 --- a/cmd/mdatagen/README.md +++ b/cmd/mdatagen/README.md @@ -1,42 +1,56 @@ # Metadata Generator -Components must contain a `metadata.yaml` file that documents various aspects of the component including: +Every component's documentation should include a brief description of the component and guidance on how to use it. +There is also some information about the component (or metadata) that should be included to help end-users understand the current state of the component and whether it is right for their use case. +Examples of this metadata about a component are: * its stability level * the distributions containing it * the types of pipelines it supports * metrics emitted in the case of a scraping receiver -Current examples: +The metadata generator defines a schema for specifying this information to ensure it is complete and well-formed. +The metadata generator is then able to ingest the metadata, validate it against the schema and produce documentation in a standardized format. +An example of how this generated documentation looks can be found in [documentation.md](./documentation.md). -* hostmetricsreceiver scrapers like the [cpuscraper](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/metadata.yaml) +## Using the Metadata Generator -See [metadata-schema.yaml](metadata-schema.yaml) for file format documentation. +In order for a component to benefit from the metadata generator (`mdatagen`) these requirements need to be met: +1. A `metadata.yaml` file containing the metadata needs to be included in the component +2. The component should declare a `go:generate mdatagen` directive which tells `mdatagen` what to generate -If adding a new receiver a `doc.go` file should also be added to trigger the generation. See below for details. +As an example, here is a minimal `metadata.yaml` for the [OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver): +```yaml +type: otlp +status: + class: receiver + stability: + beta: [logs] + stable: [metrics, traces] +``` -## Generating +Detailed information about the schema of `metadata.yaml` can be found in [metadata-schema.yaml](./metadata-schema.yaml). -`make generate` triggers the following actions: +The `go:generate mdatagen` directive is usually defined in a `doc.go` file in the same package as the component, for example: +```go +//go:generate mdatagen metadata.yaml -1. `cd cmd/mdatagen && $(GOCMD) install .` to install `mdatagen` tool in`GOBIN` +package main +``` -2. All `go:generate mdatagen` directives that are usually defined in `doc.go` files of the metric scrapers will be - executed. For example, - [/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/doc.go](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/doc.go) - runs `mdatagen` for the [metadata.yaml](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/metadata.yaml) - which generates the metrics builder code in - [internal/metadata](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/internal/metadata) - and [documentation.md](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/internal/metadata) with - generated documentation about emitted metrics. +Below are some more examples that can be used for reference: -## Development +* The ElasticSearch receiver has an extensive [metadata.yaml](../../receiver/elasticsearchreceiver/metadata.yaml) +* The host metrics receiver has internal subcomponents, each with their own `metadata.yaml` and `doc.go`. See [cpuscraper](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper) for example. -In order to introduce support of a new functionality in metadata.yaml: +You can run `cd cmd/mdatagen && $(GOCMD) install .` to install the `mdatagen` tool in `GOBIN` and then run `mdatagen metadata.yaml` to generate documentation for a specific component or you can run `make generate` to generate documentation for all components. -1. Make code changes in the (generating code)[./loader.test] and (templates)[./templates/]. -2. Add usage of the new functionality in (metadata.yaml)[./metadata.yaml]. -3. Run `make mdatagen-test`. -4. Make sure all tests are passing including (generated tests)[./internal/metadata/generated_metrics_test.go]. -5. Run `make generate`. +## Contributing to the Metadata Generator +The code for generating the documentation can be found in [loader.go](./loader.go) and the templates for rendering the documentation can be found in [templates](./templates). +When making updates to the metadata generator or introducing support for new functionality: + +1. Ensure the [metadata-schema.yaml](./metadata-schema.yaml) and [./metadata.yaml](metadata.yaml) files reflect the changes. +2. Run `make mdatagen-test`. +3. Make sure all tests are passing including [generated tests](./internal/metadata/generated_metrics_test.go). +4. Run `make generate`. diff --git a/cmd/mdatagen/metadata-schema.yaml b/cmd/mdatagen/metadata-schema.yaml index f6f2a7e6abd2..1c7fb28553b2 100644 --- a/cmd/mdatagen/metadata-schema.yaml +++ b/cmd/mdatagen/metadata-schema.yaml @@ -1,15 +1,21 @@ -# Required: The type of the component. +# Required: The type of the component - Usually the name. The type and class combined uniquely identify the component (eg. receiver/otlp) or subcomponent (eg. receiver/hostmetricsreceiver/cpu) type: -# Required: A high-level view of the development status and use of this component +# Required for components (Optional for subcomponents): A high-level view of the development status and use of this component status: # Required: The class of the component (For example receiver) class: # Required: The stability of the component - See https://github.com/open-telemetry/opentelemetry-collector#stability-levels - stability: - # Optional: The distributions that this component is bundled with (For example core or contrib) + stability: + development: [] + alpha: [] + beta: [] + stable: [] + deprecated: [] + unmaintained: [] + # Optional: The distributions that this component is bundled with (For example core or contrib). See statusdata.go for a list of common distros. distributions: []string - # Optional: A list of warnings that should be brought to the attention of developers looking to use this component + # Optional: A list of warnings that should be brought to the attention of users looking to use this component warnings: []string # Optional: OTel Semantic Conventions version that will be associated with the scraped metrics. @@ -24,7 +30,7 @@ resource_attributes: # Required: description of the attribute. description: # Optional: array of attribute values if they are static values (currently, only string type is supported). - enum: + enum: []string # Required: attribute value type. type: @@ -38,7 +44,7 @@ attributes: # Required: description of the attribute. description: # Optional: array of attribute values if they are static values (currently, only string type is supported). - enum: + enum: []string # Required: attribute value type. type: @@ -47,7 +53,7 @@ attributes: metrics: : # Required: whether the metric is collected by default. - enabled: # true | false + enabled: bool # Required: metric description. description: # Optional: extended documentation of the metric. @@ -68,11 +74,11 @@ metrics: # Required: metric type with its settings. : # Required for sum and gauge metrics: type of number data point values. - value_type: # int | double + value_type: # Required for sum metric: whether the metric is monotonic (no negative delta values). - monotonic: # true | false + monotonic: bool # Required for sum metric: whether reported values incorporate previous measurements # (cumulative) or not (delta). - aggregation: # delta | cumulative + aggregation: # Optional: array of attributes that were defined in the attributes section that are emitted by this metric. - attributes: + attributes: []string