Document how to use mdatagen and the schema of metadata.yaml

cmd/mdatagen/README.md

@@ -1,42 +1,56 @@
 # Metadata Generator
 
-Components must contain a `metadata.yaml` file that documents various aspects of the component including:
+Every component's documentation should include a brief description of the component and guidance on how to use it.
+There is also some information about the component (or metadata) that should be included to help end-users understand the current state of the component and whether it is right for their use case.
+Examples of this metadata about a component are:
 
 * its stability level
 * the distributions containing it
 * the types of pipelines it supports
 * metrics emitted in the case of a scraping receiver
 
-Current examples:
+The metadata generator defines a schema for specifying this information to ensure it is complete and well-formed.
+The metadata generator is then able to ingest the metadata, validate it against the schema and produce documentation in a standardized format.
+An example of how this generated documentation looks can be found in [documentation.md](./documentation.md).
 
-* hostmetricsreceiver scrapers like the [cpuscraper](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/metadata.yaml)
+## Using the Metadata Generator
 
-See [metadata-schema.yaml](metadata-schema.yaml) for file format documentation.
+In order for a component to benefit from the metadata generator (`mdatagen`) these requirements need to be met:
+1. A `metadata.yaml` file containing the metadata needs to be included in the component
+2. The component should declare a `go:generate mdatagen` directive which tells `mdatagen` what to generate
 
-If adding a new receiver a `doc.go` file should also be added to trigger the generation. See below for details.
+As an example, here is a minimal `metadata.yaml` for the [OTLP receiver](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver):
+```yaml
+type: otlp
+status:
+  class: receiver
+  stability:
+    beta: [logs]
+    stable: [metrics, traces]
+```
 
-## Generating
+Detailed information about the schema of `metadata.yaml` can be found in [metadata-schema.yaml](./metadata-schema.yaml).
 
-`make generate` triggers the following actions:
+The `go:generate mdatagen` directive is usually defined in a `doc.go` file in the same package as the component, for example:
+```go
+//go:generate mdatagen metadata.yaml
 
-1. `cd cmd/mdatagen && $(GOCMD) install .` to install `mdatagen` tool in`GOBIN`
+package main
+```
 
-2. All `go:generate mdatagen` directives that are usually defined in `doc.go` files of the metric scrapers will be 
-   executed. For example,
-   [/receiver/hostmetricsreceiver/internal/scraper/cpuscraper/doc.go](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/doc.go)
-   runs `mdatagen` for the [metadata.yaml](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/metadata.yaml)
-   which generates the metrics builder code in
-   [internal/metadata](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/internal/metadata)
-   and [documentation.md](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper/internal/metadata) with
-   generated documentation about emitted metrics.
+Below are some more examples that can be used for reference:
 
-## Development
+* The ElasticSearch receiver has an extensive [metadata.yaml](../../receiver/elasticsearchreceiver/metadata.yaml)
+* The host metrics receiver has internal subcomponents, each with their own `metadata.yaml` and `doc.go`. See [cpuscraper](../../receiver/hostmetricsreceiver/internal/scraper/cpuscraper) for example.
 
-In order to introduce support of a new functionality in metadata.yaml:
+You can run `cd cmd/mdatagen && $(GOCMD) install .` to install the `mdatagen` tool in `GOBIN` and then run `mdatagen metadata.yaml` to generate documentation for a specific component or you can run `make generate` to generate documentation for all components.
 
-1. Make code changes in the (generating code)[./loader.test] and (templates)[./templates/].
-2. Add usage of the new functionality in (metadata.yaml)[./metadata.yaml].
-3. Run `make mdatagen-test`.
-4. Make sure all tests are passing including (generated tests)[./internal/metadata/generated_metrics_test.go].
-5. Run `make generate`.
+## Contributing to the Metadata Generator
 
+The code for generating the documentation can be found in [loader.go](./loader.go) and the templates for rendering the documentation can be found in [templates](./templates).
+When making updates to the metadata generator or introducing support for new functionality:
+
+1. Ensure the [metadata-schema.yaml](./metadata-schema.yaml) and [./metadata.yaml](metadata.yaml) files reflect the changes.
+2. Run `make mdatagen-test`.
+3. Make sure all tests are passing including [generated tests](./internal/metadata/generated_metrics_test.go).
+4. Run `make generate`.

cmd/mdatagen/metadata-schema.yaml

@@ -1,15 +1,21 @@
-# Required: The type of the component.
+# Required: The type of the component - Usually the name. The type and class combined uniquely identify the component (eg. receiver/otlp) or subcomponent (eg. receiver/hostmetricsreceiver/cpu)
 type:
 
-# Required: A high-level view of the development status and use of this component
+# Required for components (Optional for subcomponents): A high-level view of the development status and use of this component
 status:
   # Required: The class of the component (For example receiver)
   class: <receiver|processor|exporter|connector|extension>
   # Required: The stability of the component - See https://github.com/open-telemetry/opentelemetry-collector#stability-levels
-  stability: <development|alpha|beta|stable|deprecated|unmaintained>
-  # Optional: The distributions that this component is bundled with (For example core or contrib)
+  stability:
+    development: [<metrics|traces|logs>]
+    alpha: [<metrics|traces|logs>]
+    beta: [<metrics|traces|logs>]
+    stable: [<metrics|traces|logs>]
+    deprecated: [<metrics|traces|logs>]
+    unmaintained: [<metrics|traces|logs>]
+  # Optional: The distributions that this component is bundled with (For example core or contrib). See statusdata.go for a list of common distros.
   distributions: []string
-  # Optional: A list of warnings that should be brought to the attention of developers looking to use this component
+  # Optional: A list of warnings that should be brought to the attention of users looking to use this component
   warnings: []string
 
 # Optional: OTel Semantic Conventions version that will be associated with the scraped metrics.
@@ -24,7 +30,7 @@ resource_attributes:
     # Required: description of the attribute.
     description:
     # Optional: array of attribute values if they are static values (currently, only string type is supported).
-    enum:
+    enum: []string
     # Required: attribute value type.
     type: <string|int|double|bool|bytes|slice|map>
 
@@ -38,7 +44,7 @@ attributes:
     # Required: description of the attribute.
     description:
     # Optional: array of attribute values if they are static values (currently, only string type is supported).
-    enum:
+    enum: []string
     # Required: attribute value type.
     type: <string|int|double|bool|bytes|slice|map>
 
@@ -47,7 +53,7 @@ attributes:
 metrics:
   <metric.name>:
     # Required: whether the metric is collected by default.
-    enabled: # true | false
+    enabled: bool
     # Required: metric description.
     description:
     # Optional: extended documentation of the metric.
@@ -68,11 +74,11 @@ metrics:
     # Required: metric type with its settings.
     <sum|gauge>:
       # Required for sum and gauge metrics: type of number data point values.
-      value_type: # int | double
+      value_type: <int|double>
       # Required for sum metric: whether the metric is monotonic (no negative delta values).
-      monotonic: # true | false
+      monotonic: bool
       # Required for sum metric: whether reported values incorporate previous measurements
       # (cumulative) or not (delta).
-      aggregation: # delta | cumulative
+      aggregation: <delta|cumulative>
     # Optional: array of attributes that were defined in the attributes section that are emitted by this metric.
-    attributes:
+    attributes: []string