[docs/collector/building] add more context for what you can build #4343
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2,10 +2,70 @@ | |
| title: Building custom components | ||
| description: Instructions on how to build your own collector components | ||
| weight: 90 | ||
| cSpell:ignore: mdatagen ottl | ||
| --- | ||
|
|
||
| The OpenTelemetry Collector can not only be extended by existing components, but | ||
| also by custom components, that you develop and build on your own. Here you will | ||
| find instructions on how to build some of those components. For additional | ||
| details take a look into the documents contained within the | ||
| [opentelemetry-collector-contrib repository](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/README.md). | ||
|
|
||
| There are several | ||
| [classes](https://github.com/open-telemetry/opentelemetry-collector/blob/67d37183e6ac9b7180fefc6dc3a55f2a75c12fba/cmd/mdatagen/main.go#L192) | ||
| of OpenTelemetry Collector components, such as | ||
|
|
||
| - `receivers`: Scrapers and Listeners which | ||
| [ingest data](https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver#readme), | ||
| typically used to convert data from an external source to OTLP | ||
| - `exporters`: Ways to | ||
| [export data](https://github.com/open-telemetry/opentelemetry-collector/tree/main/exporter#readme) | ||
| to non-OTLP formats, vendor-specific backends, or other Observability tools | ||
|
There was a problem hiding this comment. not only non-OTLP: we do have OTLP exporters There was a problem hiding this comment. So, this was changed in a previous review to specifically mention non-otlp formats. I was presuming that was because only collector-core hosts the otlp exporter, but you're right in that even non-otlp exporters can export in the OTLP format, so will change. This also goes to your audience question. I think the goal is to be generic enough to cover all cases, but building your own collector via OCB and adding new components to |
||
| - `processors`: Ways to | ||
| [process data](https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor#readme) | ||
| in a pipeline | ||
| - `connectors`: | ||
| [Connects](https://github.com/open-telemetry/opentelemetry-collector/tree/main/connector#readme) | ||
| the output (exporter) of one pipeline to the input (receiver) of another in a | ||
| single component | ||
| - `extensions`: Ways to | ||
| [augment the collector runtime](https://github.com/open-telemetry/opentelemetry-collector/blob/main/extension/README.md), | ||
| such as providing | ||
|
Comment on lines
+18
to
+33
There was a problem hiding this comment. This duplicates https://opentelemetry.io/docs/collector/configuration/#basics, doesn't it_ There was a problem hiding this comment. The intent is significantly different. Configuration is intended/written for aspiring users of the collector, while this guide is for aspiring contributors (or at least, developers). The introduction to the idea of different "classes" of components will be inherently similar though, yes. There was a problem hiding this comment. Sorry, it's still unclear to me what the audience and intent of this doc is. Is it for people building their custom components? Is it a "how to add your component to contrib"? Is it "how to contribute to the opentelemetry-collector and/or opentelemetry-collector-contrib repositories"? |
||
| [health checks](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/extension/healthcheckextension/README.md) | ||
| - `cmd`: Commands for building and/or maintain the collector | ||
|
There was a problem hiding this comment. This should be removed, it's not relevant for people extending the collector. There was a problem hiding this comment. When you say "this", do you mean "remove information regarding extensions", or the health check specifically? IMO it's relevant for people extending the collector, and if not, we should change the documentation which I read as indicating such
Also, we already have an article showing how to write an extension, so I think it would be a bit weird if we didn't mention extensions on the index of "building custom components" There was a problem hiding this comment. I mean specifically this item on the line I commented, |
||
| - `pkg`: | ||
|
There was a problem hiding this comment. Similarly, I don't think this should be included in this page. There was a problem hiding this comment. I would appreciate a bit more elaboration on why you feel this way. I do think the There was a problem hiding this comment. We don't have support for adding |
||
| [Packages](https://github.com/search?q=org%3Aopen-telemetry+%22class%3A+pkg%22&type=code) | ||
| for adding functionality to the collector, such as support for | ||
| [golden tests](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/golden#readme) | ||
| or | ||
| [OTTL functions](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/pkg/ottl#readme) | ||
|
There was a problem hiding this comment. Can we add an OTTL function by just adding a new package? It looks like it should be possible (cc @TylerHelmuth). However, we don't have an easy way to add a pkg to the a distribution right now. It would be a cool feature to have for the builder though.
Comment on lines
+37
to
+41
There was a problem hiding this comment. I am against promoting the usage of the There was a problem hiding this comment. That's totally fair. Would you be okay with an aside saying something to the effect of
|
||
|
|
||
| Most components are registered using | ||
|
There was a problem hiding this comment. This is only relevant if people are adding components to contrib. Perhaps this doc needs to be clearer on what's the target audience? Am I reading this because I'm developing a proprietary component to be used with my own distribution, used internally at my company? Or is this about developing an open-source component that is meant to be distributed as part of contrib? I would say that this doc is for the first case, and would expect people to check the contribution guidelines in contrib if they are looking for the second case. There was a problem hiding this comment. Hey, good call, I'll work on making the audience more explicit. We had a discussion and the current consensus as far as I understand it is
There was a problem hiding this comment.
I'll let @svrnm confirm, but I believe the focus should be those many use cases where folks want to have their "own" thing. We can keep contrib off this doc, as it would make it a lot simpler, while still pointing people to the contrib-specific guidelines at that repo. There was a problem hiding this comment. I thought about this, and sorry if this is different to what we have discussed before, but I lean towards "making it a lot simpler", so keeping things out of the docs that are contrib-specific. If we think that they would add additional value, we can still add them in later as well. |
||
| [`mdatagen`](https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/mdatagen#readme), | ||
| a tool which automatically generates code that specifies the adoption status | ||
| (stability) of your component along with documentation and component specific | ||
| information. Most significantly, `mdatagen` configuration is used to describe | ||
| out-of-the-box metrics for scraping receivers, and which | ||
| [signals](/docs/concepts/signals/) or | ||
| [attributes](/docs/specs/otel/common/#attribute) are enabled by default for | ||
| collection. The names of these signals and attributes try to follow their | ||
| respective [specification](/docs/specs/semconv/general/attribute-naming/), and | ||
| any [semantic conventions](/docs/specs/semconv/) which may exist. | ||
|
|
||
| The core | ||
| [`opentelemetry-collector`](https://github.com/open-telemetry/opentelemetry-collector/blob/main/README.md) | ||
| is the basis for all OpenTelemetry components and Collector development. For | ||
| specific, non-universal integrations, contributors look to the | ||
| [`-contrib`](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/README.md) | ||
| repository for all but the most generic integrations, and vendors often base | ||
| their distributions off of the `-contrib` repository. | ||
|
|
||
| Note that | ||
| [adding a component](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/CONTRIBUTING.md#adding-new-components) | ||
| to the `-contrib` distribution incurs a high bar of support, and while | ||
| contributions of new components are encouraged, adoption is a process that | ||
| requires ongoing support and commitment to the OpenTelemetry project. | ||
|
|
||
| If you wish to build your own [distribution](../distributions/) of the | ||
| collector, you may use the | ||
| [`OpenTelemetry Collector builder`](../custom-collector). | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would link to our documentation, I don't think a link to mdatagen's internals is the right approach here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree linking to the internal code is suboptimal. I don't think linking to configuration is the right idea either though.
Should I add a blurb to the
mdatagenreadme to talk about what a "class" is? Or should I instead link to the metadata-schema.yaml file?I think so long as we link to somewhere that enumerates all the classes, a reader could follow the breadcrumbs to more detailed documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we should link at mdatagen at all. The fact that we use mdatagen is an implementation detail: we do not require for you to use mdatagen to create a new component: you just need a
go.modand a properly-typedNewFactoryfunction and the builder can take it.