Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] Design doc for Go Launcher #2592

Closed
Closed
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions design/0000-launcher.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Launcher

## What is it?

The Go Launcher (name TBD) is a configuration layer that chooses default values for configuration options that many OpenTelemetry users would ultimately configure manually, allowing for minimal code to quickly instrument with OpenTelemetry.

## Motivation

The goal of this Launcher is to help users that aren't familiar with OpenTelemetry quickly ramp up on what they need to get going and instrument their applications. [There is currently a lot of boilerplate code required to use OpenTelemetry](https://opentelemetry.io/docs/instrumentation/go/manual/#initializing-a-new-tracer), and users must make decisions early on before instrumentation works - for example, they must determine and find the appropriate exporters, set resource attributes, etc. This launcher is intended to simplify the setup and minimize decision-making overhead to be able to quickly get started.

Separately, vendors may choose to provide a vendor-specific package compatible with this launcher for easier exporting to their own backend. Neither the launcher nor the vendor-specific package are required to use OpenTelemetry, and using the launcher does not require the end user to use a vendor-specific package.

## Explanation

The Go Launcher provides a single function to reduce the amount of code that needs to be written to get started, and provides additional, straightforward options to add in that still make additional configuration easy.

The vendor-neutral launcher will live in the opentelemetry-go-contrib repo, and vendors may create separately hosted vendor-specific configuration packages that can be used with the launcher.

### Getting started

```shell
go get github.com/opentelemetry-go-contrib/otel-launcher-go/launcher
JamieDanielson marked this conversation as resolved.
Show resolved Hide resolved
```

### Configure

Minimal setup - by default will send all telemetry to localhost:4317, with the ability to set environment variables for endpoint, headers, resource attributes, and more as listed in the configuration options noted below:

```go
import "github.com/opentelemetry-go-contrib/otel-launcher-go/launcher"

func main() {
lnchr, err := launcher.ConfigureOpentelemetry()
defer lnchr.Shutdown()
}
```

Or set headers directly in code instead:

```go
import "github.com/opentelemetry-go-contrib/otel-launcher-go/launcher"

func main() {
lnchr, err := launcher.ConfigureOpentelemetry(
launcher.WithServiceName("service-name"),
launcher.WithHeaders(map[string]string{
"service-auth-key": "value",
"service-useful-field": "testing",
}),
)
defer lnchr.Shutdown()
}
```

### Configuration Options
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For expoter-options below, do these assume OTLP is the exporter, or would they be expected to work with "registered" exporters as well?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design has been updated to remove OTLP-specific configuration options as they would be handled with the exporter itself; the design now specifies using the autoexport package.


| Config Option | Env Variable | Required | Default |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any options related to the BatchSpanProcessor. I'd assume that would be used and the user might need a way to modify it.

This makes me wonder more generally what the transition path would be for a user who outgrows the launcher and wants to convert to more explicit initialization. It would be awesome (though maybe overkill) if there was a lnchr.Generate() method that could emit the code that would be necessary to explicitly instantiate the launcher.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The launcher uses the BatchSpanProcessor by default. There currently is not a plan to swap out the batch processor with, for example, the SimpleSpanProcessor. In our other distros (Java, .NET) we have been using the batch processor with no option to swap it out, and have not yet received any feedback requesting to make it configurable.

The Generate() function you mention does sound like a great stretch goal, but it may be too much for the initial implemenetation here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I certainly wouldn't suggest using the SimpleSpanProcessor, but the BatchSpanProcessor does have several configuration options. We should consider where the breakpoint should be that forces users to drop the simple setup path and use fully manual SDK configuration. I think that adjusting batch size or timing should probably be on the simple side of that line.

| -------------------------- | ----------------------------------- | -------- | -------------------- |
| WithServiceName | OTEL_SERVICE_NAME | y | - |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When issues are created from this, we need to ensure this can also be set via OTEL_RESOURCE_ATTRIBUTES

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just make it not required? Won't the SDK pull from the environment if a resource is provided that doesn't have a service name?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the goal is to support all OTEL_* environment variables and as such, honor a service name set in OTEL_RESOURCE_ATTRIBUTES. Agreed that it shouldn't really be required per se, as it should have the default of unknown_service:<process>. I will update this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clarified the service name requirement in e039649

| WithServiceVersion | OTEL_SERVICE_VERSION | n | - |
| WithHeaders | OTEL_EXPORTER_OTLP_HEADERS | n | {} |
| WithTracesExporterEndpoint | OTEL_EXPORTER_OTLP_TRACES_ENDPOINT | n | localhost:4317 |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The OTEL_TRACES_EXPORTER and OTEL_METRICS_EXPORTER (and eventually OTEL_LOGS_EXPORTER) environment variables should be supported. Ideally, in the same way the propagators are: by allowing users to register their own and then call them here. This likely means an "autoexport" similar to autoprop needs to be created as an item for this project.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal would be to support all OTEL_* environment variables. The interface for setting configuration options in code, like WithTracesExporterEndpoint, would probably remain more limited.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design has been updated to remove OTLP-specific configuration options as they would be handled with the exporter itself; the design now specifies using the autoexport package.

| WithTracesExporterInsecure | OTEL_EXPORTER_OTLP_TRACES_INSECURE | n | false |
| WithMetricsExporterEndpoint | OTEL_EXPORTER_OTLP_METRICS_ENDPOINT | n | localhost:4317 |
| WithMetricsExporterInsecure | OTEL_EXPORTER_OTLP_METRICS_INSECURE | n | false |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These options seem generic across all exports yet they pair with OTLP exporter environment variables. How do we plan to support other exporter configuration?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be important to get right, otherwise the promise of vendor neutrality is hollow and only applies to vendors that accept OTLP.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The initial intent was to have this specific to OTEL and OTLP, not to have it extend to other exporters, as OTLP is vendor-neutral but yes, would require accepting the OTLP format. Other exporter APIs could be added later on though. As a stop-gap, the collector can be used to translate to other formats.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design has been updated to remove OTLP-specific configuration options as they would be handled with the exporter itself; the design now specifies using the autoexport package.

| WithLogLevel | OTEL_LOG_LEVEL | n | info |
| WithPropagators | OTEL_PROPAGATORS | n | tracecontext,baggage |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally this will be extensible. Possibly by using the autoprop package to allow users to register their own propagators.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seconded. It should use the autoprop.GetTextMapPropagator() function being added Soon™. When a similar registry is created for exporters it should also have this ability.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're understanding this right, the autoprop package seems like the right move for specifying propagators, and later, the autoexporter(?) package will be useful for specifying exporters. Since autoprop is already merged in we'll be able to use that sooner rather than later, and for now keep the exporters limited to http and grpc as planned.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Design has been updated to remove OTLP-specific configuration options as they would be handled with the exporter itself; the design now specifies using the autoexport package.

| WithResourceAttributes | OTEL_RESOURCE_ATTRIBUTES | n | - |
| WithMetricsReportingPeriod | OTEL_EXPORTER_OTLP_METRICS_PERIOD | n | 30s |
| WithMetricsEnabled | OTEL_METRICS_ENABLED | n | true |
| WithTracesEnabled | OTEL_TRACES_ENABLED | n | true |
| WithProtocol | OTEL_EXPORTER_OTLP_PROTOCOL | n | grpc |

### Additional Configuration Options

Not yet implemented but still desired configuration options would include:

- `OTEL_EXPORTER_OTLP_PROTOCOL`: `http/protobuf`
- ideally, selecting this would change the default endpoint to localhost:4318

### Using a Vendor-Specific Package

Vendors may create and maintain convenience configuration packages to more easily setup for export to a specific backend. For example, using a Honeycomb package would look like this:

Minimal setup, which sends to the Honeycomb endpoint and requires `HONEYCOMB_API_KEY` environment variable:

```go
import (
"github.com/opentelemetry-go-contrib/otel-launcher-go/launcher"
_ "github.com/honeycombio/otel-launcher-go/launcher/honeycomb"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work? Does the launcher package export some API that can be used by the honeycomb package in init(), thus allowing the import for side effects pattern? What is that API?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still being worked on separately, but a peek at what the vendor code would look like lives here: https://github.com/honeycombio/otel-launcher-go/blob/mike/reorganise/honeycomb/honeycomb.go

As a side note (and what you'll see in the directory for the link above), we're planning to include a baggage span processor and dynamic attribute processor in our package, along with other niceties, though they could probably be pushed upstream if there was an appetite for it. The baggage span processor allows for easily attaching baggage to child spans, and the dynamic attribute span processor allows for adding dynamic fields to spans (like current memory util).

Once we have the vendor-specific package in a cleaner place for sharing, I can link that up.

)

func main() {
lnchr, err := launcher.ConfigureOpentelemetry()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not super fond of silently imported packages. How about something like this:

lnchr, err := launcher.ConfigureOpentelemetry(
    honey.Configure(),
)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we've had some feedback to suggest the same. We liked the idea behind it when reviewing other sources, eg sql drivers, but it's easy to miss and not see why it's not working as expected.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both could be supported?

It seems to me that an anonymous import could easily be missed as well.
To help with that, the launcher could provide errors if something appears to be wrong, such as no vendor/provider being setup.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both are supported today - you can use our layer without the anonymous import and add a line of code similar to what you have. We (honeycomb) may consider not enabling the anonymous import, or at least making it not the documented/preferred way, since it can be easily missed. Good feedback that you feel the same way.

Stepping back a bit, how prescriptive do you think we (opentelemetry contributors) should be here? Should we define a set of must-have/must-not-haves for vendor layers to follow?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this feature is meant to help folks get started, I think things should be as simple and clear as possible, and avoid providing several ways to do the same thing (which goes in opposition with my previous comment, I know).
It also feels to me that we should try to let users know about anything that we can detect which seems wrong, such as telemetry configured with no provider, or a missing service name for example.
Anything that may be confusing to users when the get started, and that we can automatically detect could be provided as an information to them.

defer lnchr.Shutdown()
}
```

Alternatively, to set the Honeycomb API key directly in code:

```go
import (
"github.com/opentelemetry-go-contrib/otel-launcher-go/launcher"
"github.com/honeycombio/otel-launcher-go/launcher/honeycomb"
)

func main() {
lnchr, err := launcher.ConfigureOpentelemetry(
honeycomb.WithApiKey("api-key"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this would return a launcher.Option, probably by wrapping launcher.WithHeaders() like this:

func WithAPIKey(key string) launcher.Option {
  return launcher.WithHeaders(map[string]string{"HONEYCOMB_API_KEY": key})
}

Would the launcher.Option type operate on an exported launcher.Config type allowing the implementation of wholly-new option implementations, or would everything need to be done in terms of existing launcher.Option implementations? We've traditionally not exported the config type to avoid unintended/unsupportable behavior, but this might be a place where doing it make sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumption is correct, that it would return a launcher.Option.

Regarding unintended/unsupportable behavior, vendors can add config validation for their own use cases. For example, our vendor package (WIP) has a launcher.ValidateConfig that returns errors for invalid headers.

)
defer lnchr.Shutdown()
}
```

## Trade-offs and mitigations

This launcher is intentionally providing default configuration options, which may not precisely map out to the final configuration an end user desires. As such, there may be dependencies in the package that are not being used. For example, the default configuration will bring in both gRPC and HTTP exporters; using gRPC will result in HTTP being included but not used; similarly, using HTTP/protobuf will result in the gRPC dependency being pulled in but unused.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we decouple the launcher framework from the included components and then provide stock distributions that the user could select from in the same way they'd select a vendor distribution?

import (
  go.opentelemetry.io/contrib/launcher
  _ go.opentelemetry.io/contrib/launcher/otlp/grpc // or just /otlp for both HTTP and gRPC
)

This adds slightly more boilerplate, but also makes things somewhat more consistent.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would still want to use the package that exposes both http and grpc launchers. As far as requiring the user to add their own imports here, we feel like this takes away from the point of the launcher. If someone feels really strongly about this they can use the SDK without the launcher.


There still exists the ability to change the default configuration with environment variables or in-code configuration options, but there is not an easy way to remove unused dependencies without also keeping the minimal configuration option.

If removing unused dependencies is required, the user can avoid using the launcher and initialize OTel manually, which is the current experience today.