feat(exporters)!: rewrite exporter config logic

[pichlermarc]

Which problem is this PR solving?

Config code for exporters (user-provided, defaults, and env config) is currently hard to test individually and is spread/duplicated all over the different signal exporters.

This PR moves the configuration as much as possible into separate files/functions and splits them up into:

• defaults
• fallback merging (user-provided merging with env var config and defaults)
  • fallback merging is intentionally held modular in an effort to possibly allow a file-config fallback in the future
• parsing the config from the environment

Since most config is now moved to base-packages we can remove the in-depth tests from the specific packages. This reduces duplication and makes further changes easier, as they will touch "fewer" files. Since now the configuration-code is not part of the individual exporter-implementations, collapsing the base exporters into one class becomes simpler as we have to worry less about the inheritance structure (abstract methods were removed in this PR as they were config-related).

In addition to these improvements, this PR also fixes #3748, #3747 which was a side-effect of the previous way of implementing env var config.

Breaking changes:

• All exporters:
  • (user-facing) getDefaultUrl was intended for internal use has been removed from all exporters
  • (user-facing) getUrlFromConfig was intended for internal use and has been removed from all exporters
  • (user-facing) hostname was intended for internal use and has been removed from all exporters
  • (user-facing) url was intended for internal use and has been removed from all exporters
  • (user-facing) timeoutMillis was intended for internal use and has been removed from all exporters
  • (user-facing) onInit was intended for internal use and has been removed from all exporters
• Exporter base package:
  • (internal) parseHeaders is now not exported anymore
  • (internal) appendResourcePathToUrl is now not exported anymore
  • (internal) appendResourcePathToUrlIfNeeded is now not exported anymore
  • (internal) configureExporterTimeout is now not exported anymore
  • (internal) invalidTimeout is now not exported anymore

Follow-up steps:

• this PR did not apply the same to the metrics-specific enviornment variables like OTEL_EXPORTER_OTLP_TEMPORALITY_PREFERENCE, this will be done in a follow-up.
• consolidate all base-classes OTLPExporterNodeBase, OTLPExporterBrowserBase, OTLPGrpcExporterNodeBase into one shared base, that favors composition over inheritance. This can be done by:
  • introducing a function that converts the "legacy" (i.e. current) configuration to the new configuration model (introduced in this PR)
  • creating a factory function that creates an OTLPExporterDelegate (draft) based on the "new" config, this replaces the OTLPExporterNodeBase, OTLPExporterBrowserBase, OTLPGrpcExporterNodeBase classes which right now do essentially the same, except for the constructor which will be replaced by the factory function.
  • final exporters don't inherit from any of these anymore, but they simply implement all functions by calling the the delegate's functions.
• clean up existing tests to avoid testing export-output in-depth (this is tested in @opentelemetry/otlp-transformer, testing that the output is the correct format should suffice)

Fixes #3748
Fixes #3747

Type of change

• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Bug fix ([otlp-grpc-exporter-base] base package is using trace-specific env vars #3748 and [exporter-logs-otlp-grpc] add support for logs specific env vars #3747)

How Has This Been Tested?

• Remaining unit tests
• New unit tests

[codecov]

Codecov Report

Attention: Patch coverage is 95.59322% with 13 lines in your changes missing coverage. Please review.

Project coverage is 93.26%. Comparing base (f8ab559) to head (e24837d).
Report is 24 commits behind head on main.

Files with missing lines	Patch %	Lines
...se/src/platform/browser/OTLPExporterBrowserBase.ts	20.00%	4 Missing ⚠️
.../src/platform/node/convert-legacy-agent-options.ts	50.00%	4 Missing ⚠️
...grpc-exporter-base/src/OTLPGRPCExporterNodeBase.ts	72.72%	3 Missing ⚠️
...e/src/configuration/otlp-http-env-configuration.ts	95.74%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4971      +/-   ##
==========================================
- Coverage   93.39%   93.26%   -0.14%     
==========================================
  Files          46      317     +271     
  Lines         712     8194    +7482     
  Branches      120     1640    +1520     
==========================================
+ Hits          665     7642    +6977     
- Misses         47      552     +505     

Files with missing lines	Coverage Δ
...ges/exporter-logs-otlp-grpc/src/OTLPLogExporter.ts	100.00% <ø> (ø)
...ogs-otlp-http/src/platform/node/OTLPLogExporter.ts	100.00% <100.00%> (ø)
...gs-otlp-proto/src/platform/node/OTLPLogExporter.ts	100.00% <100.00%> (ø)
.../exporter-trace-otlp-grpc/src/OTLPTraceExporter.ts	100.00% <ø> (ø)
...e-otlp-http/src/platform/node/OTLPTraceExporter.ts	100.00% <100.00%> (ø)
...-otlp-proto/src/platform/node/OTLPTraceExporter.ts	100.00% <100.00%> (ø)
...porter-metrics-otlp-grpc/src/OTLPMetricExporter.ts	100.00% <ø> (ø)
...-otlp-http/src/platform/node/OTLPMetricExporter.ts	100.00% <100.00%> (ø)
...orter-metrics-otlp-proto/src/OTLPMetricExporter.ts	100.00% <100.00%> (ø)
...ackages/otlp-exporter-base/src/OTLPExporterBase.ts	94.44% <ø> (ø)
... and 13 more

... and 250 files with indirect coverage changes

[david-luna]

this was a big one :)

I like the approach to have config separated by functions and the merge the results. Just a couple of questions before approving it. Thanks for taking the time to simplify the config

[david-luna]

nit: do you think signals are going to grow much in the future that we need to have the identifier as a string type?

[pichlermarc]

Hmm, I don't think they will grow much. I mostly just chose string as it was the easiest choice to work with and it does not require us to export a new type that could be difficult to extend later.

Do you think we should have a SignalIdentifier type? 🤔

[david-luna]

one might think is a commodity but with the type the editor helps you to avoid passing a string which does not refer to any Signal. Is not wrong to not have a type here. I'll leave it to you

[pichlermarc]

I think I will leave it as-is - maybe I'm traumatized by making unintentional breaking changes in TypeScript libraries, but I feel like whatever type I can come up with won't be extendable in a safe way later on. 😕

The fact that libraries have to be type-compatible with other versions of themselves is kind of a mind-bender that did not exist in other languages I came from originally. This API will mostly be internal, so I'll opt to leave it like this for now. We can re-visit this choice in a later iteration - I'm planning to have a final round of API review when the whole exporter restructuring is over to work out smaller things like this. 🙂

[david-luna]

The function is not checking that

• url does not already contain a path
• the result string is a valid URL, path could contain invalid chars

So the following call is possible

appendResourcePathToUrl('http://foo.com/bar/test.txt', 'v1/logs')
// returns 'http://foo.com/bar/test.txt/v1/logs'

I guess we should let the users define the URL they want but I think we should diag.warn or diag.debug the result for debug purposes (please ignore this comment if the diag. is set in a file below, I haven't reached them yet)

[pichlermarc]

Good point about checking that the final URL is valid. 👍 I added additional checks and logs to that function. I settled on warn as it's very likely that it'll not be able to continue with an invalid URL. 5d0aae1

It's actually intended to append if it does not contain a path already - this is specified behavior for the OTEL_EXPORTER_OTLP_ENDPOINT env var where this function is used. See https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md#endpoint-urls-for-otlphttp

For signal-specific ones like OTEL_EXPORTER_OTLP_ENDPOINT no path is actually appended - just the root path if none is there already that's the function above this one 🙂

[david-luna]

bar: undefined does not appear in the result but baz: null does and it's stringified. Any reason for that?

[pichlermarc]

This is mostly to keep the same behavior of the current implementation, which also handles it this way 🙂

undefined

bar: undefined is dropped as calls with http module will throw when there's an undefined header value:

> const http = require('http');
undefined
> http.get({host: 'example.com', headers: {'foo': undefined}})
Uncaught:
TypeError [ERR_HTTP_INVALID_HEADER_VALUE]: Invalid value "undefined" for header "foo"
    at __node_internal_captureLargerStackTrace (node:internal/errors:496:5)
    at new NodeError (node:internal/errors:405:5)
    at __node_internal_ (node:_http_outgoing:624:11)
    at ClientRequest.setHeader (node:_http_outgoing:651:3)
    at new ClientRequest (node:_http_client:291:14)
    at request (node:http:100:10)
    at Object.get (node:http:111:15) {
  code: 'ERR_HTTP_INVALID_HEADER_VALUE'
}

We've had users in the past that have ended up with undefined header values as they set a header to process.env.MY_HEADER_ENV_VAR, did not set that env var and then ended up with an exporter that would constantly fail.

null

null is stringified as to ensure it matches the type for Record<string, string> for the headers. Plus it gets stringified to null anyway when it's sent via with the http module so we make it happen earlier to make the type a bit more simple to work with down the chain as we don't have to worry about null values. It's also the current behavior of the exporters, so I tried not to change it.

You can try sending it to https://webhook.site to see that it ends up the same:

> const https = require('https');
undefined
> https.get({host: 'webhook.site', headers: {'foo': null}, path: '/<your-webhook-site-path>'})
> https.get({host: 'webhook.site', headers: {'foo': 'null'}, path: '/<your-webhook-site-path>'})