Add/bring back API to programmatically change config

[pavolloffay]

Is your feature request related to a problem? Please describe.

PR #2868 broked API and removed config factory https://github.com/open-telemetry/opentelemetry-collector/pull/2868/files#diff-d6c4c49032868b2ceaa66558ff307319762c635975313d30109923f1faeab594L104. The config factory made it possible to change config.Config before supplying it to the service.

This API was used in Jaeger to programmatically change config object. It is also used at Traceable to programmatically change configured objects (we wrap processor and make them remotely configurable).

cc) @bogdandrutu

Describe the solution you'd like

API that allows me to change config.Config before it gets used by the service.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context

[bogdandrutu]

@pavolloffay you can implement this interface https://github.com/open-telemetry/opentelemetry-collector/blob/main/service/parserprovider/provider.go#L25

[pavolloffay]

@pavolloffay you can implement this interface https://github.com/open-telemetry/opentelemetry-collector/blob/main/service/parserprovider/provider.go#L25

That is not sufficient and provides a different API with much less flexibility than the API that the PR removed. The Provider returns a Parser that encapsulates viper. In other words, this API only allows me to configure viper and it does not allow me to change the resolved config.Config.

I need access to resolved config.Config and change it before it gets used by the service.

[pavolloffay]

@bogdandrutu @tigrannajaryan ^^ Could we think of a solution for this API breaking change?

[bogdandrutu]

@jcchavezs @pavolloffay Short answer, to be convinced show me some use-case where changing the config is necessary and how that would help.

If we have an example use-case we can add a "ConfigLoader" interface that takes a Parser and produces the Config, that way you also have that option.

[jcchavezs]

Our use case is pretty simple:

We want to support live config reload, for that we group all the processors of a pipeline into a single macroprocessor. Inside the macroprocessor it is easy and safe to control the processors lifecycle. For that matter we need to intercept the resulting config and:

• inject the factories for our macroprocessor
• modify the pipelines to only have one processor (the macroprocessor)
• copy all the config of the processors in the pipeline so the macroprocessor can bootstrap the processors

Given the change I considered the alternative to create my own koanf parser but that would add a lot of burden and make us sensitive to changes in koanf's implementation for the config.

My proposal is that configparser.Parser becomes an interface which default implementation is the current implementation.

Ping @bogdandrutu