Tracing Dynamic Configuration

[marcotc]

Adds support for the ddtrace to receive APM_LIBRARY Dynamic Configuration, through the existing Remote Configuration channel, and applies these changes to the local application.

Dynamic Configuration has precedence over locally configured values.

Dynamic Configuration can be disabled at any point in time, restore the locally configured values.

This PR passes our new system-tests for Dynamic Configuration: DataDog/system-tests#1172

Notes

All components are safely reconfigured on-the-fly; ddtrace doesn't need to be reconfigured (Datadog.configure{}) for them to apply.

In fact, invoking Datadog.configure{} to apply Remote Configuration has several issues:

• Dynamic Configuration today does not utilize The Update Framework's feature that allow us to know which values were changed, so we are forced to apply all values all the time, this is problematic because it would mean a Datadog.configure{} every 5 seconds. We could track the value changes ourselves, but now we are adding an unnecessary complexity layer on a system that has already solved it for us.
• The Remote Configuration configuration work is in itself a Component, thus it gets shut down on Datadog.configure{}, which causes the reconfiguration process itself to be aborted (the active thread is killed) and ddtrace ends up in an inconsistent state.
• Moving Datadog.configure{} to a separate execution context (e.g. another thread) opens room for race conditions, especially when the new Remote Configuration component initializes and tries to apply Remote Configuration values very quickly. I tried adding synchronization mechanisms, buy my tests were consistently deadlocking.

Moving Remote Configuration outside of Components is likely an option, but the complexity is quite large was we'd need a separate lifecycle management mechanism for it, and it depends on configuration values (e.g. agent.host, remote.enabled), which are naturally configured alongside the Component lifecycle management. I did not attempt this, as it seemed like its own project, but I think it's worth discussing.

[codecov-commenter]

Codecov Report

Merging #2848 (afb8bee) into master (242c247) will decrease coverage by 0.01%.
The diff coverage is 97.92%.

@@            Coverage Diff             @@
##           master    #2848      +/-   ##
==========================================
- Coverage   98.08%   98.08%   -0.01%     
==========================================
  Files        1309     1316       +7     
  Lines       73439    73820     +381     
  Branches     3378     3385       +7     
==========================================
+ Hits        72031    72403     +372     
- Misses       1408     1417       +9     

Files Changed	Coverage Δ
lib/datadog/core/transport/http/config.rb	83.72% <50.00%> (-0.54%)	⬇️
spec/support/tracer_helpers.rb	89.28% <50.00%> (-1.46%)	⬇️
spec/datadog/core/configuration/components_spec.rb	99.04% <88.23%> (-0.81%)	⬇️
...ib/datadog/tracing/configuration/dynamic/option.rb	95.65% <95.65%> (ø)
lib/datadog/tracing/remote.rb	97.29% <97.29%> (ø)
lib/datadog/core/configuration/components.rb	100.00% <100.00%> (ø)
lib/datadog/core/configuration/option.rb	99.23% <100.00%> (+<0.01%)	⬆️
lib/datadog/core/remote/client/capabilities.rb	100.00% <100.00%> (ø)
lib/datadog/core/remote/component.rb	94.11% <100.00%> (-0.33%)	⬇️
lib/datadog/tracing/component.rb	100.00% <100.00%> (ø)
... and 12 more

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more