Capability to update tail sampling process configuration dynamically at runtime

[gongcon]

Important (read before submitting)

We are currently preparing for the upcoming 1.0 GA release. Feature requests that are not aligned with
the current roadmap and are not aimed at stabilizing and preparing the Collector for the release will
not be prioritized.

Delete this paragraph before submitting.

Is your feature request related to a problem? Please describe.
Tail sampling processor buffers spans in memory to make sampling decision of traces. When we update the config, sampling percentage for example, we update the yaml file and send a SIGHUP to the collector process. However, Otel collector restarts when it receives the SIGHUP. The buffered data are lost in the restart.

Describe the solution you'd like
Either the tail sampling process provides a way to dynamically update configuration at runtime, or Otel collector can do it in general. The later one is better since other processors, e.g. batch processor, may face the same issue.

It can be a feature of OpAMP as well as @cforce mentioned in #4205.

[cforce]

flushing of buffers from samplers could also be a solution. A friendly tear down before an opamp initiated restart of the collector would make sure buffered traces from tail sampling are not dropped but send to the exporters as eviction strategy. Maybe this is even a feature already with proper cfg - see https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/processor/tailsamplingprocessor/README.md#dropped-traces

[gongcon]

It's not a feature in tail sampling processor. It drops spans when the buffer is full, nothing to do with graceful shutdown.

[cforce]

Graceful shutdown related feature open-telemetry/opamp-go@528e802