[Feat] Better control of threads executed by opentelemetry-cpp

[marcalff]

Context

The opentelemetry-cpp uses threads internally, for various components.

For example:

• the batch processor uses a thread to process records from the span buffer.
• the periodic metric reader uses a thread to trigger periodic collection, and a child thread to collect observable metrics.
• the log processor uses a thread to process records from a log buffer.
• every exporter based on CURL uses a thread internally, to process HTTP requests and responses.

Problem

An application that links with the opentelemetry-cpp library may have additional constraints, and may want to control at a very low level how the code executes within a thread.

For example:

• the application may want to name threads, to be visible in the operating system under an application controlled name.
• the application may need to initialize thread local storage, in particular when custom code is executed within a thread spawned by opentelemetry. Typically, callbacks for observable metrics may need TLS keys in place to execute properly.
• the application may need to control thread priorities, or bind internal opentelemetry threads to some CPUs.
• the application may need to control the current network namespace, when a thread opens a network connection.

Currently, there is no way to implement any of this from an application that uses opentelemetry-cpp.

[lalitb]

This was discussed earlier, with some proposal here - #1822 (comment)

[owent]

There is another discussion in #1250 , I think the best solution is implementing a event driven module, which just like libuv, libevent, boost.asio or other simular solution. We can handle timer callback and IO event in a shared thread pool.
Or can we just import libuv or libevent?

[marcalff]

This is not about thread scheduling.

The need here is to add hooks to an opentelemetry-cpp thread, to get a chance to execute at strategic locations:

• https://man7.org/linux/man-pages/man2/setns.2.html
• https://man7.org/linux/man-pages/man3/pthread_setname_np.3.html
• and many more