Document how to use with Gunicorn and uWSGI

[owais]

Batch span processor does not work well with Gunicorn's forking model out of the box. Users need to leverage gunicorns worker hooks to setup tracing for each worker after the worker has started. This should be documented with examples.

An example django+gunicorn project that solves the same problem with signalfx-python-tracing:

https://github.com/owais/django-trace-test
https://github.com/owais/django-trace-test/blob/master/src/gunicorn.config.py
https://github.com/owais/django-trace-test/blob/master/docker-entrypoint.sh#L13

[owais]

//cc @lzchen

[alan-yilun-li]

Hi, do you have any examples of how this would work with the opentelemetry-python client? I.e. how signalfx then goes to open-telemetry

[owais]

Documenting this with an example is what this task is about so we'll have one once this issue is resolved. You can replace signalfx tracing with opentelemetry in the example project I shared and it should work.

[srikanthccv]

We should probably have similar thing for ASGI servers such as uvicorn, hypercorn etc..

[lzchen]

@owais
Where do you suggest we put these docs? I believe @aabmass said we can publish docs without having an actual package?

[owais]

I think @lonewolf3739 or @aabmass suggested to add a page to the examples section. I think that makes sense. It already has some similar content such as https://opentelemetry-python.readthedocs.io/en/stable/examples/django/README.html

This doc lives in the core repo. We should move instrumentation exampes to contrib repo and add pages for gunicorn, uWSGI and any other frameworks that need special setup.

Existing examples: https://github.com/open-telemetry/opentelemetry-python/tree/master/docs

Some of these should be migrated to contrib.

[aabmass]

Batch span processor does not work well with Gunicorn's forking model out of the box. Users need to leverage gunicorns worker hooks to setup tracing for each worker after the worker has started. This should be documented with examples.

@owais, I remember we discussed this before, but how does this solution work with development servers (like manage.py runserver)? Do they need to duplicate the setup code OTel into manage.py?

[owais]

Yes, unfortunately it need to be called in a separate place for manage.py. Good thing is that it is just regular python code so it can be just a function import and call from both places

[owais]

BTW this is documented over at Splunk's Otel distribution of Python here: https://github.com/signalfx/splunk-otel-python#special-cases

Anyone wanting to take this one can refer to that as an example.

[srikanthccv]

Related issue open-telemetry/opentelemetry-python#1567

[srikanthccv]

I will pick this up and probably complete by this weekend unless someone else either working on this or want to work this.

[zionsofer]

Sorry for resurrecting, but a quick question - Is there any difference to having the app instrumentation happen in the post_fork gunicorn hook instead in the app itself?

I mean, instead of having two separate places that handle with OTEL code, like in the example we have -
app.py:

application = flask.Flask(__name__)

FlaskInstrumentor().instrument_app(application)

and in the gunicorn config:

def post_fork(server, worker):
    span_processor = BatchSpanProcessor(...)
    trace.get_tracer_provider().add_span_processor(span_processor)

Where we could just have, in the gunicorn config:

def post_fork(server, worker):
    FlaskInstrumentor().instrument_app(worker.app.callable)
    span_processor = BatchSpanProcessor(...)
    trace.get_tracer_provider().add_span_processor(span_processor)

[srikanthccv]

@zionsofer Yes, you can certainly do that.