Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python auto-instrumentation returns failed traces #3770

Closed
muandane opened this issue Mar 11, 2024 · 2 comments
Closed

Python auto-instrumentation returns failed traces #3770

muandane opened this issue Mar 11, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@muandane
Copy link

Describe your environment I'm running python applications that use Pika, tensorflow and pytorch. in a kubernetes cluster, with operator version 0.94.0 and python-auto-instrumentation version 0.44b

Steps to reproduce

Just creating this instrumentation and adding the annotation to the python deployment results in the errors bellow

---
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: python-traces
  namespace: caas-apps
spec:
  exporter:
    endpoint: http://caas-collector.caas-infra.svc.cluster.local:4318
  propagators:
    - tracecontext
    - baggage
    - b3
  sampler:
    type: parentbased_traceidratio
    argument: '1'
  python:
    env:
      - name: OTEL_PYTHON_DISABLED_INSTRUMENTATIONS
        value: "urllib3,urllib,flask"

What is the expected behavior?

Not getting the failed "connection refused" errors

What is the actual behavior?

getting multiple traces of failing connection establishment

image

trace-error

image

Additional context

i'm running other language instrumentations using different instrumentation definitions for java and nodejs and both of them work flawlessly, but the python one gives the previously show errors, i tried setting stuff like:

OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: http://python-collector.caas-infra.svc.cluster.local:4318/v1/traces
OTEL_EXPORTER_OTLP_ENDPOINT: http://python-collector.caas-infra.svc.cluster.local:4318
OTEL_PYTHON_DISABLED_INSTRUMENTATIONS: "urllib3,urllib,flask,django,kafka,redis"
@muandane muandane added the bug Something isn't working label Mar 11, 2024
@xrmx
Copy link
Contributor

xrmx commented Mar 13, 2024

The failing connections are coming from requests instrumentation, you may want to disable that too.

@muandane
Copy link
Author

The failing connections are coming from requests instrumentation, you may want to disable that too.

Thank you that fixed it @xrmx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants