Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Back-off restarting failed container opentelemetry-auto-instrumentation in pod #1236

Closed
dartagnanli opened this issue Nov 1, 2024 · 6 comments
Labels
bug Something isn't working

Comments

@dartagnanli
Copy link

Describe the bug

when i inject the auto-instrumentation container for my pod ,and then my pod is not ready ,

Normal   Scheduled  13m                   default-scheduler  Successfully assigned luban/openapi-86c5f46479-26lhw to gms-3
  Normal   Pulled     13m                   kubelet            Container image "xxxxxx/openapi:v1.4.4" already present on machine
  Normal   Created    13m                   kubelet            Created container openapi
  Normal   Started    13m                   kubelet            Started container openapi
  Normal   Pulled     12m (x5 over 13m)     kubelet            Container image "xxxxxx/autoinstrumentation-go:v0.16.0-alpha" already present on machine
  Normal   Created    12m (x5 over 13m)     kubelet            Created container opentelemetry-auto-instrumentation
  Normal   Started    12m (x5 over 13m)     kubelet            Started container opentelemetry-auto-instrumentation
  Warning  BackOff    3m44s (x46 over 13m)  kubelet            Back-off restarting failed container opentelemetry-auto-instrumentation in pod openapi-86c5f46479-26lhw_luban(be4c2d6b-723c-4263-a6b0-2fc09812e9d5)

but when i run it in host machine,like blew ,it is ok and will not be exited

sudo OTEL_GO_AUTO_TARGET_EXE=/home/xxx//openapi SERVICE_NAME=my_service OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 ./otel-go-instrumentation

my otel-go-instrumentation version is v0.15.0-alpha

Environment

@dartagnanli dartagnanli added the bug Something isn't working label Nov 1, 2024
@damemi
Copy link
Contributor

damemi commented Nov 1, 2024

Hi, could you please share the logs from the auto-instrumentation container?

@dartagnanli
Copy link
Author

this is no error log ,all is blew

{"time":"2024-11-01T08:42:52.238546198Z","level":"INFO","source":{"function":"main.main","file":"/app/cli/main.go","line":103},"msg":"building OpenTelemetry Go instrumentation ...","globalImpl":false,"version":{"Release":"v0.16.0-alpha","Revision":"unknown","Go":{"Version":"go1.23.2","OS":"linux","Arch":"amd64"}}}
{"time":"2024-11-01T08:42:54.239305134Z","level":"INFO","source":{"function":"go.opentelemetry.io/auto/internal/pkg/process.(*Analyzer).DiscoverProcessID","file":"/app/internal/pkg/process/discover.go","line":64},"msg":"found process","pid":7}

@MrAlias
Copy link
Contributor

MrAlias commented Nov 5, 2024

The lack of error message or a stacktrace from a panic/crash makes it seem like the pod is killing itself. Is there a healthcheck for the pod that is isn't being satisfied?

@RonFed
Copy link
Contributor

RonFed commented Nov 6, 2024

Might be related to #902

@dartagnanli
Copy link
Author

Might be related to #902

Similar!Is there any solution about this?

@dartagnanli
Copy link
Author

I raise my Instrumentation config go.resourceRequirements, it seem went well.Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants