-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
otel collector container keeps crashing due to nil pointer dereference when using fluentforwardreceiver #27469
Comments
Pinging code owners:
See Adding Labels via Comments if you do not have permissions to add labels yourself. |
Setting this as a P1 for now since panics should be protected against in components |
Looking at the code here: This seems to be the same root cause as this bug: Calling LogRecordCount on a plog that was previously passed to a ConsumeLogs call. Fix: call LogRecordCount before ConsumeLogs and cache the value for logging after. It is unsafe to call LogRecordCount after ConsumeLogs. |
Thank's for following up @ms-jcorley! I added the comment then thought "I should look at the code first before I claim they are linked". I appreciate you doing the due diligence for me with a bit more investigation. |
No problem! Also note that a quick (unreliable) search shows instances of this same pattern in several other places: |
We should consider opening up a larger meta issue track all of these once we confirm that this pattern will lead to a panic. |
I agree that this appears to be due to a race condition. I've opened #29274 with the quick fix of calling |
Component(s)
receiver/fluentforward
What happened?
Description
The otel collector container running as a side car in ECS fargate to capture logs from app container that uses aws firelens log
driver is crashing with nil pointer de-reference error, as shown below
Steps to Reproduce
Expected Result
Expected to work without panic error
Actual Result
Container is crashing
Collector version
0.79.0, 0.85.0
Environment information
Environment
OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
Log output
Additional context
No response
The text was updated successfully, but these errors were encountered: