-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Segmentation fault during shutdown. #613
Comments
Why run bash command inside the image? Please explain more. Thanks! |
You dont need bash. At first I had created the fluent-bit.conf file in heredoc using the reproduction procedure, but when I rewrote it, I must have forgotten to delete bash. Sorry for the distruction. So the same thing happens with the following procedure:
|
@m15o How quickly does the crash happen? Can you share example logs that you sent? What's the input in this case? I don't see an input... |
When I run this myself it does not immediately crash... |
You don't need to feed any input to reproduce. I tried to reproduce my personal Apple Silicon Mac, the behavior is slightly different from a x86 machine. Here is my log:
|
As I expected, this problem is caused by the destroy callback running twice, once for each
Each time it runs it frees the loaded Go output plugin... so the second time it runs there is a crash. Need to figure out why it runs twice... I checked that the go plugin seems to only be registered once... |
I think I've found a solution, it will take some time to test and clean up: https://github.com/PettitWesley/fluent-bit/commits/go-exit-debug-1_9 When it comes to plugins, there are two concepts:
Basically, on shut down there are two separate steps:
The current code basically mixes these two together and performs them in one function. |
Looks like this was already fixed in master, I should've checked that first: fluent/fluent-bit#6469 |
This will be resolved soon with 2.31.9 |
Describe the question/issue
Multiple output configuration cause SIGSEGV during shutdown.
Configuration
Fluent Bit Log Output
Fluent Bit Version Info
I tried
v2.31.7
public.ecr.aws/aws-observability/aws-for-fluent-bit:2.31.7
Fluent Bit v1.9.10
Cluster Details
It can be reproduced with docker run
Application Details
Steps to reproduce issue
kill -TERM <pid>
orCTRL-C
Related Issues
The text was updated successfully, but these errors were encountered: