in_tail: avoid double free for multiline msgpack buffer

[hossain-rayhan]

Signed-off-by: Rayhan Hossain [email protected]

We are double freeing &file->mult_sbuf for multiline flush and Fluent Bit is crasing. This happens when we use multiline parser and rotate the file.

The first free is happening at tail_multiline.c:491 and the next free is happening at tail_file.c:1024.

From my investigation, I think we can avoid it in two different ways.

1. Assign NULL after freeing the memory at tail_multiline.c:491 which I am doing in this PR. This way we won't have an issue if tail_file.c tries again to free that NULL assigned memory. I tested and it works.
2. Rmove the double fee from line tail_file.c:1024. I am not sure if it has any other use case. But this will also work and I have tested it.

Fixes #4177
Copy: aws/aws-for-fluent-bit#255

Valgrind Output:

==22561== HEAP SUMMARY:
==22561==     in use at exit: 99,848 bytes in 3,381 blocks
==22561==   total heap usage: 7,559 allocs, 4,178 frees, 7,455,312 bytes allocated
==22561== 
==22561== LEAK SUMMARY:
==22561==    definitely lost: 0 bytes in 0 blocks
==22561==    indirectly lost: 0 bytes in 0 blocks
==22561==      possibly lost: 0 bytes in 0 blocks
==22561==    still reachable: 99,848 bytes in 3,381 blocks
==22561==         suppressed: 0 bytes in 0 blocks
==22561== Rerun with --leak-check=full to see details of leaked memory
==22561== 
==22561== For counts of detected and suppressed errors, rerun with: -v
==22561== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• Debug log output from testing the change

• Attached Valgrind output that shows no leaks or memory corruption was found

Documentation

• [N/A] Documentation required for this feature

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[PettitWesley]

So by setting it to NULL, then later on line 1024, when msgpack_sbuffer_destroy is called, it won't free it again since calling free on NULL does nothing, and that function is just a wrapper around free: https://github.com/msgpack/msgpack-c/blob/c_master/include/msgpack/sbuffer.h#L39

[RalfWenzel]

@hossain-rayhan Just for clarification, you wrote "Fluent Bit is crashing". I'm wondering if your finding is related to the "[error] [multiline] invalid stream id , could not append content to multiline context" error which seams to plague quite a couple of installations (including us) when looking at forums.

fluent-bit is not crashing in this case, did'nt check code until now, but i guess exception is handled, msg written to stderr and fb continues.

So do you expect that your fix will solve this when included in upcoming release ?

[hossain-rayhan]

Hi @RalfWenzel , I don't think I saw [error] [multiline] invalid stream id , could not append content to multiline context. Our customer was forcefully rotating log files when FluentBit was running. It was thwroing double free error and aborted. Please check #4177 for details. It should solve 4177 but not sure if it has any connection with the error you mentioned.

[RalfWenzel]

Hi @RalfWenzel , I don't think I saw [error] [multiline] invalid stream id , could not append content to multiline context. Our customer was forcefully rotating log files when FluentBit was running. It was thwroing double free error and aborted. Please check #4177 for details. It should solve 4177 but not sure if it has any connection with the error you mentioned.

Thanks for explanation. I'll dig a little deeper.

[RalfWenzel]

follow up: issue related to message i mentioned above: #4190

[nokute78]

LGTM.

By the way, msgpack-c supports an API to set NULL.
How about it ?
https://github.com/fluent/fluent-bit/blob/v1.8.8/lib/msgpack-c/include/msgpack/sbuffer.h#L94

[PettitWesley]

@hossain-rayhan 1.8 PR?

[hossain-rayhan]

Hey sorry, I will cut this today.

[hossain-rayhan]

PR against 1.8: #4243