Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_kinesis_streams: integrate with shared compression lib #9200

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

reegnz
Copy link

@reegnz reegnz commented Aug 13, 2024

Support for gzip compression. Basically copied the approach used in out_kinesis_firehose.


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
  • Debug log output from testing the change
  • Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

Basically copied the approach used in out_kinesis_firehose.

Signed-off-by: Zoltán Reegn <[email protected]>
@reegnz reegnz force-pushed the feat/kinesis_streams_compression branch from 1bc52f1 to 2155872 Compare August 13, 2024 17:03
@reegnz
Copy link
Author

reegnz commented Aug 13, 2024

I basically copied what got implemented in out_kinesis_firehose, but I'm sort of a C noob, so I'd appreciate some help in testing (can probably do it myself given enough time but other people are more savvy with this stuff than me).

@reegnz
Copy link
Author

reegnz commented Aug 13, 2024

I want to migrate away from fluentd where this feature was present, and it's a pretty big blocker for me right now.

Original PR was #4371 where I'm backporting the compression implementation from.

@PettitWesley maybe I can get you to help out, given that you were the original implementor of the out_kinesis_stream feature?

@reegnz
Copy link
Author

reegnz commented Oct 1, 2024

Opened a ticket that covers this issue: #9451

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant