kinesis_firehose: Crashing, log loss/duplication

[pranavmarla]

Opening here for visibility, as per @PettitWesley 's recommendation -- please see fluent/fluent-bit#3917 for details on the issue we're experiencing with the core kinesis_firehose plugin (written in C).

[PettitWesley]

For others who might see this issue, we have been working with @pranavmarla and have identified a number of bugs which seem to contribute to instability. These are in the process of making their way upstream. In the meantime, here is the branch and image based off of 1.7.5 that we are testing with and which seems to be very performant and stable:

Code is here: https://github.com/krispraws/fluent-bit/commits/v1_7_5_openssl_fix

Image is here: 144718711470.dkr.ecr.us-west-2.amazonaws.com/core-network-fixes:1.7.5

Pull it with:

ecs-cli pull --region us-west-2 --registry-id 144718711470 144718711470.dkr.ecr.us-west-2.amazonaws.com/core-network-fixes:1.7.5

[sparrc]

These appear to be fixed in fluentbit 1.9