AWS for Fluent Bit 2.23.4

Changelog

2.23.4

This release includes:

• An Amazon Linux 2 Base
• Fluent Bit 1.8.15
• Amazon CloudWatch Logs for Fluent Bit 1.7.0
• Amazon Kinesis Streams for Fluent Bit 1.9.0
• Amazon Kinesis Firehose for Fluent Bit 1.6.1

Compared to 2.23.3 this release adds:

• Go version upgrade to 1.17.9

Same as 2.23.3, this release includes the following fix for AWS customers that we are working on getting accepted upstream:

• Bug - Resolve IMDSv1 fallback error introduced in 2.21.0 aws-for-fluent-bit:259

Important Note:

• A security vulnerability was found in amazonlinux which we use as our base image. This new image will be based on an updated version of amazonlinux that resolves this CVE.

---

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load.

plugin		20 MB/s	25 MB/s	30 MB/s
kinesis_firehose	Log Loss	✅	✅	✅
	Log Duplication	✅	0%(841)	0%(500)
kinesis_streams	Log Loss	0% (3113)	0% (6127)	0% (12780)
	Log Duplication	0.1%(12936)	0% (1339)	0% (491)
s3	Log Loss	✅	✅	✅
	Log Duplication	✅	✅	✅

plugin		1 MB/s	2 MB/s	3 MB/s
cloudwatch_logs	Log Loss	✅	0.1%(1339)	0%(968)
	Log Duplication	✅	5%(63441)	4%(73511)

Note:

• The green check ✅ in the table means no log loss or no log duplication.
• Number in parentheses means the number of records out of total records. For example, 0%(1064) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
• CloudWatch has own throughput limit for single log stream. Based on our tests, it starts to appear throttling issue after input load > 1Mb/s.
• Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.