Skip to content

Releases: aws/aws-for-fluent-bit

2.31.12.20230727 Linux re-build

28 Jul 05:50
@PettitWesley PettitWesley
v2.31.12.20230727
83548a6
Compare
Choose a tag to compare
2.31.12.20230727 Linux re-build

2.31.12.20230727 Linux re-build

This release has the same Fluent Bit contents as 2.31.12, and is simply a linux-only re-build for recent patches in dependencies installed in the image. There are no windows images for this release.

Assets 2
Loading

2.31.12.20230629 Linux re-build

30 Jun 02:21
@PettitWesley PettitWesley
v2.31.12.20230629
dfb7a7f
Compare
Choose a tag to compare
2.31.12.20230629 Linux re-build

This release has the same Fluent Bit contents as 2.31.12, and is simply a linux-only re-build for recent patches in dependencies installed in the image. There are no windows images for this release. See our versioning FAQ.

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
kinesis_streams stdstream Log Loss ✅ ✅ 0%(347)
Log Duplication 0%(1000) ✅ 0%(65400)
kinesis_streams tcp Log Loss ✅ ✅ ✅
Log Duplication 0%(500) 0%(2970) 0%(8467)
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
kinesis_firehose tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(13000) 0%(7000)
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ 0%(5403) ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.12

19 Jun 19:54
@PettitWesley PettitWesley
v2.31.12
69366b1
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.12

2.31.12

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.4
  • Amazon Kinesis Streams for Fluent Bit 1.10.2
  • Amazon Kinesis Firehose for Fluent Bit 1.7.2

Compared to 2.31.11 this release adds:

  • Bug - Fix incorrect decrementing of s3_key_format $INDEX on Multipart Upload failure aws-for-fluent-bit:653
  • Bug - cloudwatch go plugin: fix utf-8 calculation of payload length to account for invalid unicode bytes that will be replaced with the 3 byte unicode replacement character. This bug can lead to an InvalidParameterException from CloudWatch when the payload sent is calculated to be over the limit due to character replacement. cloudwatch:330
  • Bug - fix SIGSEGV crash issue in exec input due to memory initialization aws-for-fluent-bit:661
  • Bug - record_accessor/rewrite_tag fix to allow single character rules fluent-bit:7330
  • Bug - filter_modify: fix memory clean up that can lead to crash fluent-bit:7368
  • Bug - http input: fix memory initialization issue and enable on Windows fluent-bit:7008
  • Enhancement - this version also includes a number of patches from upstream 2.x releases and library upgrades which are not associated with any end-user impacting issue but should improve stability.

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(5500) ✅
kinesis_firehose tcp Log Loss ✅ 0%(5557) ✅
Log Duplication ✅ 0%(4000) 0%(6000)
kinesis_streams stdstream Log Loss ✅ ✅ 0%(10462)
Log Duplication ✅ ✅ 0%(117769)
kinesis_streams tcp Log Loss ✅ ✅ ✅
Log Duplication 0%(1000) ✅ 0%(9915)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.11

16 May 17:38
@PettitWesley PettitWesley
v2.31.11
6c4b6c4
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.11

2.31.11

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.3
  • Amazon Kinesis Streams for Fluent Bit 1.10.2
  • Amazon Kinesis Firehose for Fluent Bit 1.7.2

Compared to 2.31.10 this release adds:

  • Bug - Improve parsing of STS AssumeRole response fluent-bit:7313
  • Bug - Fix potential null dereference in configuration parsing fluent-bit:6874
  • Bug - ElasticSearch output: fix potential bulk buffer over-run fluent-bit:5770
  • Bug - Fix parsing of time zone offsets on Windows fluent-bit:6368
  • Bug - Fix memory cleanup of failed retries fluent-bit:6862
  • Bug - Fix printf format string in flb_time_pop_from_mpack warning fluent-bit:7262
  • Enhancement - TCP Input: user friendly warning message when records are skipped fluent-bit:6061

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(4500) 0%(1000)
kinesis_firehose tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(7500) 0%(5500)
kinesis_streams stdstream Log Loss ✅ ✅ 0%(5462)
Log Duplication 0%(1000) 0%(19597) 0%(123943)
kinesis_streams tcp Log Loss ✅ ✅ 0%(4237)
Log Duplication 0%(20278) 0%(43773) 0%(111891)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ 1%(21569) 16%(297769)
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ 16%(200244) ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.10

26 Apr 18:35
@matthewfala matthewfala
v2.31.10
63d670f
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.10

2.31.10

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.3
  • Amazon Kinesis Streams for Fluent Bit 1.10.2
  • Amazon Kinesis Firehose for Fluent Bit 1.7.2

Compared to 2.31.9 this release adds:

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(5500) ✅
kinesis_firehose tcp Log Loss ✅ ✅ 0%(17906)
Log Duplication ✅ ✅ 0%(33500)
kinesis_streams stdstream Log Loss ✅ ✅ 0%(4735)
Log Duplication ✅ 0%(5640) 0%(71809)
kinesis_streams tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ 0%(26632)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ 19%(346317)
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.9

18 Apr 19:50
@matthewfala matthewfala
v2.31.9
6586a09
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.9

2.31.9

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.3
  • Amazon Kinesis Streams for Fluent Bit 1.10.2
  • Amazon Kinesis Firehose for Fluent Bit 1.7.2

Compared to 2.31.8 this release adds:

  • Enhancement - Add clear info message when chunks are removed because storage.total_limit_size is reached fluent-bit:6719
  • Bug - Fix S3 ARN parsing in init image that prevents it from being used in US Gov Cloud and China partitions aws-for-fluent-bit:617
  • Bug - Fix SIGSEGV on shutdown when multiple instances of the same go plugin are configured aws-for-fluent-bit:613
  • Bug - Fix off by one error that can lead to SDS string truncation fluent-bit:7143
  • Bug - fix minor memory leak in cloudwatch_logs that leads no more than ~1KB of un-freed memory when the log_stream_name option is configured.
  • Bug - Fix SIGSEGV on shutdown when multiple instances of the same go plugin are configured aws-for-fluent-bit:613

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(3000) 0%(500)
kinesis_firehose tcp Log Loss ✅ ✅ 0%(9623)
Log Duplication ✅ ✅ 0%(16500)
kinesis_streams stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ 0%(21162)
kinesis_streams tcp Log Loss ✅ ✅ 0%(5850)
Log Duplication ✅ ✅ 0%(62574)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ 1%(15080) ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.8

12 Apr 23:33
@matthewfala matthewfala
v2.31.8
1c6c5da
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.8

2.31.8

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.3
  • Amazon Kinesis Streams for Fluent Bit 1.10.2
  • Amazon Kinesis Firehose for Fluent Bit 1.7.2

Compared to 2.31.7 this release adds:

This release is a CVE patch release for 2.31.7. It contains the same contents but re-built to uptake the most recent Amazon Linux packages with patches.

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(3000) 0%(14511)
kinesis_firehose tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ 0%(5000)
kinesis_streams stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(23170) 0%(41590)
kinesis_streams tcp Log Loss ✅ ✅ 0%(28)
Log Duplication ✅ ✅ 0%(37752)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.28.5

31 Mar 02:41
@PettitWesley PettitWesley
v2.28.5
7913d6e
Compare
Choose a tag to compare
AWS for Fluent Bit 2.28.5

2.28.5

This release includes:

  • Fluent Bit 1.9.9
  • Amazon CloudWatch Logs for Fluent Bit 1.9.1
  • Amazon Kinesis Streams for Fluent Bit 1.10.1
  • Amazon Kinesis Firehose for Fluent Bit 1.7.1

This release is a CVE patch release for 2.28.4. It contains the same contents but re-built to uptake the most recent Amazon Linux packages with patches.

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication 0%(2500) 0%(2000) ✅
kinesis_firehose tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(2500) 0%(11000)
kinesis_streams stdstream Log Loss ✅ 0%(9) 0%(1922)
Log Duplication ✅ 0%(18536) 0%(62308)
kinesis_streams tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ 0%(28273)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.7

15 Mar 05:30
@PettitWesley PettitWesley
v2.31.7
30483a6
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.7

2.31.7

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.2
  • Amazon Kinesis Streams for Fluent Bit 1.10.1
  • Amazon Kinesis Firehose for Fluent Bit 1.7.1

Compared to 2.23.6 this release adds:

This release includes the same fixes and features that we are working on getting accepted upstream as AWS for Fluent Bit 2.31.6.

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ 0%(4695) ✅
Log Duplication ✅ 0%(6000) ✅
kinesis_firehose tcp Log Loss ✅ ✅ 0%(672)
Log Duplication ✅ 0%(4500) 0%(26896)
kinesis_streams stdstream Log Loss ✅ ✅ ✅
Log Duplication 0%(500) 0%(7490) 0%(1986)
kinesis_streams tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ 0%(25960)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ 8%(102321) 2%(49719)
Log Duplication ✅ ✅ ✅

Note:

  • The green check ✅ in the table means no log loss or no log duplication.
  • Number in parentheses means the number of records out of total records. For example, 0%(1064/1.8M) under 30Mb/s throughput means 1064 duplicate records out of 18M input records by which log duplication percentage is 0%.
  • Log loss is the percentage of data lost and log duplication is the percentage of duplicate logs received at the destination. Your results may differ because they can be influenced by many factors like different configs and environment settings. Log duplication is exclusively caused by partially succeeded batches that were retried, which means it is random.
Assets 2
Loading

AWS for Fluent Bit 2.31.6

08 Mar 03:17
@PettitWesley PettitWesley
v2.31.6
0a836ec
Compare
Choose a tag to compare
AWS for Fluent Bit 2.31.6

2.31.6

This release includes:

  • Fluent Bit 1.9.10
  • Amazon CloudWatch Logs for Fluent Bit 1.9.2
  • Amazon Kinesis Streams for Fluent Bit 1.10.1
  • Amazon Kinesis Firehose for Fluent Bit 1.7.1

Compared to 2.23.5 this release adds:

  • Bug - Fixed Log Loss can occur when log group creation or retention policy API calls fail cloudwatch:314

This release includes the same fixes and features that we are working on getting accepted upstream as AWS for Fluent Bit 2.31.5.

plugin source 20 MB/s 25 MB/s 30 MB/s
kinesis_firehose stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(6000) ✅
kinesis_firehose tcp Log Loss ✅ ✅ ✅
Log Duplication 0%(11000) 0%(5500) 0%(2500)
kinesis_streams stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ 0%(3969) 0%(7601)
kinesis_streams tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ 0%(29274)
s3 stdstream Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
s3 tcp Log Loss ✅ ✅ ✅
Log Duplication ✅ ✅ ✅
plugin source 1 MB/s 2 MB/s 3 MB/s
cloudwatch_logs stdstream Log Loss ✅ ✅ 16%(299626)
Log Duplication ✅ ✅ ✅
cloudwatch_logs tcp Log Loss ✅ ✅ 16%(300137)
Log Duplication ✅ ✅ ✅
Assets 2
Loading