From 4f34882fe863c88b87d5431b2a0d2a5f6e4a2c73 Mon Sep 17 00:00:00 2001 From: ErMao Date: Mon, 23 Oct 2023 15:07:47 +0800 Subject: [PATCH 1/2] Fixed storage.total_limit_size misleading document content Signed-off-by: ErMao --- administration/buffering-and-storage.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/administration/buffering-and-storage.md b/administration/buffering-and-storage.md index d5153e917..951a9af32 100644 --- a/administration/buffering-and-storage.md +++ b/administration/buffering-and-storage.md @@ -91,7 +91,7 @@ Fluent Bit implements the concept of logical queues: based on its Tag, a Chunk c It's common to find cases where if we have multiple destinations for a Chunk, one of the destinations might be slower than the other, or maybe one is generating backpressure and not all of them. In this scenario, how do we limit the amount of filesystem Chunks that we are logically queueing? -Starting from Fluent Bit v1.6, we introduced the new configuration property for output plugins called `storage.total_limit_size` which limits the number of Chunks that exist in the filesystem for a certain logical output destination. If one of the destinations reaches the `storage.total_limit_size`, the oldest Chunk from its queue for that logical output destination will be discarded. +Starting from Fluent Bit v1.6, we introduced the new configuration property for output plugins called `storage.total_limit_size` which limits the total size in bytes of chunks that can exist in the filesystem for a certain logical output destination. If one of the destinations reaches the configured `storage.total_limit_size`, the oldest Chunk from its queue for that logical output destination will be discarded to make room for new data. ## Configuration @@ -167,7 +167,7 @@ If certain chunks are filesystem _storage.type_ based, it's possible to control | Key | Description | Default | | :--- | :--- | :--- | -| storage.total\_limit\_size | Limit the maximum number of Chunks in the filesystem for the current output logical destination. | | +| storage.total\_limit\_size | Limit the maximum disk space size in bytes for buffering chunks in the filesystem for the current output logical destination. | | The following example create records with CPU usage samples in the filesystem and then they are delivered to Google Stackdriver service limiting the logical queue \(buffering\) to 5M: @@ -191,5 +191,5 @@ The following example create records with CPU usage samples in the filesystem an storage.total_limit_size 5M ``` -If for some reason Fluent Bit gets offline because of a network issue, it will continue buffering CPU samples but just keep a maximum of 5M of the newest data. +If for some reason Fluent Bit gets offline because of a network issue, it will continue buffering CPU samples but just keep a maximum of 5M bytes of the newest data. From f207293b984b97ad2812a17bcdff86ee6fe9930e Mon Sep 17 00:00:00 2001 From: ErMao Date: Wed, 25 Oct 2023 10:05:45 +0800 Subject: [PATCH 2/2] Fixed storage.total_limit_size misleading document content Signed-off-by: ErMao --- administration/buffering-and-storage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/administration/buffering-and-storage.md b/administration/buffering-and-storage.md index 951a9af32..bd5142737 100644 --- a/administration/buffering-and-storage.md +++ b/administration/buffering-and-storage.md @@ -191,5 +191,5 @@ The following example create records with CPU usage samples in the filesystem an storage.total_limit_size 5M ``` -If for some reason Fluent Bit gets offline because of a network issue, it will continue buffering CPU samples but just keep a maximum of 5M bytes of the newest data. +If for some reason Fluent Bit gets offline because of a network issue, it will continue buffering CPU samples but just keep a maximum of 5MB of the newest data.