Skip to content

Commit

Permalink
Fixed storage.total_limit_size misleading document content (#1244)
Browse files Browse the repository at this point in the history
* Fixed storage.total_limit_size misleading document content

Signed-off-by: ErMao <[email protected]>

* Fixed storage.total_limit_size misleading document content

Signed-off-by: ErMao <[email protected]>

---------

Signed-off-by: ErMao <[email protected]>
Co-authored-by: ErMao <[email protected]>
  • Loading branch information
opencmit2 and ErMao authored Dec 18, 2023
1 parent 01f5a7d commit 54b3c3d
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions administration/buffering-and-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Fluent Bit implements the concept of logical queues: based on its Tag, a Chunk c

It's common to find cases where if we have multiple destinations for a Chunk, one of the destinations might be slower than the other, or maybe one is generating backpressure and not all of them. In this scenario, how do we limit the amount of filesystem Chunks that we are logically queueing?

Starting from Fluent Bit v1.6, we introduced the new configuration property for output plugins called `storage.total_limit_size` which limits the number of Chunks that exist in the filesystem for a certain logical output destination. If one of the destinations reaches the `storage.total_limit_size`, the oldest Chunk from its queue for that logical output destination will be discarded.
Starting from Fluent Bit v1.6, we introduced the new configuration property for output plugins called `storage.total_limit_size` which limits the total size in bytes of chunks that can exist in the filesystem for a certain logical output destination. If one of the destinations reaches the configured `storage.total_limit_size`, the oldest Chunk from its queue for that logical output destination will be discarded to make room for new data.

## Configuration

Expand Down Expand Up @@ -167,7 +167,7 @@ If certain chunks are filesystem _storage.type_ based, it's possible to control

| Key | Description | Default |
| :--- | :--- | :--- |
| storage.total\_limit\_size | Limit the maximum number of Chunks in the filesystem for the current output logical destination. | |
| storage.total\_limit\_size | Limit the maximum disk space size in bytes for buffering chunks in the filesystem for the current output logical destination. | |

The following example create records with CPU usage samples in the filesystem and then they are delivered to Google Stackdriver service limiting the logical queue \(buffering\) to 5M:

Expand All @@ -191,5 +191,5 @@ The following example create records with CPU usage samples in the filesystem an
storage.total_limit_size 5M
```

If for some reason Fluent Bit gets offline because of a network issue, it will continue buffering CPU samples but just keep a maximum of 5M of the newest data.
If for some reason Fluent Bit gets offline because of a network issue, it will continue buffering CPU samples but just keep a maximum of 5MB of the newest data.

0 comments on commit 54b3c3d

Please sign in to comment.