Merge branch 'master' into loki_plain_string

Signed-off-by: Andrew Titmuss <[email protected]>

.gitbook.yaml

@@ -57,6 +57,7 @@ redirects:
     output/cloudwatch:     ./pipeline/outputs/cloudwatch.md
     output/datadog:        ./pipeline/outputs/datadog.md
     output/es:             ./pipeline/outputs/elasticsearch.md
+    output/fabric:         ./pipeline/outputs/azure_kusto.md
     output/file:           ./pipeline/outputs/file.md
     output/firehose:       ./pipeline/outputs/firehose.md
     output/flowcounter:    ./pipeline/outputs/flowcounter.md

.gitignore

@@ -2,4 +2,4 @@ _book/*
 node_modules/*
 .gitignore
 old/*
-
+.DS_Store

README.md

@@ -2,11 +2,15 @@
 description: High Performance Telemetry Agent for Logs, Metrics and Traces
 ---
 
-# Fluent Bit v3.0 Documentation
+# Fluent Bit v3.1 Documentation
 
-<figure><img src=".gitbook/assets/3.0.png" alt=""><figcaption></figcaption></figure>
+<figure><img src=".gitbook/assets/3.1.png" alt=""><figcaption></figcaption></figure>
 
-[Fluent Bit](http://fluentbit.io) is a Fast and Lightweight **Telemetry Agent** for Logs, Metrics, and Traces for Linux, macOS, Windows, and BSD family operating systems. It has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
+[Fluent Bit](http://fluentbit.io) is a fast and lightweight **telemetry agent**
+for logs, metrics, and traces for Linux, macOS, Windows, and BSD family
+operating systems. It has been made with a strong focus on performance to allow
+the collection and processing of telemetry data from different sources without
+complexity.![](https://static.scarf.sh/a.png?x-pxid=71f0e011-761f-4c6f-9a89-38817887faae)
 
 ## Features
 
@@ -37,4 +41,7 @@ description: High Performance Telemetry Agent for Logs, Metrics and Traces
 
 [Fluent Bit](http://fluentbit.io) is a [CNCF](https://cncf.io) **graduated** sub-project under the umbrella of [Fluentd](http://fluentd.org). Fluent Bit is licensed under the terms of the [Apache License v2.0](http://www.apache.org/licenses/LICENSE-2.0).
 
-Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/). As a CNCF-hosted project, it is a fully **vendor-neutral** and community-driven project.
+Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/)
+and is now sponsored by [Chronosphere](https://chronosphere.io/). As a
+CNCF-hosted project, it is a fully **vendor-neutral** and community-driven
+project.

SUMMARY.md

@@ -1,6 +1,6 @@
 # Table of contents
 
-* [Fluent Bit v3.0 Documentation](README.md)
+* [Fluent Bit v3.1 Documentation](README.md)
 
 ## About
 
@@ -67,6 +67,7 @@
 * [Networking](administration/networking.md)
 * [Memory Management](administration/memory-management.md)
 * [Monitoring](administration/monitoring.md)
+* [Multithreading](administration/multithreading.md)
 * [HTTP Proxy](administration/http-proxy.md)
 * [Hot Reload](administration/hot-reload.md)
 * [Troubleshooting](administration/troubleshooting.md)
@@ -106,6 +107,7 @@
   * [Process Log Based Metrics](pipeline/inputs/process.md)
   * [Process Exporter Metrics](pipeline/inputs/process-exporter-metrics.md)
   * [Prometheus Scrape Metrics](pipeline/inputs/prometheus-scrape-metrics.md)
+  * [Prometheus Remote Write](pipeline/inputs/prometheus-remote-write.md)
   * [Random](pipeline/inputs/random.md)
   * [Serial Interface](pipeline/inputs/serial-interface.md)
   * [Splunk](pipeline/inputs/splunk.md)
@@ -130,7 +132,9 @@
   * [Decoders](pipeline/parsers/decoders.md)
 * [Processors](pipeline/processors/README.md)
   * [Content Modifier](pipeline/processors/content-modifier.md)
+  * [Labels](pipeline/processors/labels.md)
   * [Metrics Selector](pipeline/processors/metrics-selector.md)
+  * [OpenTelemetry Envelope](pipeline/processors/opentelemetry-envelope.md)
   * [SQL](pipeline/processors/sql.md)
 * [Filters](pipeline/filters/README.md)
   * [AWS Metadata](pipeline/filters/aws-metadata.md)
@@ -179,6 +183,7 @@
   * [Kafka REST Proxy](pipeline/outputs/kafka-rest-proxy.md)
   * [LogDNA](pipeline/outputs/logdna.md)
   * [Loki](pipeline/outputs/loki.md)
+  * [Microsoft Fabric](pipeline/outputs/azure\_kusto.md)
   * [NATS](pipeline/outputs/nats.md)
   * [New Relic](pipeline/outputs/new-relic.md)
   * [NULL](pipeline/outputs/null.md)

administration/backpressure.md

@@ -1,5 +1,7 @@
 # Backpressure
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=63e37cfe-9ce3-4a18-933a-76b9198958c1" />
+
 Under certain scenarios it is possible for logs or data to be ingested or created faster than the ability to flush it to some destinations. One such common scenario is when reading from big log files, especially with a large backlog, and dispatching the logs to a backend over the network, which takes time to respond. This generates backpressure leading to high memory consumption in the service.
 
 In order to avoid backpressure, Fluent Bit implements a mechanism in the engine that restricts the amount of data that an input plugin can ingest, this is done through the configuration parameters **Mem\_Buf\_Limit** and **storage.Max\_Chunks\_Up**.
@@ -44,7 +46,7 @@ The setting behaves similarly to the above scenario with `Mem_Buf_Limit` when th
 
 When (default) `storage.pause_on_chunks_overlimit` is disabled, the input will not pause when the memory limit is reached. Instead, it will switch to only buffering logs in the filesystem. The disk spaced used for filesystem buffering can be limited with `storage.total_limit_size`.
 
-Please consule the [Buffering & Storage](buffering-and-storage.md) docs for more information.
+See the [Buffering & Storage](buffering-and-storage.md) docs for more information.
 
 ## About pause and resume Callbacks
 

administration/buffering-and-storage.md

@@ -1,5 +1,7 @@
 # Buffering & Storage
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=cde12327-09ed-409c-ac02-7c0afa5eff51" />
+
 The end-goal of [Fluent Bit](https://fluentbit.io) is to collect, parse, filter and ship logs to a central place. In this workflow there are many phases and one of the critical pieces is the ability to do _buffering_ : a mechanism to place processed data into a temporary location until is ready to be shipped.
 
 By default when Fluent Bit processes data, it uses Memory as a primary and temporary place to store the records, but there are certain scenarios where it would be ideal to have a persistent buffering mechanism based in the filesystem to provide aggregation and data safety capabilities.

administration/configuring-fluent-bit/classic-mode/configuration-file.md

@@ -4,6 +4,8 @@ description: This page describes the main configuration file used by Fluent Bit
 
 # Configuration File
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5e67142e-3887-4b56-b940-18494bcc23a7" />
+
 One of the ways to configure Fluent Bit is using a main configuration file. Fluent Bit allows to use one configuration file which works at a global scope and uses the [Format and Schema](format-schema.md) defined previously.
 
 The main configuration file supports four types of sections:
@@ -23,7 +25,7 @@ The _Service_ section defines global properties of the service, the keys availab
 
 | Key             | Description                                                                                                                                                                                                                                                                                             | Default Value |
 | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
-| flush           | Set the flush time in `seconds.nanoseconds`. The engine loop uses a Flush timeout to define when is required to flush the records ingested by input plugins through the defined output plugins.                                                                                                         | 5             |
+| flush           | Set the flush time in `seconds.nanoseconds`. The engine loop uses a Flush timeout to define when is required to flush the records ingested by input plugins through the defined output plugins.                                                                                                         | 1             |
 | grace           | Set the grace time in `seconds` as Integer value. The engine loop uses a Grace timeout to define wait time on exit                                                                                                                                                                                      | 5             |
 | daemon          | Boolean value to set if Fluent Bit should run as a Daemon (background) or not. Allowed values are: yes, no, on and off.  note: If you are using a Systemd based unit as the one we provide in our packages, do not turn on this option.                                                                 | Off           |
 | dns.mode        | Set the primary transport layer protocol used by the asynchronous DNS resolver which can be overridden on a per plugin basis                                                                                                                                                                             | UDP           |

administration/configuring-fluent-bit/classic-mode/variables.md

@@ -1,5 +1,7 @@
 # Variables
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=1731c7b5-34c6-424f-bfc6-88c2aa71e81f" />
+
 Fluent Bit supports the usage of environment variables in any value associated to a key when using a configuration file.
 
 The variables are case sensitive and can be used in the following format:

administration/configuring-fluent-bit/multiline-parsing.md

@@ -2,6 +2,8 @@
 
 In an ideal world, applications might log their messages within a single line, but in reality applications generate multiple log messages that sometimes belong to the same context. But when is time to process such information it gets really complex. Consider application stack traces which always have multiple log lines.
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=e19a4c14-a9e4-4163-8f3a-52196eb9a585" />
+
 Starting from Fluent Bit v1.8, we have implemented a unified Multiline core functionality to solve all the user corner cases. In this section, you will learn about the features and configuration options available.
 
 ## Concepts

administration/configuring-fluent-bit/yaml/configuration-file.md

@@ -2,6 +2,8 @@
 description: This page describes the yaml configuration file used by Fluent Bit
 ---
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=864c6f0e-8977-4838-8772-84416943548e" />
+
 # YAML Configuration File
 
 One of the ways to configure Fluent Bit is using a YAML configuration file that works at a global scope.

administration/memory-management.md

@@ -1,5 +1,7 @@
 # Memory Management
 
+<img referrerpolicy="no-referrer-when-downgrade" src="https://static.scarf.sh/a.png?x-pxid=5cc3ce54-e910-4ebf-85f5-f02530b3e11b" />
+
 In certain scenarios it would be ideal to estimate how much memory Fluent Bit could be using, this is very useful for containerized environments where memory limits are a must.
 
 In order to that we will assume that the input plugins have set the **Mem\_Buf\_Limit** option \(you can learn more about it in the [Backpressure](backpressure.md) section\).