From 20981f57739712ec52c14bd613561dd789259499 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Tue, 29 Oct 2024 09:04:44 -0700 Subject: [PATCH] config: file: Updating configuraton file for style Signed-off-by: Lynette Miles --- .../classic-mode/configuration-file.md | 189 ++++++++++-------- vale-styles/FluentBit/Spelling-exceptions.txt | 2 + 2 files changed, 105 insertions(+), 86 deletions(-) diff --git a/administration/configuring-fluent-bit/classic-mode/configuration-file.md b/administration/configuring-fluent-bit/classic-mode/configuration-file.md index 34c8cbfc4..23790df86 100644 --- a/administration/configuring-fluent-bit/classic-mode/configuration-file.md +++ b/administration/configuring-fluent-bit/classic-mode/configuration-file.md @@ -1,49 +1,51 @@ --- -description: This page describes the main configuration file used by Fluent Bit +description: This page describes the main configuration file used by Fluent Bit. --- -# Configuration File +# Configuration file -One of the ways to configure Fluent Bit is using a main configuration file. Fluent Bit allows to use one configuration file which works at a global scope and uses the [Format and Schema](format-schema.md) defined previously. - -The main configuration file supports four types of sections: - -* Service -* Input -* Filter -* Output - -In addition, it's also possible to split the main configuration file in multiple files using the feature to include external files: - -* Include File - -## Service - -The _Service_ section defines global properties of the service, the keys available as of this version are described in the following table: - -| Key | Description | Default Value | -| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -| flush | Set the flush time in `seconds.nanoseconds`. The engine loop uses a Flush timeout to define when is required to flush the records ingested by input plugins through the defined output plugins. | 1 | -| grace | Set the grace time in `seconds` as Integer value. The engine loop uses a Grace timeout to define wait time on exit | 5 | -| daemon | Boolean value to set if Fluent Bit should run as a Daemon (background) or not. Allowed values are: yes, no, on and off. note: If you are using a Systemd based unit as the one we provide in our packages, do not turn on this option. | Off | -| dns.mode | Set the primary transport layer protocol used by the asynchronous DNS resolver which can be overridden on a per plugin basis | UDP | -| log_file | Absolute path for an optional log file. By default all logs are redirected to the standard error interface (stderr). | | -| log_level | Set the logging verbosity level. Allowed values are: off, error, warn, info, debug and trace. Values are accumulative, e.g: if 'debug' is set, it will include error, warning, info and debug. Note that _trace_ mode is only available if Fluent Bit was built with the _WITH\_TRACE_ option enabled. | info | -| parsers_file | Path for a `parsers` configuration file. Multiple Parsers_File entries can be defined within the section. | | -| plugins_file | Path for a `plugins` configuration file. A _plugins_ configuration file allows to define paths for external plugins, for an example [see here](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | | -| streams_file | Path for the Stream Processor configuration file. To learn more about Stream Processing configuration go [here](../../../stream-processing/introduction.md). | | -| http_server | Enable built-in HTTP Server | Off | -| http_listen | Set listening interface for HTTP Server when it's enabled | 0.0.0.0 | -| http_port | Set TCP Port for the HTTP Server | 2020 | -| coro_stack_size | Set the coroutines stack size in bytes. The value must be greater than the page size of the running system. Don't set too small value (say 4096), or coroutine threads can overrun the stack buffer. Do not change the default value of this parameter unless you know what you are doing. | 24576 | -| scheduler.cap | Set a maximum retry time in second. The property is supported from v1.8.7. | 2000 | -| scheduler.base | Set a base of exponential backoff. The property is supported from v1.8.7. | 5 | -| json.convert_nan_to_null | If enabled, NaN is converted to null when fluent-bit converts msgpack to json. | false | -| sp.convert_from_str_to_num | If enabled, Stream processor converts from number string to number type. | true | - -The following is an example of a _SERVICE_ section: +One of the ways to configure Fluent Bit is using a main configuration file. Fluent +Bit allows the use one configuration file which works at a global scope and uses the +defined [Format and Schema](format-schema.md). + +The main configuration file supports four sections: + +- Service +- Input +- Filter +- Output + +It's also possible to split the main configuration file into multiple files using +the Include File feature to include external files. + +## Service + +The `Service` section defines global properties of the service. The following keys +are: + +| Key | Description | Default Value | +| --------------- | ------------- | ------------- | +| `flush` | Set the flush time in `seconds.nanoseconds`. The engine loop uses a Flush timeout to define when it's required to flush the records ingested by input plugins through the defined output plugins. | `1` | +| `grace` | Set the grace time in `seconds` as an integer value. The engine loop uses a Grace timeout to define wait time on exit. | `5` | +| daemon | Boolean. Set if Fluent Bit should run as a Daemon (background) or not. Allowed values are: `yes`, `no`, `on`, and `off`. When using a Systemd based unit as the one provided in FLuent Bit packages, don't turn on this option.  | `Off` | +| `dns.mode` | Set the primary transport layer protocol used by the asynchronous DNS resolver. Can be overridden on a per plugin basis. | `UDP` | +| `log_file` | Absolute path for an optional log file. By default all logs are redirected to the standard error interface (stderr). | _none_ | +| `log_level` | Set the logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. If 'debug' is set, it will include `error`, `warning`, `info`, and `debug`. Trace mode is only available if Fluent Bit was built with the _`WITH_TRACE`_ option enabled. | `info` | +| `parsers_file` | Path for a `parsers` configuration file. Multiple `Parsers_File` entries can be defined within the section. | _none_ | +| `plugins_file` | Path for a `plugins` configuration file. A `plugins` configuration file defines paths for external plugins. [See an example](https://github.com/fluent/fluent-bit/blob/master/conf/plugins.conf). | _none_ | +| `streams_file` | Path for the Stream Processor configuration file. [Learn more about Stream Processing configuration](../../../stream-processing/introduction.md). | _none_| +| `http_server` | Enable the built-in HTTP Server | `Off` | +| `http_listen` | Set listening interface for HTTP Server when it's enabled | `0.0.0.0` | +| `http_port` | Set TCP Port for the HTTP Server. | `2020` | +| `coro_stack_size` | Set the coroutines stack size in bytes. The value must be greater than the page size of the running system. Setting the value too small (`4096`) can cause coroutine threads to overrun the stack buffer. The default value of this parameter shouldn't be changed. | `24576` | +| `scheduler.cap` | Set a maximum retry time in seconds. Supported in v1.8.7 and greater. | `2000` | +| `scheduler.base` | Set a base of exponential backoff. Supported in v1.8.7 and greater. | `5` | +| `json.convert_nan_to_null` | If enabled, `NaN` converts to `null` when Fluent Bit converts `msgpack` to `json`. | `false` | +| `sp.convert_from_str_to_num` | If enabled, Stream processor converts from number string to number type. | `true` | + +The following is an example of a `SERVICE` section: ```python [SERVICE] @@ -51,23 +53,28 @@ The following is an example of a _SERVICE_ section: Daemon off Log_Level debug ``` -For scheduler and retry details, please check there: [scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries) -## Input +For scheduler and retry details, see [scheduling and retries](../../scheduling-and-retries.md#Scheduling-and-Retries). + +## Config input -An _INPUT_ section defines a source (related to an input plugin), here we will describe the base configuration for each _INPUT_ section. Note that each input plugin may add it own configuration keys: +The `INPUT` section defines a source (related to an input plugin). Each +[input plugin](https://docs.fluentbit.io/manual/pipeline/inputs) can add its own +configuration keys: -| Key | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Name | Name of the input plugin. | -| Tag | Tag name associated to all records coming from this plugin. | -| Log_Level | Set the plugin's logging verbosity level. Allowed values are: off, error, warn, info, debug and trace. Defaults to the _SERVICE_ section's _Log_Level._ | +| Key | Description | +| ----------- | ------------| +| `Name` | Name of the input plugin. | +| `Tag` | Tag name associated to all records coming from this plugin. | +| `Log_Level` | Set the plugin's logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Defaults to the `SERVICE` section's `Log_Level`. | -The _Name_ is mandatory and it let Fluent Bit know which input plugin should be loaded. The _Tag_ is mandatory for all plugins except for the _input forward_ plugin (as it provides dynamic tags). +`Name` is mandatory and tells Fluent Bit which input plugin to load. `Tag` is +mandatory for all plugins except for the `input forward` plugin, which provides +dynamic tags. ### Example -The following is an example of an _INPUT_ section: +The following is an example of an `INPUT` section: ```python [INPUT] @@ -75,22 +82,26 @@ The following is an example of an _INPUT_ section: Tag my_cpu ``` -## Filter +## Config filter -A _FILTER_ section defines a filter (related to an filter plugin), here we will describe the base configuration for each _FILTER_ section. Note that each filter plugin may add it own configuration keys: +The `FILTER` section defines a filter (related to an filter plugin). Each filter +plugin can add it own configuration keys. The base configuration for each +`FILTER` section contains: -| Key | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Name | Name of the filter plugin. | -| Match | A pattern to match against the tags of incoming records. It's case sensitive and support the star (\*) character as a wildcard. | -| Match_Regex | A regular expression to match against the tags of incoming records. Use this option if you want to use the full regex syntax. | -| Log_Level | Set the plugin's logging verbosity level. Allowed values are: off, error, warn, info, debug and trace. Defaults to the _SERVICE_ section's _Log_Level._ | +| Key | Description | +| ----------- | ------------ | +| `Name` | Name of the filter plugin. | +| `Match` | A pattern to match against the tags of incoming records. Case sensitive, supports asterisk (`*`) as a wildcard. | +| `Match_Regex` | A regular expression to match against the tags of incoming records. Use this option if you want to use the full regular expression syntax. | +| `Log_Level` | Set the plugin's logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Defaults to the `SERVICE` section's `Log_Level`. | -The _Name_ is mandatory and it let Fluent Bit know which filter plugin should be loaded. The _Match_ or _Match_Regex_ is mandatory for all plugins. If both are specified, _Match_Regex_ takes precedence. +`Name` is mandatory and lets Fluent Bit know which filter plugin should be loaded. +`Match` or `Match_Regex` is mandatory for all plugins. If both are specified, +`Match_Regex` takes precedence. -### Example +### Filter example -The following is an example of an _FILTER_ section: +The following is an example of a `FILTER` section: ```python [FILTER] @@ -99,20 +110,22 @@ The following is an example of an _FILTER_ section: Regex log aa ``` -## Output +## Config output -The _OUTPUT_ section specify a destination that certain records should follow after a Tag match. Currently, Fluent Bit can route up to 256 _OUTPUT_ plugins. The configuration support the following keys: +The `OUTPUT` section specifies a destination that certain records should go to +after a `Tag` match. Fluent Bit can route up to 256 `OUTPUT` plugins. The +configuration supports the following keys: -| Key | Description | -| ----------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Name | Name of the output plugin. | -| Match | A pattern to match against the tags of incoming records. It's case sensitive and support the star (\*) character as a wildcard. | -| Match_Regex | A regular expression to match against the tags of incoming records. Use this option if you want to use the full regex syntax. | -| Log_Level | Set the plugin's logging verbosity level. Allowed values are: off, error, warn, info, debug and trace. Defaults to the _SERVICE_ section's _Log_Level._ | +| Key | Description | +| ----------- | -------------- | +| `Name` | Name of the output plugin. | +| `Match` | A pattern to match against the tags of incoming records. Case sensitive and supports the asterisk (`*`) character as a wildcard. | +| `Match_Regex` | A regular expression to match against the tags of incoming records. Use this option if you want to use the full regular expression syntax. | +| `Log_Level` | Set the plugin's logging verbosity level. Allowed values are: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Defaults to the `SERVICE` section's `Log_Level`. | -### Example +### Output example -The following is an example of an _OUTPUT_ section: +The following is an example of an `OUTPUT` section: ```python [OUTPUT] @@ -122,7 +135,8 @@ The following is an example of an _OUTPUT_ section: ### Example: collecting CPU metrics -The following configuration file example demonstrates how to collect CPU metrics and flush the results every five seconds to the standard output: +The following configuration file example demonstrates how to collect CPU metrics and +flush the results every five seconds to the standard output: ```python [SERVICE] @@ -139,29 +153,32 @@ The following configuration file example demonstrates how to collect CPU metrics Match my*cpu ``` -## Include File +## Config Include File -To avoid complicated long configuration files is better to split specific parts in different files and call them (include) from one main file. +To avoid complicated long configuration files is better to split specific parts in +different files and call them (include) from one main file. The `@INCLUDE`can be used +in the following way: -Starting from Fluent Bit 0.12 the new configuration command _@INCLUDE_ has been added and can be used in the following way: - -``` +```text @INCLUDE somefile.conf ``` -The configuration reader will try to open the path _somefile.conf_, if not found, it will assume it's a relative path based on the path of the base configuration file, e.g: +The configuration reader will try to open the path `somefile.conf`. If not found, the +reader assumes the file is on a relative path based on the path of the base +configuration file: -* Main configuration file path: /tmp/main.conf -* Included file: somefile.conf -* Fluent Bit will try to open somefile.conf, if it fails it will try /tmp/somefile.conf. +- Main configuration path: `/tmp/main.conf` +- Included file: `somefile.conf` +- Fluent Bit will try to open `somefile.conf`, if it fails it will try `/tmp/somefile.conf`. -The _@INCLUDE_ command only works at top-left level of the configuration line, it cannot be used inside sections. +The `@INCLUDE` command only works at top-left level of the configuration line, and +can't be used inside sections. -Wildcard character (\*) is supported to include multiple files, e.g: +Wildcard character (`*`) supports including multiple files. For example: -``` +```text @INCLUDE input_*.conf ``` -Note files matching the wildcard character are included unsorted. -If plugins ordering between files need to be preserved, the files should be included explicitly. +Files matching the wildcard character are included unsorted. If plugin ordering +between files needs to be preserved, the files should be included explicitly. diff --git a/vale-styles/FluentBit/Spelling-exceptions.txt b/vale-styles/FluentBit/Spelling-exceptions.txt index 18b032117..f2da97b32 100644 --- a/vale-styles/FluentBit/Spelling-exceptions.txt +++ b/vale-styles/FluentBit/Spelling-exceptions.txt @@ -19,6 +19,8 @@ clickstreams CloudWatch Config Coralogix +coroutine +coroutines Crowdstrike CRDs DaemonSet