From 32321b18eef69907df54da33e015b9bf6f04c772 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Fri, 18 Oct 2024 09:58:45 -0700 Subject: [PATCH 1/5] Upating record-modifier for style Signed-off-by: Lynette Miles --- pipeline/filters/record-modifier.md | 89 +++++++++++++++++------------ 1 file changed, 51 insertions(+), 38 deletions(-) diff --git a/pipeline/filters/record-modifier.md b/pipeline/filters/record-modifier.md index 0a1f74572..7a739bd48 100644 --- a/pipeline/filters/record-modifier.md +++ b/pipeline/filters/record-modifier.md @@ -1,24 +1,27 @@ # Record Modifier -The _Record Modifier Filter_ plugin allows to append fields or to exclude specific fields. +The _Record Modifier_ filter plugin lets you append fields to a record, or exclude +specific fields. ## Configuration Parameters -The plugin supports the following configuration parameters: _Remove\_key_ and _Allowlist\_key_ are exclusive. +The plugin supports the following configuration parameters. `Remove_key` and +`Allowlist_key` are exclusive. | Key | Description | | :--- | :--- | -| Record | Append fields. This parameter needs key and value pair. | -| Remove\_key | If the key is matched, that field is removed. | -| Allowlist\_key | If the key is **not** matched, that field is removed. | -| Whitelist\_key | An alias of `Allowlist_key` for backwards compatibility. | -| Uuid\_key| If set, the plugin appends uuid to each record. The value assigned becomes the key in the map.| +| `Record` | Append fields. This parameter needs a key/value pair. | +| `Remove_key` | If the key is matched, that field is removed. | +| `Allowlist_key` | If the key isn't matched, that field is removed. | +| `Whitelist_key` | An alias of `Allowlist_key` for backwards compatibility. | +| `Uuid_key`| If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map.| ## Getting Started -In order to start filtering records, you can run the filter from the command line or through the configuration file. +To start filtering records, run the filter from the command line or through a +configuration file. -This is a sample in\_mem record to filter. +This is a sample `in_mem` record to filter. ```text {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724} @@ -26,11 +29,13 @@ This is a sample in\_mem record to filter. ### Append fields -The following configuration file is to append product name and hostname \(via environment variable\) to record. +The following configuration file appends a product name and hostname to a record +using an environment variable: {% tabs %} {% tab title="fluent-bit.conf" %} -```python + +```python copy [INPUT] Name mem Tag mem.local @@ -45,10 +50,12 @@ The following configuration file is to append product name and hostname \(via en Record hostname ${HOSTNAME} Record product Awesome_Tool ``` + {% endtab %} {% tab title="fluent-bit.yaml" %} -```yaml + +```yaml copy pipeline: inputs: - name: mem @@ -56,37 +63,37 @@ pipeline: filters: - name: record_modifier match: '*' - record: + record: - hostname ${HOSTNAME} - product Awesome_Tool outputs: - name: stdout match: '*' ``` + {% endtab %} {% endtabs %} +You can run the filter from command line: -You can also run the filter from command line. - -```text -$ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*' +```shell copy +fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*' ``` -The output will be +The output looks something like: -```python +```python copy [0] mem.local: [1492436882.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724, "hostname"=>"localhost.localdomain", "product"=>"Awesome_Tool"}] ``` -### Remove fields with Remove\_key - -The following configuration file is to remove 'Swap.\*' fields. +### Remove fields with `Remove_key` +The following configuration file removes `Swap.*` fields: {% tabs %} {% tab title="fluent-bit.conf" %} -```python + +```python copy [INPUT] Name mem Tag mem.local @@ -102,10 +109,12 @@ The following configuration file is to remove 'Swap.\*' fields. Remove_key Swap.used Remove_key Swap.free ``` + {% endtab %} {% tab title="fluent-bit.yaml" %} -```yaml + +```yaml copy pipeline: inputs: - name: mem @@ -113,7 +122,7 @@ pipeline: filters: - name: record_modifier match: '*' - remove_key: + remove_key: - Swap.total - Swap.used - Swap.free @@ -121,28 +130,30 @@ pipeline: - name: stdout match: '*' ``` + {% endtab %} {% endtabs %} You can also run the filter from command line. -```text -$ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*' +```shell copy +fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*' ``` -The output will be +The output looks something like: ```python [0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}] ``` -### Remove fields with Allowlist\_key +### Remove fields with `Allowlist_key` -The following configuration file is to remain 'Mem.\*' fields. +The following configuration file retains `Mem.*` fields. {% tabs %} {% tab title="fluent-bit.conf" %} -```python + +```python copy [INPUT] Name mem Tag mem.local @@ -158,10 +169,12 @@ The following configuration file is to remain 'Mem.\*' fields. Allowlist_key Mem.used Allowlist_key Mem.free ``` + {% endtab %} {% tab title="fluent-bit.yaml" %} -```yaml + +```yaml copy pipeline: inputs: - name: mem @@ -169,7 +182,7 @@ pipeline: filters: - name: record_modifier match: '*' - Allowlist_key: + Allowlist_key: - Mem.total - Mem.used - Mem.free @@ -177,18 +190,18 @@ pipeline: - name: stdout match: '*' ``` + {% endtab %} {% endtabs %} -You can also run the filter from command line. +You can also run the filter from command line: -```text -$ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*' +```shell copy +fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*' ``` -The output will be +The output looks something like: ```python [0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}] ``` - From 694538ad9595af6edba48c06431fe5eac932dd12 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Fri, 18 Oct 2024 10:09:06 -0700 Subject: [PATCH 2/5] Testing escapes Signed-off-by: Lynette Miles --- pipeline/filters/record-modifier.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pipeline/filters/record-modifier.md b/pipeline/filters/record-modifier.md index 7a739bd48..8451b6454 100644 --- a/pipeline/filters/record-modifier.md +++ b/pipeline/filters/record-modifier.md @@ -11,10 +11,10 @@ The plugin supports the following configuration parameters. `Remove_key` and | Key | Description | | :--- | :--- | | `Record` | Append fields. This parameter needs a key/value pair. | -| `Remove_key` | If the key is matched, that field is removed. | -| `Allowlist_key` | If the key isn't matched, that field is removed. | -| `Whitelist_key` | An alias of `Allowlist_key` for backwards compatibility. | -| `Uuid_key`| If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map.| +| `Remove\_key` | If the key is matched, that field is removed. | +| `Allowlist\_key` | If the key isn't matched, that field is removed. | +| `Whitelist\_key` | An alias of `Allowlist_key` for backwards compatibility. | +| `Uuid\_key`| If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map.| ## Getting Started From b64a39c6e787050cea991c43acbbc640c3139772 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Fri, 18 Oct 2024 10:22:51 -0700 Subject: [PATCH 3/5] docs: testing formatting Signed-off-by: Lynette Miles --- pipeline/filters/record-modifier.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/pipeline/filters/record-modifier.md b/pipeline/filters/record-modifier.md index 8451b6454..5c448ad4e 100644 --- a/pipeline/filters/record-modifier.md +++ b/pipeline/filters/record-modifier.md @@ -11,10 +11,10 @@ The plugin supports the following configuration parameters. `Remove_key` and | Key | Description | | :--- | :--- | | `Record` | Append fields. This parameter needs a key/value pair. | -| `Remove\_key` | If the key is matched, that field is removed. | -| `Allowlist\_key` | If the key isn't matched, that field is removed. | -| `Whitelist\_key` | An alias of `Allowlist_key` for backwards compatibility. | -| `Uuid\_key`| If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map.| +| `Remove_key` | If the key is matched, that field is removed. | +| `Allowlist_key` | If the key isn't matched, that field is removed. | +| `Whitelist_key` | An alias of `Allowlist_key` for backwards compatibility. | +| `Uuid_key` | If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map.| ## Getting Started From ee2e074232249c706c6151adf05212376c2c4dfe Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Fri, 18 Oct 2024 11:03:36 -0700 Subject: [PATCH 4/5] docs: minor corretions Signed-off-by: Lynette Miles --- pipeline/filters/record-modifier.md | 17 ++++++++--------- 1 file changed, 8 insertions(+), 9 deletions(-) diff --git a/pipeline/filters/record-modifier.md b/pipeline/filters/record-modifier.md index 5c448ad4e..220d1b8f3 100644 --- a/pipeline/filters/record-modifier.md +++ b/pipeline/filters/record-modifier.md @@ -1,22 +1,21 @@ # Record Modifier -The _Record Modifier_ filter plugin lets you append fields to a record, or exclude -specific fields. +The _Record Modifier_ [filter](pipeline/filters.md) plugin lets you append +fields to a record, or exclude specific fields. -## Configuration Parameters +## Configuration parameters -The plugin supports the following configuration parameters. `Remove_key` and -`Allowlist_key` are exclusive. +The plugin supports the following configuration parameters: | Key | Description | | :--- | :--- | | `Record` | Append fields. This parameter needs a key/value pair. | -| `Remove_key` | If the key is matched, that field is removed. | -| `Allowlist_key` | If the key isn't matched, that field is removed. | +| `Remove_key` | If the key is matched, that field is removed. You can this or `Allowlist_key`.| +| `Allowlist_key` | If the key isn't matched, that field is removed. You can this or `Remove_key`. | | `Whitelist_key` | An alias of `Allowlist_key` for backwards compatibility. | -| `Uuid_key` | If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map.| +| `Uuid_key` | If set, the plugin appends Uuid to each record. The value assigned becomes the key in the map. | -## Getting Started +## Get started To start filtering records, run the filter from the command line or through a configuration file. From 182b122a74dc63491c2b970cc3dc1553887e4503 Mon Sep 17 00:00:00 2001 From: Lynette Miles Date: Fri, 18 Oct 2024 11:17:23 -0700 Subject: [PATCH 5/5] pipeline: docs: fixing a header Signed-off-by: Lynette Miles (lynette.miles@chronosphere.io) --- pipeline/filters/record-modifier.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pipeline/filters/record-modifier.md b/pipeline/filters/record-modifier.md index 220d1b8f3..dc64aea7c 100644 --- a/pipeline/filters/record-modifier.md +++ b/pipeline/filters/record-modifier.md @@ -145,7 +145,7 @@ The output looks something like: [0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}] ``` -### Remove fields with `Allowlist_key` +### Retain fields with `Allowlist_key` The following configuration file retains `Mem.*` fields.