From e2dea2f1aace5474faa56b8b8bfc543c1433ba5e Mon Sep 17 00:00:00 2001 From: Ricardo Ahumada <112411485+RicardoAAD@users.noreply.github.com> Date: Mon, 9 Oct 2023 10:10:13 -0300 Subject: [PATCH] Record accessor limitation (#1015) * administration/configuring-fluent-bit/classic-mode/record-accessor adding record accessos syntax limitation Signed-off-by: RicardoAAD * administration/configuring-fluent-bit/classic-mode/record-accessor fixed links Signed-off-by: RicardoAAD * administration:configuring-fluent-bit:classic-mode:record-accessor fixed links Signed-off-by: RicardoAAD * administration:configuring-fluent-bit:classic-mode:record-accessor fixed links Signed-off-by: RicardoAAD * administration:configuring-fluent-bit:classic-mode:record-accessor fixed links Signed-off-by: RicardoAAD * pipeline:filters:ecs-metadata git Changed references to the record_accessor library limitation Signed-off-by: RicardoAAD * pipeline:filters:ecs-metadata git Changed references to the record_accessor library limitation Signed-off-by: RicardoAAD * pipeline:filters:ecs-metadata git Changed references to the record_accessor library limitation typos Signed-off-by: RicardoAAD --------- Signed-off-by: RicardoAAD --- .../classic-mode/record-accessor.md | 20 ++++++++++++++++ pipeline/filters/ecs-metadata.md | 23 ++----------------- 2 files changed, 22 insertions(+), 21 deletions(-) diff --git a/administration/configuring-fluent-bit/classic-mode/record-accessor.md b/administration/configuring-fluent-bit/classic-mode/record-accessor.md index 81fffd20c..9b950efa8 100644 --- a/administration/configuring-fluent-bit/classic-mode/record-accessor.md +++ b/administration/configuring-fluent-bit/classic-mode/record-accessor.md @@ -95,3 +95,23 @@ Fluent Bit v1.x.x {"date":1599862267.483692,"log":"message 4","labels":{"color":"blue"}} ``` +### Limitations of record_accessor templating + +The Fluent Bit record_accessor library has a limitation in the characters that can separate template variables- only dots and commas (`.` and `,`) can come after a template variable. This is because the templating library must parse the template and determine the end of a variable. + +The following would be invalid templates because the two template variables are not separated by commas or dots: + +- `$TaskID-$ECSContainerName` +- `$TaskID/$ECSContainerName` +- `$TaskID_$ECSContainerName` +- `$TaskIDfooo$ECSContainerName` + +However, the following are valid: +- `$TaskID.$ECSContainerName` +- `$TaskID.ecs_resource.$ECSContainerName` +- `$TaskID.fooo.$ECSContainerName` + +And the following are valid since they only contain one template variable with nothing after it: +- `fooo$TaskID` +- `fooo____$TaskID` +- `fooo/bar$TaskID` diff --git a/pipeline/filters/ecs-metadata.md b/pipeline/filters/ecs-metadata.md index ce154755f..e2d6f567b 100644 --- a/pipeline/filters/ecs-metadata.md +++ b/pipeline/filters/ecs-metadata.md @@ -116,7 +116,8 @@ The output log would be similar to: } ``` -Notice that the template variables in the value for the `resource` key are separated by dot characters. Please see the section below about limitations in which characters can be used to separate template variables. +Notice that the template variables in the value for the `resource` key are separated by dot characters, only dots and commas + (`.` and `,`) can come after a template variable. For more information, please check the [Record accessor limitation's section](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md#limitations-of-record_accessor-templating). #### Example 3: Attach cluster metadata to non-container logs @@ -148,23 +149,3 @@ This examples shows a use case for the `Cluster_Metadata_Only` option- attaching Format json_lines ``` -### Limitations of record_accessor templating - -Notice in example 2, that the template values are separated by dot characters. This is important; the Fluent Bit record_accessor library has a limitation in the characters that can separate template variables- only dots and commas (`.` and `,`) can come after a template variable. This is because the templating library must parse the template and determine the end of a variable. - -The following would be invalid templates because the two template variables are not separated by commas or dots: - -- `$TaskID-$ECSContainerName` -- `$TaskID/$ECSContainerName` -- `$TaskID_$ECSContainerName` -- `$TaskIDfooo$ECSContainerName` - -However, the following are valid: -- `$TaskID.$ECSContainerName` -- `$TaskID.ecs_resource.$ECSContainerName` -- `$TaskID.fooo.$ECSContainerName` - -And the following are valid since they only contain one template variable with nothing after it: -- `fooo$TaskID` -- `fooo____$TaskID` -- `fooo/bar$TaskID`