Fluent-bit 3.0.4 throwing repeated warnings ( [ warn] [record accessor] translation failed, root key=hec_token) #8859
Comments
|
I think there's a problem with this PR. We're getting spammed with the same message as the user above, millions of times per minute: |
|
We have same error |
|
Same error It started appearing right after we started using the latest helm chart 0.46.7 with app version 3.0.4 |
|
we are looking into this. |
The following patch perform 2 changes in the code that helps to fix the problems found with Splunk hec token handling: 1. In the recent PR #8793, when using the record accessor API flb_ra_translate_check() to validate if the hec_token field exists, leads to noisy log messages since that function warns the issue if the field is not found. Most of users are not using hec_token set by Splunk input plugin, so their logging gets noisy. This patch replaces that call with flb_ra_translate() which fixes the problem. 2. If hec_token was set in the record metadata, it was being store in the main context of the plugin, however the flush callbacks that formats and deliver the data runs in separate/parallel threads that could lead to a race condition if more than onen thread tries to manipulate the value. This patch adds protection to the context value so it becomes thread safe. Signed-off-by: Eduardo Silva <[email protected]>
|
Fix will be there shortly: #8864 |
|
We are facing security issue and need to upgrade to 3.0.4 asap. Getting the same error. |
|
With 3.0.5 getting below error. |
@nehjain17 What is your config? |
|
Output Config: [OUTPUT] |
|
output plugin
[OUTPUT]
Name splunk
Match index1
event_index index1
host splunk_host_name
Port 443
splunk_token ${TOKEN}
tls On
tls.verify Off
Retry_Limit 5
net.connect_timeout 30
net.keepalive on
…On Fri, May 24, 2024 at 5:07 AM Eduardo Silva ***@***.***> wrote:
With 3.0.5 getting below error. [2024/05/24 05:47:13] [ warn]
[output:splunk:splunk.16] http_status=401: {"text":"Token is
required","code":2}
@nehjain17 <https://github.com/nehjain17> What is your config?
—
Reply to this email directly, view it on GitHub
<#8859 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AETMRSSU7SK63B6MLRWJ3CTZD4URTAVCNFSM6AAAAABIDULTUGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMRZGM3TCMBXGA>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
The following patch perform 2 changes in the code that helps to fix the problems found with Splunk hec token handling: 1. In the recent PR fluent#8793, when using the record accessor API flb_ra_translate_check() to validate if the hec_token field exists, leads to noisy log messages since that function warns the issue if the field is not found. Most of users are not using hec_token set by Splunk input plugin, so their logging gets noisy. This patch replaces that call with flb_ra_translate() which fixes the problem. 2. If hec_token was set in the record metadata, it was being store in the main context of the plugin, however the flush callbacks that formats and deliver the data runs in separate/parallel threads that could lead to a race condition if more than onen thread tries to manipulate the value. This patch adds protection to the context value so it becomes thread safe. Signed-off-by: Eduardo Silva <[email protected]> Signed-off-by: Markus Bergholz <[email protected]>
Bug Report
Describe the bug
After upgrading fluent-bit from 3.0.3 to 3.0.4, we have noticed fluent-bit pods started throwing repeated warnings in the logs
To Reproduce
upgrade to 3.0.4 version of fluent-bit
The text was updated successfully, but these errors were encountered: