filter_parser: fix timestamp and metadata to be properly copied

[chitoku-k]

This patch fixes parser filters where they have dropped timestamp and metadata if the configured parser does not process a log (when out_buf is NULL) since #7133.

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

• Example configuration file for the change
• Debug log output from testing the change
• Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• [N/A] Documentation required for this feature

Backporting

• Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[chitoku-k]

Config

[SERVICE]
    Parsers_File    parsers.conf

[INPUT]
    Name            systemd
    Systemd_Filter  _SYSTEMD_UNIT=systemd-logind.service
    Read_From_Tail  On

[OUTPUT]
    Name            stdout
    Match           *

[FILTER]
    Name            parser
    Match           *
    Key_Name        message
    Parser          json

[PARSER]
    Name            json
    Format          json

Log

$ bin/fluent-bit -c config
Fluent Bit v2.1.3
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/05/01 04:29:43] [ info] [fluent bit] version=2.1.3, commit=43682ad1f8, pid=151037
[2023/05/01 04:29:43] [ info] [storage] ver=1.1.6, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/05/01 04:29:43] [ info] [cmetrics] version=0.6.1
[2023/05/01 04:29:43] [ info] [ctraces ] version=0.3.0
[2023/05/01 04:29:43] [ info] [input:systemd:systemd.0] initializing
[2023/05/01 04:29:43] [ info] [input:systemd:systemd.0] storage_strategy='memory' (memory only)
[2023/05/01 04:29:43] [ info] [sp] stream processor started
[2023/05/01 04:29:43] [ info] [output:stdout:stdout.0] worker #0 started
[0] systemd.0: [[1682882986.440271000, {}], {"priority"=>"6", "_uid"=>"0", "_gid"=>"0", "_systemd_slice"=>"system.slice", "_boot_id"=>"4c5b7a833e334b5fa506650deace56d3", "_machine_id"=>"09cb1f91a4af4686bc1b57cad2710c32", "_hostname"=>"lychee", "_runtime_scope"=>"system", "_transport"=>"journal", "syslog_facility"=>"4", "tid"=>"425", "code_file"=>"src/login/logind-session.c", "code_line"=>"779", "code_func"=>"session_start", "syslog_identifier"=>"systemd-logind", "message_id"=>"8d45620c1a4348dbb17410da57c60c66", "user_id"=>"chitoku", "_pid"=>"425", "_comm"=>"systemd-logind", "_exe"=>"/usr/lib/systemd/systemd-logind", "_cmdline"=>"/usr/lib/systemd/systemd-logind", "_cap_effective"=>"24420020f", "_systemd_cgroup"=>"/system.slice/systemd-logind.service", "_systemd_unit"=>"systemd-logind.service", "_systemd_invocation_id"=>"fecc3bd3786943678a33036e3878ea3e", "session_id"=>"37", "leader"=>"151060", "message"=>"New session 37 of user chitoku.", "_source_realtime_timestamp"=>"1682882986440244"}]
[0] systemd.0: [[1682882989.646445000, {}], {"priority"=>"6", "_uid"=>"0", "_gid"=>"0", "_systemd_slice"=>"system.slice", "_boot_id"=>"4c5b7a833e334b5fa506650deace56d3", "_machine_id"=>"09cb1f91a4af4686bc1b57cad2710c32", "_hostname"=>"lychee", "_runtime_scope"=>"system", "_transport"=>"journal", "syslog_facility"=>"4", "tid"=>"425", "code_file"=>"src/login/logind-session.c", "syslog_identifier"=>"systemd-logind", "user_id"=>"chitoku", "_pid"=>"425", "_comm"=>"systemd-logind", "_exe"=>"/usr/lib/systemd/systemd-logind", "_cmdline"=>"/usr/lib/systemd/systemd-logind", "_cap_effective"=>"24420020f", "_systemd_cgroup"=>"/system.slice/systemd-logind.service", "_systemd_unit"=>"systemd-logind.service", "_systemd_invocation_id"=>"fecc3bd3786943678a33036e3878ea3e", "code_line"=>"850", "code_func"=>"session_stop_scope", "session_id"=>"37", "leader"=>"151060", "message"=>"Session 37 logged out. Waiting for processes to exit.", "_source_realtime_timestamp"=>"1682882989646423"}]
[1] systemd.0: [[1682882989.646822000, {}], {"priority"=>"6", "_uid"=>"0", "_gid"=>"0", "_systemd_slice"=>"system.slice", "_boot_id"=>"4c5b7a833e334b5fa506650deace56d3", "_machine_id"=>"09cb1f91a4af4686bc1b57cad2710c32", "_hostname"=>"lychee", "_runtime_scope"=>"system", "_transport"=>"journal", "syslog_facility"=>"4", "tid"=>"425", "code_file"=>"src/login/logind-session.c", "syslog_identifier"=>"systemd-logind", "user_id"=>"chitoku", "_pid"=>"425", "_comm"=>"systemd-logind", "_exe"=>"/usr/lib/systemd/systemd-logind", "_cmdline"=>"/usr/lib/systemd/systemd-logind", "_cap_effective"=>"24420020f", "_systemd_cgroup"=>"/system.slice/systemd-logind.service", "_systemd_unit"=>"systemd-logind.service", "_systemd_invocation_id"=>"fecc3bd3786943678a33036e3878ea3e", "code_line"=>"907", "code_func"=>"session_finalize", "message_id"=>"3354939424b4456d9802ca8333ed424a", "session_id"=>"37", "leader"=>"151060", "message"=>"Removed session 37.", "_source_realtime_timestamp"=>"1682882989646809"}]
^C[2023/05/01 04:29:53] [engine] caught signal (SIGINT)
[2023/05/01 04:29:53] [ warn] [engine] service will shutdown in max 5 seconds
[2023/05/01 04:29:53] [ info] [input] pausing systemd.0
[2023/05/01 04:29:53] [ info] [engine] service has stopped (0 pending tasks)
[2023/05/01 04:29:53] [ info] [input] pausing systemd.0
[2023/05/01 04:29:53] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/05/01 04:29:53] [ info] [output:stdout:stdout.0] thread worker #0 stopped

Valgrind output

$ valgrind --leak-check=full bin/fluent-bit -c config
==150803== Memcheck, a memory error detector
==150803== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==150803== Using Valgrind-3.20.0 and LibVEX; rerun with -h for copyright info
==150803== Command: bin/fluent-bit -c config
==150803==
Fluent Bit v2.1.3
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/05/01 04:29:41] [ info] [fluent bit] version=2.1.3, commit=43682ad1f8, pid=150803
[2023/05/01 04:29:41] [ info] [storage] ver=1.1.6, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/05/01 04:29:41] [ info] [cmetrics] version=0.6.1
[2023/05/01 04:29:41] [ info] [ctraces ] version=0.3.0
[2023/05/01 04:29:41] [ info] [input:systemd:systemd.0] initializing
[2023/05/01 04:29:41] [ info] [input:systemd:systemd.0] storage_strategy='memory' (memory only)
[2023/05/01 04:29:41] [ info] [sp] stream processor started
[2023/05/01 04:29:41] [ info] [output:stdout:stdout.0] worker #0 started
[0] systemd.0: [[1682882986.440271000, {}], {"priority"=>"6", "_uid"=>"0", "_gid"=>"0", "_systemd_slice"=>"system.slice", "_boot_id"=>"4c5b7a833e334b5fa506650deace56d3", "_machine_id"=>"09cb1f91a4af4686bc1b57cad2710c32", "_hostname"=>"lychee", "_runtime_scope"=>"system", "_transport"=>"journal", "syslog_facility"=>"4", "tid"=>"425", "code_file"=>"src/login/logind-session.c", "code_line"=>"779", "code_func"=>"session_start", "syslog_identifier"=>"systemd-logind", "message_id"=>"8d45620c1a4348dbb17410da57c60c66", "user_id"=>"chitoku", "_pid"=>"425", "_comm"=>"systemd-logind", "_exe"=>"/usr/lib/systemd/systemd-logind", "_cmdline"=>"/usr/lib/systemd/systemd-logind", "_cap_effective"=>"24420020f", "_systemd_cgroup"=>"/system.slice/systemd-logind.service", "_systemd_unit"=>"systemd-logind.service", "_systemd_invocation_id"=>"fecc3bd3786943678a33036e3878ea3e", "session_id"=>"37", "leader"=>"151060", "message"=>"New session 37 of user chitoku.", "_source_realtime_timestamp"=>"1682882986440244"}]
[0] systemd.0: [[1682882989.646445000, {}], {"priority"=>"6", "_uid"=>"0", "_gid"=>"0", "_systemd_slice"=>"system.slice", "_boot_id"=>"4c5b7a833e334b5fa506650deace56d3", "_machine_id"=>"09cb1f91a4af4686bc1b57cad2710c32", "_hostname"=>"lychee", "_runtime_scope"=>"system", "_transport"=>"journal", "syslog_facility"=>"4", "tid"=>"425", "code_file"=>"src/login/logind-session.c", "syslog_identifier"=>"systemd-logind", "user_id"=>"chitoku", "_pid"=>"425", "_comm"=>"systemd-logind", "_exe"=>"/usr/lib/systemd/systemd-logind", "_cmdline"=>"/usr/lib/systemd/systemd-logind", "_cap_effective"=>"24420020f", "_systemd_cgroup"=>"/system.slice/systemd-logind.service", "_systemd_unit"=>"systemd-logind.service", "_systemd_invocation_id"=>"fecc3bd3786943678a33036e3878ea3e", "code_line"=>"850", "code_func"=>"session_stop_scope", "session_id"=>"37", "leader"=>"151060", "message"=>"Session 37 logged out. Waiting for processes to exit.", "_source_realtime_timestamp"=>"1682882989646423"}]
[1] systemd.0: [[1682882989.646822000, {}], {"priority"=>"6", "_uid"=>"0", "_gid"=>"0", "_systemd_slice"=>"system.slice", "_boot_id"=>"4c5b7a833e334b5fa506650deace56d3", "_machine_id"=>"09cb1f91a4af4686bc1b57cad2710c32", "_hostname"=>"lychee", "_runtime_scope"=>"system", "_transport"=>"journal", "syslog_facility"=>"4", "tid"=>"425", "code_file"=>"src/login/logind-session.c", "syslog_identifier"=>"systemd-logind", "user_id"=>"chitoku", "_pid"=>"425", "_comm"=>"systemd-logind", "_exe"=>"/usr/lib/systemd/systemd-logind", "_cmdline"=>"/usr/lib/systemd/systemd-logind", "_cap_effective"=>"24420020f", "_systemd_cgroup"=>"/system.slice/systemd-logind.service", "_systemd_unit"=>"systemd-logind.service", "_systemd_invocation_id"=>"fecc3bd3786943678a33036e3878ea3e", "code_line"=>"907", "code_func"=>"session_finalize", "message_id"=>"3354939424b4456d9802ca8333ed424a", "session_id"=>"37", "leader"=>"151060", "message"=>"Removed session 37.", "_source_realtime_timestamp"=>"1682882989646809"}]
^C[2023/05/01 04:29:54] [engine] caught signal (SIGINT)
[2023/05/01 04:29:54] [ warn] [engine] service will shutdown in max 5 seconds
[2023/05/01 04:29:54] [ info] [input] pausing systemd.0
[2023/05/01 04:29:54] [ info] [engine] service has stopped (0 pending tasks)
[2023/05/01 04:29:54] [ info] [input] pausing systemd.0
[2023/05/01 04:29:54] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/05/01 04:29:54] [ info] [output:stdout:stdout.0] thread worker #0 stopped
==150803==
==150803== HEAP SUMMARY:
==150803==     in use at exit: 0 bytes in 0 blocks
==150803==   total heap usage: 2,620 allocs, 2,620 frees, 1,985,937 bytes allocated
==150803==
==150803== All heap blocks were freed -- no leaks are possible
==150803==
==150803== For lists of detected and suppressed errors, rerun with: -s
==150803== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

[leonardo-albertovich]

Hi @chitoku-k, I had made another comment explaining a different approach to the issue that was based on what I actually missed in the code when I wrote it but after taking a second look your patch is totally valid so I'll accept it as is.

Thank you very much for the contribution!