Input tail cannot work with Input exec?

[zhu733756]

Bug Report

Describe the bug

With the configuration below(location at ./conf/local/fluentbitconf):

[SERVICE]
    Flush        1
    Daemon       Off
    Log_Level    debug
    Parsers_File /etc/fluent-bit/parsers.conf 
    HTTP_Server  on
    HTTP_Listen  0.0.0.0
    HTTP_Port    2020
    
[INPUT]
    Name          tail
    Tag           json
    Path          ./static/*.json
    Path_Key       filename
    DB            ./static/json.db
    Parser         docker

[INPUT]
    Name          exec
    Tag           bin
    Command       ls ./static/*.bin
    Interval_Sec  2
    Interval_NSec 0

[OUTPUT]
    Name null
    Match *

With the cmd /opt/fluent-bit/bin/fluent-bit --config ./conf/local/fluentbitconf, the outputs are what described as below:

2022/07/14 12:36:15] [ info] Configuration:
[2022/07/14 12:36:15] [ info]  flush time     | 1.000000 seconds
[2022/07/14 12:36:15] [ info]  grace          | 5 seconds
[2022/07/14 12:36:15] [ info]  daemon         | 0
[2022/07/14 12:36:15] [ info] ___________
[2022/07/14 12:36:15] [ info]  inputs:
[2022/07/14 12:36:15] [ info]      tail
[2022/07/14 12:36:15] [ info]      exec
[2022/07/14 12:36:15] [ info] ___________
[2022/07/14 12:36:15] [ info]  filters:
[2022/07/14 12:36:15] [ info] ___________
[2022/07/14 12:36:15] [ info]  outputs:
[2022/07/14 12:36:15] [ info]      null.0
[2022/07/14 12:36:15] [ info] ___________
[2022/07/14 12:36:15] [ info]  collectors:
[2022/07/14 12:36:15] [ info] [fluent bit] version=1.9.5, commit=, pid=54220
[2022/07/14 12:36:15] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2022/07/14 12:36:15] [ info] [storage] version=1.2.0, type=memory-only, sync=normal, checksum=disabled, max_chunks_up=128
[2022/07/14 12:36:15] [ info] [cmetrics] version=0.3.4
[2022/07/14 12:36:15] [debug] [tail:tail.0] created event channels: read=23 write=25
[2022/07/14 12:36:15] [debug] [input:tail:tail.0] flb_tail_fs_inotify_init() initializing inotify tail input
[2022/07/14 12:36:15] [debug] [input:tail:tail.0] inotify watch fd=31
[2022/07/14 12:36:15] [debug] [input:tail:tail.0] scanning path ./static/*.json
[2022/07/14 12:36:15] [engine] caught signal (SIGSEGV)
[2022/07/14 12:36:15] [debug] [input:tail:tail.0] inode=4063679 with offset=17627 appended as ./static/test.json
[2022/07/14 12:36:15] [debug] [input:tail:tail.0] scan_glob add(): ./static/test.json, inode 4063679
[2022/07/14 12:36:15] [debug] [input:tail:tail.0] 1 new files found on path './static/*.json'
[2022/07/14 12:36:15] [debug] [exec:exec.1] created event channels: read=35 write=36
#0  0x7f68be39ebf2      in  ???() at 4/multiarch/strcmp-avx2.S:102
#1  0x55c878c7d40d      in  flb_parser_get() at src/flb_parser.c:895
#2  0x55c878cf41e9      in  in_exec_config_read() at plugins/in_exec/in_exec.c:164
#3  0x55c878cf44b1      in  in_exec_init() at plugins/in_exec/in_exec.c:229
#4  0x55c878c4cec0      in  flb_input_instance_init() at src/flb_input.c:671
#5  0x55c878c4cfbf      in  flb_input_init_all() at src/flb_input.c:707
#6  0x55c878c64468      in  flb_engine_start() at src/flb_engine.c:653
#7  0x55c878c400d3      in  flb_lib_worker() at src/flb_lib.c:626
#8  0x7f68bf1f5608      in  start_thread() at QYS/glibc-2.31/nptl/pthread_create.c:477
#9  0x7f68be33a132      in  ???() at sysv/linux/x86_64/clone.S:95
#10 0xffffffffffffffff  in  ???() at ???:0

To Reproduce
Fluentbit v1.9.4 installed through the official docs, with the given configuration, will reproduce.

Expected behavior

Both plug-ins work properly at the same time.

Screenshots

Your Environment

• Version used: 1.9.4
• Configuration: see above
• Environment name and version (e.g. Kubernetes? What version?): Linux node1 5.13.0-39-generic libxbee3 memory leak #44~20.04.1-Ubuntu SMP Thu Mar 24 16:43:35 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
• Server type and version:
• Operating System and version: ubuntu
• Filters and plugins: no

Additional context

none

Thanks In advance!

[patrick-stephens]

Can you test it with 1.9.6 to confirm it is not fixed already?

[nokute78]

I tested valgrind and it reports below warnings.

[2022/07/14 21:49:22] [debug] [input:tail:tail.0] inode=1450535 with offset=518 appended as ./static/a.json
[2022/07/14 21:49:22] [debug] [input:tail:tail.0] scan_glob add(): ./static/a.json, inode 1450535
[2022/07/14 21:49:22] [debug] [input:tail:tail.0] 1 new files found on path './static/*.json'
[2022/07/14 21:49:22] [debug] [exec:exec.1] created event channels: read=32 write=33
==27340== Thread 2 flb-pipeline:
==27340== Conditional jump or move depends on uninitialised value(s)
==27340==    at 0x23EFF7: in_exec_config_read (in_exec.c:163)
==27340==    by 0x23F2DB: in_exec_init (in_exec.c:229)
==27340==    by 0x193F56: flb_input_instance_init (flb_input.c:792)
==27340==    by 0x1940EA: flb_input_init_all (flb_input.c:846)
==27340==    by 0x1AE0F8: flb_engine_start (flb_engine.c:662)
==27340==    by 0x186E24: flb_lib_worker (flb_lib.c:626)
==27340==    by 0x4866608: start_thread (pthread_create.c:477)
==27340==    by 0x4BE3132: clone (clone.S:95)
==27340== 
==27340== Conditional jump or move depends on uninitialised value(s)
==27340==    at 0x23F14D: in_exec_config_read (in_exec.c:181)
==27340==    by 0x23F2DB: in_exec_init (in_exec.c:229)
==27340==    by 0x193F56: flb_input_instance_init (flb_input.c:792)
==27340==    by 0x1940EA: flb_input_init_all (flb_input.c:846)
==27340==    by 0x1AE0F8: flb_engine_start (flb_engine.c:662)
==27340==    by 0x186E24: flb_lib_worker (flb_lib.c:626)
==27340==    by 0x4866608: start_thread (pthread_create.c:477)
==27340==    by 0x4BE3132: clone (clone.S:95)
==27340== 

[nokute78]

I sent a patch #5720

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.