Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

out_stdout: new STDOUT Output Plugin. #5

Merged
merged 1 commit into from
Jun 2, 2015

Conversation

pandax381
Copy link
Contributor

I implemented a new STDOUT Output Plugin.
I hope that it will assist debugging.

$ ./fluent-bit -i cpu -o stdout
Fluent-Bit v0.1.0
Copyright (C) Treasure Data

[2015/06/02 18:32:54] [ info] starting engine
[0] {"time"=>1433237575, "cpu"=>0.000000}
[1] {"time"=>1433237576, "cpu"=>0.000000}
[2] {"time"=>1433237577, "cpu"=>0.000000}
[3] {"time"=>1433237578, "cpu"=>0.000000}
[2015/06/02 18:32:59] [ info] Flush buf 96 bytes
[0] {"time"=>1433237579, "cpu"=>0.000000}
[1] {"time"=>1433237580, "cpu"=>0.000000}
[2] {"time"=>1433237581, "cpu"=>0.000000}
[3] {"time"=>1433237582, "cpu"=>0.000000}
[4] {"time"=>1433237583, "cpu"=>0.000000}
[2015/06/02 18:33:04] [ info] Flush buf 120 bytes
...

Note: An error has occurred that "libmsgpack.a is not a PIC format" in the process of generating a shared library of fluent-bit. So, I added the code to add the -fPIC option to CMakeLists.txt of msgpack.

Scanning dependencies of target fluent-bit-shared
[ 81%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_pack.c.o
[ 84%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_input.c.o
[ 87%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_output.c.o
[ 90%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_config.c.o
[ 93%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_network.c.o
[ 96%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_utils.c.o
[100%] Building C object src/CMakeFiles/fluent-bit-shared.dir/flb_engine.c.o
Linking C shared library ../library/libfluent-bit.so
/usr/bin/ld: ../library/libmsgpack.a(unpack.c.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC
../library/libmsgpack.a: could not read symbols: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [library/libfluent-bit.so] Error 1
make[1]: *** [src/CMakeFiles/fluent-bit-shared.dir/all] Error 2
make: *** [all] Error 2

Signed-off-by: Masaya YAMAMOTO [email protected]

edsiper added a commit that referenced this pull request Jun 2, 2015
out_stdout: new STDOUT Output Plugin.
@edsiper edsiper merged commit 634584e into fluent:master Jun 2, 2015
@edsiper
Copy link
Member

edsiper commented Jun 2, 2015

thanks!

@edsiper
Copy link
Member

edsiper commented Jun 3, 2015

would you please check the following issue ?

#6

@pandax381
Copy link
Contributor Author

Sorry, It's my mistake.
The cause of this problem was in in_mem plugin.
I will pull request to fix the problem urgently.

@pandax381 pandax381 deleted the out_stdout branch June 3, 2015 05:14
@prashantvicky prashantvicky mentioned this pull request Aug 28, 2018
fujimotos pushed a commit to fujimotos/fluent-bit that referenced this pull request Jul 22, 2019
api: add new API flb_service_set()
fujimotos pushed a commit to fujimotos/fluent-bit that referenced this pull request Jan 15, 2020
When Fluent Bit encounters with a partial parser definition, it
crashes badly with a segmentation fault.

    $ ./bin/fluent-bit -R parser.conf -c tail.conf
    ...
    [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser'
    [engine] caught signal (SIGSEGV)
    #0  0x558bc4a0a226      in  flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700
    fluent#1  0x558bc4a05d75      in  flb_parser_conf_file() at src/flb_parser.c:566
    fluent#2  0x558bc49f4bdd      in  flb_config_set_property() at src/flb_config.c:406
    fluent#3  0x558bc49e24ae      in  flb_service_conf() at src/fluent-bit.c:446
    fluent#4  0x558bc49e2f90      in  main() at src/fluent-bit.c:807
    fluent#5  0x7fa1cb7f109a      in  ???() at ???:0
    fluent#6  0x558bc49e13a9      in  ???() at ???:0
    fluent#7  0xffffffffffffffff  in  ???() at ???:0
    Aborted

This is just because `decoders` is not being initialized properly,
and that confuses Fluent Bit to deallocate a random memmory block
on the cleanup path. Fix it.

Signed-off-by: Fujimoto Seiji <[email protected]>
edsiper pushed a commit that referenced this pull request Jan 16, 2020
When Fluent Bit encounters with a partial parser definition, it
crashes badly with a segmentation fault.

    $ ./bin/fluent-bit -R parser.conf -c tail.conf
    ...
    [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser'
    [engine] caught signal (SIGSEGV)
    #0  0x558bc4a0a226      in  flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700
    #1  0x558bc4a05d75      in  flb_parser_conf_file() at src/flb_parser.c:566
    #2  0x558bc49f4bdd      in  flb_config_set_property() at src/flb_config.c:406
    #3  0x558bc49e24ae      in  flb_service_conf() at src/fluent-bit.c:446
    #4  0x558bc49e2f90      in  main() at src/fluent-bit.c:807
    #5  0x7fa1cb7f109a      in  ???() at ???:0
    #6  0x558bc49e13a9      in  ???() at ???:0
    #7  0xffffffffffffffff  in  ???() at ???:0
    Aborted

This is just because `decoders` is not being initialized properly,
and that confuses Fluent Bit to deallocate a random memmory block
on the cleanup path. Fix it.

Signed-off-by: Fujimoto Seiji <[email protected]>
edsiper pushed a commit that referenced this pull request Jan 17, 2020
When Fluent Bit encounters with a partial parser definition, it
crashes badly with a segmentation fault.

    $ ./bin/fluent-bit -R parser.conf -c tail.conf
    ...
    [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser'
    [engine] caught signal (SIGSEGV)
    #0  0x558bc4a0a226      in  flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700
    #1  0x558bc4a05d75      in  flb_parser_conf_file() at src/flb_parser.c:566
    #2  0x558bc49f4bdd      in  flb_config_set_property() at src/flb_config.c:406
    #3  0x558bc49e24ae      in  flb_service_conf() at src/fluent-bit.c:446
    #4  0x558bc49e2f90      in  main() at src/fluent-bit.c:807
    #5  0x7fa1cb7f109a      in  ???() at ???:0
    #6  0x558bc49e13a9      in  ???() at ???:0
    #7  0xffffffffffffffff  in  ???() at ???:0
    Aborted

This is just because `decoders` is not being initialized properly,
and that confuses Fluent Bit to deallocate a random memmory block
on the cleanup path. Fix it.

Signed-off-by: Fujimoto Seiji <[email protected]>
edsiper pushed a commit that referenced this pull request Jan 23, 2020
When Fluent Bit encounters with a partial parser definition, it
crashes badly with a segmentation fault.

    $ ./bin/fluent-bit -R parser.conf -c tail.conf
    ...
    [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser'
    [engine] caught signal (SIGSEGV)
    #0  0x558bc4a0a226      in  flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700
    #1  0x558bc4a05d75      in  flb_parser_conf_file() at src/flb_parser.c:566
    #2  0x558bc49f4bdd      in  flb_config_set_property() at src/flb_config.c:406
    #3  0x558bc49e24ae      in  flb_service_conf() at src/fluent-bit.c:446
    #4  0x558bc49e2f90      in  main() at src/fluent-bit.c:807
    #5  0x7fa1cb7f109a      in  ???() at ???:0
    #6  0x558bc49e13a9      in  ???() at ???:0
    #7  0xffffffffffffffff  in  ???() at ???:0
    Aborted

This is just because `decoders` is not being initialized properly,
and that confuses Fluent Bit to deallocate a random memmory block
on the cleanup path. Fix it.

Signed-off-by: Fujimoto Seiji <[email protected]>
allamand pushed a commit to allamand/fluent-bit that referenced this pull request Oct 26, 2020
Clean CloudWatch log group once the validation is done
edsiper added a commit that referenced this pull request Feb 25, 2021
When libco starts, it might enter in a race condition if multiple
threads are trying to initialize the 'co_swap' function, this check
is done on every coroutine creation:

  ==346246== Possible data race during read of size 8 at 0x5CA890 by thread #5
  ==346246== Locks held: none
  ==346246==    at 0x48EFAE: co_create (amd64.c:132)
  ==346246==    by 0x173035: flb_output_coro_create (flb_output.h:511)
  ==346246==    by 0x173035: output_thread (flb_output_thread.c:281)
  ==346246==    by 0x1889BE: step_callback (flb_worker.c:44)
  ==346246==    by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
  ==346246==    by 0x487E58F: start_thread (pthread_create.c:463)
  ==346246==    by 0x4F47222: clone (clone.S:95)
  ==346246==
  ==346246== This conflicts with a previous write of size 8 by thread #4
  ==346246== Locks held: none
  ==346246==    at 0x48EFCB: co_create (amd64.c:134)
  ==346246==    by 0x173035: flb_output_coro_create (flb_output.h:511)
  ==346246==    by 0x173035: output_thread (flb_output_thread.c:281)
  ==346246==    by 0x1889BE: step_callback (flb_worker.c:44)
  ==346246==    by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
  ==346246==    by 0x487E58F: start_thread (pthread_create.c:463)
  ==346246==    by 0x4F47222: clone (clone.S:95)
  ==346246==  Address 0x5ca890 is 0 bytes inside data symbol "co_swap"

This patch introduce a new API for flb_coro interface that aims to
be called inside every worker thread. The access to this first
initialization is protected.

No more race conditions on that piece of code has been seen with valgrind
after the usage of this new function (next patches).

Signed-off-by: Eduardo Silva <[email protected]>
edsiper added a commit that referenced this pull request Mar 2, 2021
When libco starts, it might enter in a race condition if multiple
threads are trying to initialize the 'co_swap' function, this check
is done on every coroutine creation:

  ==346246== Possible data race during read of size 8 at 0x5CA890 by thread #5
  ==346246== Locks held: none
  ==346246==    at 0x48EFAE: co_create (amd64.c:132)
  ==346246==    by 0x173035: flb_output_coro_create (flb_output.h:511)
  ==346246==    by 0x173035: output_thread (flb_output_thread.c:281)
  ==346246==    by 0x1889BE: step_callback (flb_worker.c:44)
  ==346246==    by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
  ==346246==    by 0x487E58F: start_thread (pthread_create.c:463)
  ==346246==    by 0x4F47222: clone (clone.S:95)
  ==346246==
  ==346246== This conflicts with a previous write of size 8 by thread #4
  ==346246== Locks held: none
  ==346246==    at 0x48EFCB: co_create (amd64.c:134)
  ==346246==    by 0x173035: flb_output_coro_create (flb_output.h:511)
  ==346246==    by 0x173035: output_thread (flb_output_thread.c:281)
  ==346246==    by 0x1889BE: step_callback (flb_worker.c:44)
  ==346246==    by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so)
  ==346246==    by 0x487E58F: start_thread (pthread_create.c:463)
  ==346246==    by 0x4F47222: clone (clone.S:95)
  ==346246==  Address 0x5ca890 is 0 bytes inside data symbol "co_swap"

This patch introduce a new API for flb_coro interface that aims to
be called inside every worker thread. The access to this first
initialization is protected.

No more race conditions on that piece of code has been seen with valgrind
after the usage of this new function (next patches).

Signed-off-by: Eduardo Silva <[email protected]>
cosmo0920 added a commit that referenced this pull request Oct 5, 2022
…es strictly

Without this check, the following weird error is occurred
intermittently:

```log
[0] dummy.0: [1664938706.407551000, {"message"=>"dummy"}]
[2022/10/05 11:58:27] [ info] [test] flush record
flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** error for object 0x600002600074: pointer being realloc'd was not allocated
flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** set a breakpoint in malloc_error_break to debug
```

The main reason is, num_records index is broken in some cases:

```
flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** error for object 0x600002600074: pointer being realloc'd was not allocated
flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** set a breakpoint in malloc_error_break to debug
[2022/10/05 11:58:27] [ info] [input] pausing dummy.0
Process 32205 stopped
* thread #2, name = 'flb-pipeline', stop reason = breakpoint 1.1
    frame #0: 0x00000001b34a3120 libsystem_malloc.dylib`malloc_error_break
libsystem_malloc.dylib`malloc_error_break:
->  0x1b34a3120 <+0>:  pacibsp
    0x1b34a3124 <+4>:  stp    x29, x30, [sp, #-0x10]!
    0x1b34a3128 <+8>:  mov    x29, sp
    0x1b34a312c <+12>: nop
Target 0: (flb-rt-core_chunk_trace) stopped.
(lldb) bt
* thread #2, name = 'flb-pipeline', stop reason = breakpoint 1.1
  * frame #0: 0x00000001b34a3120 libsystem_malloc.dylib`malloc_error_break
    frame #1: 0x00000001b3494844 libsystem_malloc.dylib`malloc_vreport + 428
    frame #2: 0x00000001b3497f34 libsystem_malloc.dylib`malloc_report + 64
    frame #3: 0x00000001b3488210 libsystem_malloc.dylib`realloc + 328
    frame #4: 0x0000000100006154 flb-rt-core_chunk_trace`flb_realloc(ptr=0x0000600002600074, size=18446744064764412176) at flb_mem.h:94:12
    frame #5: 0x0000000100005fc8 flb-rt-core_chunk_trace`callback_add_record(data=0x0000600003014000, size=135, cb_data=0x0000600000004010) at core_chunk_trace.c:51:28
    frame #6: 0x00000001001268b0 flb-rt-core_chunk_trace`out_lib_flush(event_chunk=0x0000600000c14000, out_flush=0x0000600001714000, i_ins=0x0000000100b09ab0, out_context=0x0000600000204a80, config=0x000000010181d200) at out_lib.c:197:9
    frame #7: 0x0000000100029d70 flb-rt-core_chunk_trace`output_pre_cb_flush at flb_output.h:517:5
    frame #8: 0x000000010044fa64 flb-rt-core_chunk_trace`co_switch(handle=0x000000010044fa64) at aarch64.c:133:4
(lldb) frane select 5
error: 'frane' is not a valid command.
(lldb) frame select 5
frame #5: 0x0000000100005fc8 flb-rt-core_chunk_trace`callback_add_record(data=0x0000600003014000, size=135, cb_data=0x0000600000004010) at core_chunk_trace.c:51:28
   48  	                           flb_calloc(1, sizeof(struct callback_record));
   49  	        } else {
   50  	            ctx->records = (struct callback_record *)
-> 51  	                           flb_realloc(ctx->records,
   52  	                                       (ctx->num_records+1)*sizeof(struct callback_record));
   53  	        }
   54  	        if (ctx->records ==  NULL) {
(lldb) po ctx->records
0x0000600002600074

(lldb) po ctx->records
0x0000600002600074

(lldb) po ctx->num_records
-559071216
```

Signed-off-by: Hiroshi Hatake <[email protected]>
zecke added a commit to zecke/fluent-bit that referenced this pull request May 25, 2024
The tls variable for out_flush_params is not initialized as the
flb_start function is not called during the dry run. Call flb_init
directly and then shutdown the engine.

configuration test is successful
=================================================================
==63633==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001f71b3ac0 in thread T0
    #0 0x103c9f260 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53260)
    fluent#1 0x100179d9c in flb_free flb_mem.h:127
    fluent#2 0x10017f4a0 in flb_output_exit flb_output.c:481
    fluent#3 0x1001cb038 in flb_engine_shutdown flb_engine.c:1119
    fluent#4 0x10010d45c in flb_destroy flb_lib.c:240
    fluent#5 0x100008c40 in flb_main fluent-bit.c:1348
    fluent#6 0x10000c644 in main fluent-bit.c:1456
    fluent#7 0x18f11e0dc  (<unknown module>)

frame fluent#6: 0x000000010017f4a4 fluent-bit`flb_output_exit(config=0x0000000102b00200) at flb_output.c:481:9
   478
   479 	    params = FLB_TLS_GET(out_flush_params);
   480 	    if (params) {
-> 481 	        flb_free(params);
   482 	    }
   483 	}

Signed-off-by: Holger Hans Peter Freyther <[email protected]>
@zecke zecke mentioned this pull request May 25, 2024
1 task
edsiper pushed a commit that referenced this pull request May 26, 2024
The tls variable for out_flush_params is not initialized as the
flb_start function is not called during the dry run. Call flb_init
directly and then shutdown the engine.

configuration test is successful
=================================================================
==63633==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001f71b3ac0 in thread T0
    #0 0x103c9f260 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53260)
    #1 0x100179d9c in flb_free flb_mem.h:127
    #2 0x10017f4a0 in flb_output_exit flb_output.c:481
    #3 0x1001cb038 in flb_engine_shutdown flb_engine.c:1119
    #4 0x10010d45c in flb_destroy flb_lib.c:240
    #5 0x100008c40 in flb_main fluent-bit.c:1348
    #6 0x10000c644 in main fluent-bit.c:1456
    #7 0x18f11e0dc  (<unknown module>)

frame #6: 0x000000010017f4a4 fluent-bit`flb_output_exit(config=0x0000000102b00200) at flb_output.c:481:9
   478
   479 	    params = FLB_TLS_GET(out_flush_params);
   480 	    if (params) {
-> 481 	        flb_free(params);
   482 	    }
   483 	}

Signed-off-by: Holger Hans Peter Freyther <[email protected]>
markuman pushed a commit to markuman/fluent-bit that referenced this pull request May 29, 2024
The tls variable for out_flush_params is not initialized as the
flb_start function is not called during the dry run. Call flb_init
directly and then shutdown the engine.

configuration test is successful
=================================================================
==63633==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001f71b3ac0 in thread T0
    #0 0x103c9f260 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53260)
    fluent#1 0x100179d9c in flb_free flb_mem.h:127
    fluent#2 0x10017f4a0 in flb_output_exit flb_output.c:481
    fluent#3 0x1001cb038 in flb_engine_shutdown flb_engine.c:1119
    fluent#4 0x10010d45c in flb_destroy flb_lib.c:240
    fluent#5 0x100008c40 in flb_main fluent-bit.c:1348
    fluent#6 0x10000c644 in main fluent-bit.c:1456
    fluent#7 0x18f11e0dc  (<unknown module>)

frame fluent#6: 0x000000010017f4a4 fluent-bit`flb_output_exit(config=0x0000000102b00200) at flb_output.c:481:9
   478
   479 	    params = FLB_TLS_GET(out_flush_params);
   480 	    if (params) {
-> 481 	        flb_free(params);
   482 	    }
   483 	}

Signed-off-by: Holger Hans Peter Freyther <[email protected]>
Signed-off-by: Markus Bergholz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants