-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
out_stdout: new STDOUT Output Plugin. #5
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Masaya YAMAMOTO <[email protected]>
edsiper
added a commit
that referenced
this pull request
Jun 2, 2015
out_stdout: new STDOUT Output Plugin.
thanks! |
would you please check the following issue ? |
Sorry, It's my mistake. |
Closed
fujimotos
pushed a commit
to fujimotos/fluent-bit
that referenced
this pull request
Jul 22, 2019
api: add new API flb_service_set()
fujimotos
pushed a commit
to fujimotos/fluent-bit
that referenced
this pull request
Jan 15, 2020
When Fluent Bit encounters with a partial parser definition, it crashes badly with a segmentation fault. $ ./bin/fluent-bit -R parser.conf -c tail.conf ... [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser' [engine] caught signal (SIGSEGV) #0 0x558bc4a0a226 in flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700 fluent#1 0x558bc4a05d75 in flb_parser_conf_file() at src/flb_parser.c:566 fluent#2 0x558bc49f4bdd in flb_config_set_property() at src/flb_config.c:406 fluent#3 0x558bc49e24ae in flb_service_conf() at src/fluent-bit.c:446 fluent#4 0x558bc49e2f90 in main() at src/fluent-bit.c:807 fluent#5 0x7fa1cb7f109a in ???() at ???:0 fluent#6 0x558bc49e13a9 in ???() at ???:0 fluent#7 0xffffffffffffffff in ???() at ???:0 Aborted This is just because `decoders` is not being initialized properly, and that confuses Fluent Bit to deallocate a random memmory block on the cleanup path. Fix it. Signed-off-by: Fujimoto Seiji <[email protected]>
edsiper
pushed a commit
that referenced
this pull request
Jan 16, 2020
When Fluent Bit encounters with a partial parser definition, it crashes badly with a segmentation fault. $ ./bin/fluent-bit -R parser.conf -c tail.conf ... [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser' [engine] caught signal (SIGSEGV) #0 0x558bc4a0a226 in flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700 #1 0x558bc4a05d75 in flb_parser_conf_file() at src/flb_parser.c:566 #2 0x558bc49f4bdd in flb_config_set_property() at src/flb_config.c:406 #3 0x558bc49e24ae in flb_service_conf() at src/fluent-bit.c:446 #4 0x558bc49e2f90 in main() at src/fluent-bit.c:807 #5 0x7fa1cb7f109a in ???() at ???:0 #6 0x558bc49e13a9 in ???() at ???:0 #7 0xffffffffffffffff in ???() at ???:0 Aborted This is just because `decoders` is not being initialized properly, and that confuses Fluent Bit to deallocate a random memmory block on the cleanup path. Fix it. Signed-off-by: Fujimoto Seiji <[email protected]>
edsiper
pushed a commit
that referenced
this pull request
Jan 17, 2020
When Fluent Bit encounters with a partial parser definition, it crashes badly with a segmentation fault. $ ./bin/fluent-bit -R parser.conf -c tail.conf ... [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser' [engine] caught signal (SIGSEGV) #0 0x558bc4a0a226 in flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700 #1 0x558bc4a05d75 in flb_parser_conf_file() at src/flb_parser.c:566 #2 0x558bc49f4bdd in flb_config_set_property() at src/flb_config.c:406 #3 0x558bc49e24ae in flb_service_conf() at src/fluent-bit.c:446 #4 0x558bc49e2f90 in main() at src/fluent-bit.c:807 #5 0x7fa1cb7f109a in ???() at ???:0 #6 0x558bc49e13a9 in ???() at ???:0 #7 0xffffffffffffffff in ???() at ???:0 Aborted This is just because `decoders` is not being initialized properly, and that confuses Fluent Bit to deallocate a random memmory block on the cleanup path. Fix it. Signed-off-by: Fujimoto Seiji <[email protected]>
edsiper
pushed a commit
that referenced
this pull request
Jan 23, 2020
When Fluent Bit encounters with a partial parser definition, it crashes badly with a segmentation fault. $ ./bin/fluent-bit -R parser.conf -c tail.conf ... [2020/01/15 16:11:21] [error] [parser] no parser 'format' found for 'simple' in file 'conf/timestamp.parser' [engine] caught signal (SIGSEGV) #0 0x558bc4a0a226 in flb_parser_decoder_list_destroy() at src/flb_parser_decoder.c:700 #1 0x558bc4a05d75 in flb_parser_conf_file() at src/flb_parser.c:566 #2 0x558bc49f4bdd in flb_config_set_property() at src/flb_config.c:406 #3 0x558bc49e24ae in flb_service_conf() at src/fluent-bit.c:446 #4 0x558bc49e2f90 in main() at src/fluent-bit.c:807 #5 0x7fa1cb7f109a in ???() at ???:0 #6 0x558bc49e13a9 in ???() at ???:0 #7 0xffffffffffffffff in ???() at ???:0 Aborted This is just because `decoders` is not being initialized properly, and that confuses Fluent Bit to deallocate a random memmory block on the cleanup path. Fix it. Signed-off-by: Fujimoto Seiji <[email protected]>
allamand
pushed a commit
to allamand/fluent-bit
that referenced
this pull request
Oct 26, 2020
Clean CloudWatch log group once the validation is done
4 tasks
edsiper
added a commit
that referenced
this pull request
Feb 25, 2021
When libco starts, it might enter in a race condition if multiple threads are trying to initialize the 'co_swap' function, this check is done on every coroutine creation: ==346246== Possible data race during read of size 8 at 0x5CA890 by thread #5 ==346246== Locks held: none ==346246== at 0x48EFAE: co_create (amd64.c:132) ==346246== by 0x173035: flb_output_coro_create (flb_output.h:511) ==346246== by 0x173035: output_thread (flb_output_thread.c:281) ==346246== by 0x1889BE: step_callback (flb_worker.c:44) ==346246== by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==346246== by 0x487E58F: start_thread (pthread_create.c:463) ==346246== by 0x4F47222: clone (clone.S:95) ==346246== ==346246== This conflicts with a previous write of size 8 by thread #4 ==346246== Locks held: none ==346246== at 0x48EFCB: co_create (amd64.c:134) ==346246== by 0x173035: flb_output_coro_create (flb_output.h:511) ==346246== by 0x173035: output_thread (flb_output_thread.c:281) ==346246== by 0x1889BE: step_callback (flb_worker.c:44) ==346246== by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==346246== by 0x487E58F: start_thread (pthread_create.c:463) ==346246== by 0x4F47222: clone (clone.S:95) ==346246== Address 0x5ca890 is 0 bytes inside data symbol "co_swap" This patch introduce a new API for flb_coro interface that aims to be called inside every worker thread. The access to this first initialization is protected. No more race conditions on that piece of code has been seen with valgrind after the usage of this new function (next patches). Signed-off-by: Eduardo Silva <[email protected]>
edsiper
added a commit
that referenced
this pull request
Mar 2, 2021
When libco starts, it might enter in a race condition if multiple threads are trying to initialize the 'co_swap' function, this check is done on every coroutine creation: ==346246== Possible data race during read of size 8 at 0x5CA890 by thread #5 ==346246== Locks held: none ==346246== at 0x48EFAE: co_create (amd64.c:132) ==346246== by 0x173035: flb_output_coro_create (flb_output.h:511) ==346246== by 0x173035: output_thread (flb_output_thread.c:281) ==346246== by 0x1889BE: step_callback (flb_worker.c:44) ==346246== by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==346246== by 0x487E58F: start_thread (pthread_create.c:463) ==346246== by 0x4F47222: clone (clone.S:95) ==346246== ==346246== This conflicts with a previous write of size 8 by thread #4 ==346246== Locks held: none ==346246== at 0x48EFCB: co_create (amd64.c:134) ==346246== by 0x173035: flb_output_coro_create (flb_output.h:511) ==346246== by 0x173035: output_thread (flb_output_thread.c:281) ==346246== by 0x1889BE: step_callback (flb_worker.c:44) ==346246== by 0x4843B1A: ??? (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_helgrind-amd64-linux.so) ==346246== by 0x487E58F: start_thread (pthread_create.c:463) ==346246== by 0x4F47222: clone (clone.S:95) ==346246== Address 0x5ca890 is 0 bytes inside data symbol "co_swap" This patch introduce a new API for flb_coro interface that aims to be called inside every worker thread. The access to this first initialization is protected. No more race conditions on that piece of code has been seen with valgrind after the usage of this new function (next patches). Signed-off-by: Eduardo Silva <[email protected]>
This was referenced Jul 25, 2021
This was referenced Jun 21, 2022
cosmo0920
added a commit
that referenced
this pull request
Oct 5, 2022
…es strictly Without this check, the following weird error is occurred intermittently: ```log [0] dummy.0: [1664938706.407551000, {"message"=>"dummy"}] [2022/10/05 11:58:27] [ info] [test] flush record flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** error for object 0x600002600074: pointer being realloc'd was not allocated flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** set a breakpoint in malloc_error_break to debug ``` The main reason is, num_records index is broken in some cases: ``` flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** error for object 0x600002600074: pointer being realloc'd was not allocated flb-rt-core_chunk_trace(32205,0x16fe87000) malloc: *** set a breakpoint in malloc_error_break to debug [2022/10/05 11:58:27] [ info] [input] pausing dummy.0 Process 32205 stopped * thread #2, name = 'flb-pipeline', stop reason = breakpoint 1.1 frame #0: 0x00000001b34a3120 libsystem_malloc.dylib`malloc_error_break libsystem_malloc.dylib`malloc_error_break: -> 0x1b34a3120 <+0>: pacibsp 0x1b34a3124 <+4>: stp x29, x30, [sp, #-0x10]! 0x1b34a3128 <+8>: mov x29, sp 0x1b34a312c <+12>: nop Target 0: (flb-rt-core_chunk_trace) stopped. (lldb) bt * thread #2, name = 'flb-pipeline', stop reason = breakpoint 1.1 * frame #0: 0x00000001b34a3120 libsystem_malloc.dylib`malloc_error_break frame #1: 0x00000001b3494844 libsystem_malloc.dylib`malloc_vreport + 428 frame #2: 0x00000001b3497f34 libsystem_malloc.dylib`malloc_report + 64 frame #3: 0x00000001b3488210 libsystem_malloc.dylib`realloc + 328 frame #4: 0x0000000100006154 flb-rt-core_chunk_trace`flb_realloc(ptr=0x0000600002600074, size=18446744064764412176) at flb_mem.h:94:12 frame #5: 0x0000000100005fc8 flb-rt-core_chunk_trace`callback_add_record(data=0x0000600003014000, size=135, cb_data=0x0000600000004010) at core_chunk_trace.c:51:28 frame #6: 0x00000001001268b0 flb-rt-core_chunk_trace`out_lib_flush(event_chunk=0x0000600000c14000, out_flush=0x0000600001714000, i_ins=0x0000000100b09ab0, out_context=0x0000600000204a80, config=0x000000010181d200) at out_lib.c:197:9 frame #7: 0x0000000100029d70 flb-rt-core_chunk_trace`output_pre_cb_flush at flb_output.h:517:5 frame #8: 0x000000010044fa64 flb-rt-core_chunk_trace`co_switch(handle=0x000000010044fa64) at aarch64.c:133:4 (lldb) frane select 5 error: 'frane' is not a valid command. (lldb) frame select 5 frame #5: 0x0000000100005fc8 flb-rt-core_chunk_trace`callback_add_record(data=0x0000600003014000, size=135, cb_data=0x0000600000004010) at core_chunk_trace.c:51:28 48 flb_calloc(1, sizeof(struct callback_record)); 49 } else { 50 ctx->records = (struct callback_record *) -> 51 flb_realloc(ctx->records, 52 (ctx->num_records+1)*sizeof(struct callback_record)); 53 } 54 if (ctx->records == NULL) { (lldb) po ctx->records 0x0000600002600074 (lldb) po ctx->records 0x0000600002600074 (lldb) po ctx->num_records -559071216 ``` Signed-off-by: Hiroshi Hatake <[email protected]>
rawahars
referenced
this pull request
in rawahars/fluent-bit
Oct 24, 2022
4 tasks
zecke
added a commit
to zecke/fluent-bit
that referenced
this pull request
May 25, 2024
The tls variable for out_flush_params is not initialized as the flb_start function is not called during the dry run. Call flb_init directly and then shutdown the engine. configuration test is successful ================================================================= ==63633==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001f71b3ac0 in thread T0 #0 0x103c9f260 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53260) fluent#1 0x100179d9c in flb_free flb_mem.h:127 fluent#2 0x10017f4a0 in flb_output_exit flb_output.c:481 fluent#3 0x1001cb038 in flb_engine_shutdown flb_engine.c:1119 fluent#4 0x10010d45c in flb_destroy flb_lib.c:240 fluent#5 0x100008c40 in flb_main fluent-bit.c:1348 fluent#6 0x10000c644 in main fluent-bit.c:1456 fluent#7 0x18f11e0dc (<unknown module>) frame fluent#6: 0x000000010017f4a4 fluent-bit`flb_output_exit(config=0x0000000102b00200) at flb_output.c:481:9 478 479 params = FLB_TLS_GET(out_flush_params); 480 if (params) { -> 481 flb_free(params); 482 } 483 } Signed-off-by: Holger Hans Peter Freyther <[email protected]>
edsiper
pushed a commit
that referenced
this pull request
May 26, 2024
The tls variable for out_flush_params is not initialized as the flb_start function is not called during the dry run. Call flb_init directly and then shutdown the engine. configuration test is successful ================================================================= ==63633==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001f71b3ac0 in thread T0 #0 0x103c9f260 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53260) #1 0x100179d9c in flb_free flb_mem.h:127 #2 0x10017f4a0 in flb_output_exit flb_output.c:481 #3 0x1001cb038 in flb_engine_shutdown flb_engine.c:1119 #4 0x10010d45c in flb_destroy flb_lib.c:240 #5 0x100008c40 in flb_main fluent-bit.c:1348 #6 0x10000c644 in main fluent-bit.c:1456 #7 0x18f11e0dc (<unknown module>) frame #6: 0x000000010017f4a4 fluent-bit`flb_output_exit(config=0x0000000102b00200) at flb_output.c:481:9 478 479 params = FLB_TLS_GET(out_flush_params); 480 if (params) { -> 481 flb_free(params); 482 } 483 } Signed-off-by: Holger Hans Peter Freyther <[email protected]>
markuman
pushed a commit
to markuman/fluent-bit
that referenced
this pull request
May 29, 2024
The tls variable for out_flush_params is not initialized as the flb_start function is not called during the dry run. Call flb_init directly and then shutdown the engine. configuration test is successful ================================================================= ==63633==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x0001f71b3ac0 in thread T0 #0 0x103c9f260 in wrap_free+0x98 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x53260) fluent#1 0x100179d9c in flb_free flb_mem.h:127 fluent#2 0x10017f4a0 in flb_output_exit flb_output.c:481 fluent#3 0x1001cb038 in flb_engine_shutdown flb_engine.c:1119 fluent#4 0x10010d45c in flb_destroy flb_lib.c:240 fluent#5 0x100008c40 in flb_main fluent-bit.c:1348 fluent#6 0x10000c644 in main fluent-bit.c:1456 fluent#7 0x18f11e0dc (<unknown module>) frame fluent#6: 0x000000010017f4a4 fluent-bit`flb_output_exit(config=0x0000000102b00200) at flb_output.c:481:9 478 479 params = FLB_TLS_GET(out_flush_params); 480 if (params) { -> 481 flb_free(params); 482 } 483 } Signed-off-by: Holger Hans Peter Freyther <[email protected]> Signed-off-by: Markus Bergholz <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I implemented a new STDOUT Output Plugin.
I hope that it will assist debugging.
Note: An error has occurred that "libmsgpack.a is not a PIC format" in the process of generating a shared library of fluent-bit. So, I added the code to add the -fPIC option to CMakeLists.txt of msgpack.
Signed-off-by: Masaya YAMAMOTO [email protected]