initial readme for oss #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
RafalKorepta
added a commit
to RafalKorepta/redpanda
that referenced
this pull request
Jan 14, 2021
…3-alpha Initial project structure generated by kubebuilder
dotnwat
added a commit
that referenced
this pull request
Aug 9, 2021
When a continuation like `.then([..] {...})` runs the lambda parameter
is not executed if it is a attached to a failed future. In this case
the lambda will be destroyed without running. In the case of kafka two
stage dispatch handling, we had the following:
first_stage.then([f = std::move(second_stage)] { ... }).then(...);
and we encountered a situation where the first and second stages failed.
when the continuation attached to the first_stage ran the captured `f`
exceptional future was destroyed (via the lambda that captured it). if
the first stage had not failed, then the lambda had proper handling for
a failed `f`.
This is an instance of a class of bug that is pretty hard to address. In this
case it was a bad_alloc that was thrown which caused the issue, not an
exceptional case that we explicitly handle.
to fix this we check if the first stage failed, and if so, we explicitly
consume the second stage to avoid this case. if the first stage
succeeds, then we operate as normal and background the second stage.
Aug 07 19:53:30 ip-172-31-45-184 rpk[21830]: WARN 2021-08-07 19:53:30,680 [shard 30] raft - [group_id:3069, {kafka/foo3/4}] replicate_entries_stm.cc:100 - Error while replicating entries std::bad_alloc (std::bad_alloc)
Aug 07 19:53:30 ip-172-31-45-184 rpk[21830]: WARN 2021-08-07 19:53:30,680 [shard 31] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace:
[Backtrace #0]
void seastar::backtrace<seastar::current_backtrace_tasklocal()::$_3>(seastar::current_backtrace_tasklocal()::$_3&&) at /v/build/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/backtrace.hh:59
(inlined by) seastar::current_backtrace_tasklocal() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/backtrace.cc:86
(inlined by) seastar::current_tasktrace() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/backtrace.cc:137
(inlined by) seastar::current_backtrace() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/backtrace.cc:170
seastar::report_failed_future(std::exception_ptr const&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/future.cc:210
(inlined by) seastar::report_failed_future(seastar::future_state_base::any&&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/future.cc:218
addr2line: DWARF error: could not find variable specification at offset a8a
addr2line: DWARF error: could not find variable specification at offset c49
addr2line: DWARF error: could not find variable specification at offset 46223
addr2line: DWARF error: could not find variable specification at offset 4d49d
addr2line: DWARF error: could not find variable specification at offset 5f261
seastar::future_state_base::any::check_failure() at /vectorized/include/seastar/core/future.hh:567
(inlined by) seastar::future_state<seastar::foreign_ptr<std::__1::unique_ptr<kafka::response, std::__1::default_delete<kafka::response> > > >::clear() at /vectorized/include/seastar/core/future.hh:609
(inlined by) ~future_state at /vectorized/include/seastar/core/future.hh:614
(inlined by) ~future at /vectorized/include/seastar/core/future.hh:1337
(inlined by) ~ at /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0eeaee3e12186c1d5-1/vectorized/redpanda/vbuild/release/clang/../../../src/v/kafka/server/connection_context.cc:257
(inlined by) ~continuation at /vectorized/include/seastar/core/future.hh:750
(inlined by) seastar::continuation<seastar::internal::promise_base_with_type<void>, kafka::connection_context::dispatch_method_once(kafka::request_header, unsigned long)::$_0::operator()(kafka::connection_context::session_resources)::{lambda(iobuf)#1}::operator()(iobuf)::{lambda(
)#1}, seastar::future<void>::then_impl_nrvo<{lambda(iobuf)#1}, kafka::connection_context::dispatch_method_once(kafka::request_header, unsigned long)::$_0::operator()(kafka::connection_context::session_resources)::{lambda(iobuf)#1}::operator()(iobuf)::{lambda()#1}>({lambda(iobuf)#1
}&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, {lambda(iobuf)#1}&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>::run_and_dispose() at /vectorized/include/seastar/core/future.hh:771
addr2line: DWARF error: could not find variable specification at offset 333c9
addr2line: DWARF error: could not find variable specification at offset 4afb7
addr2line: DWARF error: could not find variable specification at offset 4ead7
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2263
(inlined by) seastar::reactor::run_some_tasks() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2672
seastar::reactor::run() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2831
operator() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4022
std::__1::__function::__value_func<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:1885
(inlined by) std::__1::function<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:2560
(inlined by) seastar::posix_thread::start_routine(void*) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:60
addr2line: '/opt/redpanda/lib/libpthread.so.0': No such file
/opt/redpanda/lib/libpthread.so.0 0x9298 addr2line: '/opt/redpanda/lib/libc.so.6': No such file
/opt/redpanda/lib/libc.so.6 0x1006a2
Signed-off-by: Noah Watkins <[email protected]>
dotnwat
added a commit
that referenced
this pull request
Aug 9, 2021
When the produce handler first stage failed the original code would also
immediately fail the second stage return future with the same error.
There is nothing wrong with those semantics, but the problem is that by
immediately failing the second stage the actual futures for the second
stage were run in the background while holding a live reference to the
request context, leading to the segfault below.
What's most interesting about this is that it requires the produce
request to have data for two partitions for the bug to trigger. In the
scenario the handling for one partition append succeeds in the first and
second stages, while the other partition append fails in the first
stage. In this case the first stage failure causes the request context
to be freed while the second stage still contains a live append request.
Is is the live append request that eventually touches the request
context.
void seastar::backtrace<seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)#1}>(seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)#1}&&) at /v/build/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/backtrace.hh:59
(inlined by) seastar::backtrace_buffer::append_backtrace() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:758
(inlined by) seastar::print_with_backtrace(seastar::backtrace_buffer&, bool) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:788
(inlined by) seastar::print_with_backtrace(char const*, bool) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:800
(inlined by) seastar::sigsegv_action() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3593
(inlined by) operator() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3579
(inlined by) __invoke at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3575
?? ??:0
addr2line: DWARF error: could not find variable specification at offset 4b
addr2line: DWARF error: could not find variable specification at offset ec210
addr2line: DWARF error: could not find variable specification at offset ec3d7
lw_shared_ptr at /vectorized/include/seastar/core/shared_ptr.hh:291
(inlined by) kafka::request_context::connection() at /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0eeaee3e12186c1d5-1/vectorized/redpanda/vbuild/release/clang/../../../src/v/kafka/server/request_context.h:81
(inlined by) operator() at /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0eeaee3e12186c1d5-1/vectorized/redpanda/vbuild/release/clang/../../../src/v/kafka/server/handlers/produce.cc:315
(inlined by) _ZNSt3__18__invokeIRZN5kafkaL23produce_topic_partitionERNS1_11produce_ctxERNS1_18topic_produce_dataERNS1_22partition_produce_dataEE3$_3JNS1_26partition_produce_responseEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSB_DpOSC_ at /vectorized/llvm/bin/../include/c++/v1/type_traits:3694
(inlined by) std::__1::invoke_result<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response>::type std::__1::invoke<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response>(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response&&) at /vectorized/llvm/bin/../include/c++/v1/functional:2989
(inlined by) auto seastar::internal::future_invoke<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response>(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response&&) at /vectorized/include/seastar/core/future.hh:1211
(inlined by) operator() at /vectorized/include/seastar/core/future.hh:1582
(inlined by) void seastar::futurize<kafka::partition_produce_response>::satisfy_with_result_of<seastar::future<kafka::partition_produce_response>::then_impl_nrvo<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3, seastar::future<kafka::partition_produce_response> >(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&&)::{lambda(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, seastar::future_state<kafka::partition_produce_response>&&)#1}::operator()(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, seastar::future_state<kafka::partition_produce_response>&&) const::{lambda()#1}>(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&&) at /vectorized/include/seastar/core/future.hh:2122
(inlined by) operator() at /vectorized/include/seastar/core/future.hh:1575
(inlined by) seastar::continuation<seastar::internal::promise_base_with_type<kafka::partition_produce_response>, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3, seastar::future<kafka::partition_produce_response>::then_impl_nrvo<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3, seastar::future<kafka::partition_produce_response> >(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&&)::{lambda(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, seastar::future_state<kafka::partition_produce_response>&&)#1}, kafka::partition_produce_response>::run_and_dispose() at /vectorized/include/seastar/core/future.hh:767
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2263
(inlined by) seastar::reactor::run_some_tasks() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2672
seastar::reactor::run() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2831
operator() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4022
std::__1::__function::__value_func<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:1885
(inlined by) std::__1::function<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:2560
(inlined by) seastar::posix_thread::start_routine(void*) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:60
addr2line: '/opt/redpanda/lib/libpthread.so.0': No such file
/opt/redpanda/lib/libpthread.so.0 0x9298
addr2line: '/opt/redpanda/lib/libc.so.6': No such file
/opt/redpanda/lib/libc.so.6 0x1006a2
Signed-off-by: Noah Watkins <[email protected]>
ivotron
pushed a commit
that referenced
this pull request
Aug 11, 2021
When a continuation like `.then([..] {...})` runs the lambda parameter
is not executed if it is a attached to a failed future. In this case
the lambda will be destroyed without running. In the case of kafka two
stage dispatch handling, we had the following:
first_stage.then([f = std::move(second_stage)] { ... }).then(...);
and we encountered a situation where the first and second stages failed.
when the continuation attached to the first_stage ran the captured `f`
exceptional future was destroyed (via the lambda that captured it). if
the first stage had not failed, then the lambda had proper handling for
a failed `f`.
This is an instance of a class of bug that is pretty hard to address. In this
case it was a bad_alloc that was thrown which caused the issue, not an
exceptional case that we explicitly handle.
to fix this we check if the first stage failed, and if so, we explicitly
consume the second stage to avoid this case. if the first stage
succeeds, then we operate as normal and background the second stage.
Aug 07 19:53:30 ip-172-31-45-184 rpk[21830]: WARN 2021-08-07 19:53:30,680 [shard 30] raft - [group_id:3069, {kafka/foo3/4}] replicate_entries_stm.cc:100 - Error while replicating entries std::bad_alloc (std::bad_alloc)
Aug 07 19:53:30 ip-172-31-45-184 rpk[21830]: WARN 2021-08-07 19:53:30,680 [shard 31] seastar - Exceptional future ignored: std::bad_alloc (std::bad_alloc), backtrace:
[Backtrace #0]
void seastar::backtrace<seastar::current_backtrace_tasklocal()::$_3>(seastar::current_backtrace_tasklocal()::$_3&&) at /v/build/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/backtrace.hh:59
(inlined by) seastar::current_backtrace_tasklocal() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/backtrace.cc:86
(inlined by) seastar::current_tasktrace() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/backtrace.cc:137
(inlined by) seastar::current_backtrace() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/backtrace.cc:170
seastar::report_failed_future(std::exception_ptr const&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/future.cc:210
(inlined by) seastar::report_failed_future(seastar::future_state_base::any&&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/future.cc:218
addr2line: DWARF error: could not find variable specification at offset a8a
addr2line: DWARF error: could not find variable specification at offset c49
addr2line: DWARF error: could not find variable specification at offset 46223
addr2line: DWARF error: could not find variable specification at offset 4d49d
addr2line: DWARF error: could not find variable specification at offset 5f261
seastar::future_state_base::any::check_failure() at /vectorized/include/seastar/core/future.hh:567
(inlined by) seastar::future_state<seastar::foreign_ptr<std::__1::unique_ptr<kafka::response, std::__1::default_delete<kafka::response> > > >::clear() at /vectorized/include/seastar/core/future.hh:609
(inlined by) ~future_state at /vectorized/include/seastar/core/future.hh:614
(inlined by) ~future at /vectorized/include/seastar/core/future.hh:1337
(inlined by) ~ at /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0eeaee3e12186c1d5-1/vectorized/redpanda/vbuild/release/clang/../../../src/v/kafka/server/connection_context.cc:257
(inlined by) ~continuation at /vectorized/include/seastar/core/future.hh:750
(inlined by) seastar::continuation<seastar::internal::promise_base_with_type<void>, kafka::connection_context::dispatch_method_once(kafka::request_header, unsigned long)::$_0::operator()(kafka::connection_context::session_resources)::{lambda(iobuf)#1}::operator()(iobuf)::{lambda(
)#1}, seastar::future<void>::then_impl_nrvo<{lambda(iobuf)#1}, kafka::connection_context::dispatch_method_once(kafka::request_header, unsigned long)::$_0::operator()(kafka::connection_context::session_resources)::{lambda(iobuf)#1}::operator()(iobuf)::{lambda()#1}>({lambda(iobuf)#1
}&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, {lambda(iobuf)#1}&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>::run_and_dispose() at /vectorized/include/seastar/core/future.hh:771
addr2line: DWARF error: could not find variable specification at offset 333c9
addr2line: DWARF error: could not find variable specification at offset 4afb7
addr2line: DWARF error: could not find variable specification at offset 4ead7
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2263
(inlined by) seastar::reactor::run_some_tasks() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2672
seastar::reactor::run() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2831
operator() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4022
std::__1::__function::__value_func<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:1885
(inlined by) std::__1::function<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:2560
(inlined by) seastar::posix_thread::start_routine(void*) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:60
addr2line: '/opt/redpanda/lib/libpthread.so.0': No such file
/opt/redpanda/lib/libpthread.so.0 0x9298 addr2line: '/opt/redpanda/lib/libc.so.6': No such file
/opt/redpanda/lib/libc.so.6 0x1006a2
Signed-off-by: Noah Watkins <[email protected]>
(cherry picked from commit f906d20)
ivotron
pushed a commit
that referenced
this pull request
Aug 11, 2021
When the produce handler first stage failed the original code would also
immediately fail the second stage return future with the same error.
There is nothing wrong with those semantics, but the problem is that by
immediately failing the second stage the actual futures for the second
stage were run in the background while holding a live reference to the
request context, leading to the segfault below.
What's most interesting about this is that it requires the produce
request to have data for two partitions for the bug to trigger. In the
scenario the handling for one partition append succeeds in the first and
second stages, while the other partition append fails in the first
stage. In this case the first stage failure causes the request context
to be freed while the second stage still contains a live append request.
Is is the live append request that eventually touches the request
context.
void seastar::backtrace<seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)#1}>(seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)#1}&&) at /v/build/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/backtrace.hh:59
(inlined by) seastar::backtrace_buffer::append_backtrace() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:758
(inlined by) seastar::print_with_backtrace(seastar::backtrace_buffer&, bool) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:788
(inlined by) seastar::print_with_backtrace(char const*, bool) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:800
(inlined by) seastar::sigsegv_action() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3593
(inlined by) operator() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3579
(inlined by) __invoke at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3575
?? ??:0
addr2line: DWARF error: could not find variable specification at offset 4b
addr2line: DWARF error: could not find variable specification at offset ec210
addr2line: DWARF error: could not find variable specification at offset ec3d7
lw_shared_ptr at /vectorized/include/seastar/core/shared_ptr.hh:291
(inlined by) kafka::request_context::connection() at /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0eeaee3e12186c1d5-1/vectorized/redpanda/vbuild/release/clang/../../../src/v/kafka/server/request_context.h:81
(inlined by) operator() at /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0eeaee3e12186c1d5-1/vectorized/redpanda/vbuild/release/clang/../../../src/v/kafka/server/handlers/produce.cc:315
(inlined by) _ZNSt3__18__invokeIRZN5kafkaL23produce_topic_partitionERNS1_11produce_ctxERNS1_18topic_produce_dataERNS1_22partition_produce_dataEE3$_3JNS1_26partition_produce_responseEEEEDTclclsr3std3__1E7forwardIT_Efp_Espclsr3std3__1E7forwardIT0_Efp0_EEEOSB_DpOSC_ at /vectorized/llvm/bin/../include/c++/v1/type_traits:3694
(inlined by) std::__1::invoke_result<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response>::type std::__1::invoke<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response>(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response&&) at /vectorized/llvm/bin/../include/c++/v1/functional:2989
(inlined by) auto seastar::internal::future_invoke<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response>(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, kafka::partition_produce_response&&) at /vectorized/include/seastar/core/future.hh:1211
(inlined by) operator() at /vectorized/include/seastar/core/future.hh:1582
(inlined by) void seastar::futurize<kafka::partition_produce_response>::satisfy_with_result_of<seastar::future<kafka::partition_produce_response>::then_impl_nrvo<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3, seastar::future<kafka::partition_produce_response> >(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&&)::{lambda(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, seastar::future_state<kafka::partition_produce_response>&&)#1}::operator()(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, seastar::future_state<kafka::partition_produce_response>&&) const::{lambda()#1}>(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&&) at /vectorized/include/seastar/core/future.hh:2122
(inlined by) operator() at /vectorized/include/seastar/core/future.hh:1575
(inlined by) seastar::continuation<seastar::internal::promise_base_with_type<kafka::partition_produce_response>, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3, seastar::future<kafka::partition_produce_response>::then_impl_nrvo<kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3, seastar::future<kafka::partition_produce_response> >(kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&&)::{lambda(seastar::internal::promise_base_with_type<kafka::partition_produce_response>&&, kafka::produce_topic_partition(kafka::produce_ctx&, kafka::topic_produce_data&, kafka::partition_produce_data&)::$_3&, seastar::future_state<kafka::partition_produce_response>&&)#1}, kafka::partition_produce_response>::run_and_dispose() at /vectorized/include/seastar/core/future.hh:767
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2263
(inlined by) seastar::reactor::run_some_tasks() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2672
seastar::reactor::run() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2831
operator() at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4022
std::__1::__function::__value_func<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:1885
(inlined by) std::__1::function<void ()>::operator()() const at /vectorized/llvm/bin/../include/c++/v1/functional:2560
(inlined by) seastar::posix_thread::start_routine(void*) at /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:60
addr2line: '/opt/redpanda/lib/libpthread.so.0': No such file
/opt/redpanda/lib/libpthread.so.0 0x9298
addr2line: '/opt/redpanda/lib/libc.so.6': No such file
/opt/redpanda/lib/libc.so.6 0x1006a2
Signed-off-by: Noah Watkins <[email protected]>
(cherry picked from commit 7e35133)
felixguendling
pushed a commit
to felixguendling/redpanda
that referenced
this pull request
Dec 4, 2021
Regarding the test `serde_test` in `src/v/storage/tests/index_state_test.cc`
$> ninja storage_log_index_rpunit
$> bin/storage_log_index_rpunit
Running 7 test cases...
AddressSanitizer:DEADLYSIGNAL
=================================================================
==934285==ERROR: AddressSanitizer: stack-overflow on address 0x7fff2a209f58 (pc 0x558e9c2eab0e bp 0x7fff2a20a7a0 sp 0x7fff2a209f60 T0)
#0 0x558e9c2eab0e in __asan_memmove /home/nwatkins/src/redpanda/vbuild/llvm/src/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:30:3
redpanda-data#1 0x558e9c448ec2 in std::__1::enable_if<(is_same<std::__1::remove_const<char const>::type, char>::value) && (is_trivially_copy_assignable<char>::value), char*>::type std::__1::__copy<char const, char>(char const*, char const*, char*) /home/nwatkins/src/redpanda/vbuild/llvm/in
stall/bin/../include/c++/v1/algorithm:1735:9
redpanda-data#2 0x558e9c448e1e in char* std::__1::copy<char const*, char*>(char const*, char const*, char*) /home/nwatkins/src/redpanda/vbuild/llvm/install/bin/../include/c++/v1/algorithm:1748:16
redpanda-data#3 0x558e9c448c9a in std::__1::enable_if<__is_cpp17_random_access_iterator<char const*>::value, char*>::type std::__1::copy_n<char const*, unsigned long, char*>(char const*, unsigned long, char*) /home/nwatkins/src/redpanda/vbuild/llvm/install/bin/../include/c++/v1/algorithm:1
867:12
redpanda-data#4 0x558e9c674104 in iobuf_copy(details::io_iterator_consumer&, unsigned long)::$_4::operator()(char const*, unsigned long) const /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/bytes/iobuf.cc:145:13
redpanda-data#5 0x558e9c65aa69 in unsigned long details::io_iterator_consumer::consume<iobuf_copy(details::io_iterator_consumer&, unsigned long)::$_4>(unsigned long, iobuf_copy(details::io_iterator_consumer&, unsigned long)::$_4&&) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../sr
c/v/bytes/details/io_iterator_consumer.h:141:45
redpanda-data#6 0x558e9c659af1 in iobuf_copy(details::io_iterator_consumer&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/bytes/iobuf.cc:143:12
redpanda-data#7 0x558e9c57eeb3 in iobuf_parser_base::peek(unsigned long) const /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/bytes/iobuf_parser.h:100:16
redpanda-data#8 0x558e9c5751b6 in serde::peek_version(iobuf_parser&) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/serde/serde.h:547:43
redpanda-data#9 0x558e9c55ae54 in storage::read_nested(iobuf_parser&, storage::index_state&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/storage/index_state.cc:257:9
redpanda-data#10 0x558e9c475297 in std::__1::decay<storage::index_state>::type serde::read_nested<storage::index_state>(iobuf_parser&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/serde/serde.h:475:5
redpanda-data#11 0x558e9c55aedb in storage::read_nested(iobuf_parser&, storage::index_state&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/storage/index_state.cc:258:14
redpanda-data#12 0x558e9c475297 in std::__1::decay<storage::index_state>::type serde::read_nested<storage::index_state>(iobuf_parser&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/serde/serde.h:475:5
redpanda-data#13 0x558e9c55aedb in storage::read_nested(iobuf_parser&, storage::index_state&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/storage/index_state.cc:258:14
redpanda-data#14 0x558e9c475297 in std::__1::decay<storage::index_state>::type serde::read_nested<storage::index_state>(iobuf_parser&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/serde/serde.h:475:5
redpanda-data#15 0x558e9c55aedb in storage::read_nested(iobuf_parser&, storage::index_state&, unsigned long) /home/nwatkins/src/redpanda/vbuild/debug/clang/../../../src/v/storage/index_state.cc:258:14
...
===
This makes the recursion go away, but the deserialized output is all zeros. So that
probably isn't the solution.
diff --git a/src/v/storage/index_state.cc b/src/v/storage/index_state.cc
index 57808eae2..4aceb904f 100644
--- a/src/v/storage/index_state.cc
+++ b/src/v/storage/index_state.cc
@@ -255,7 +255,7 @@ iobuf index_state::checksum_and_serialize() {
void read_nested(iobuf_parser& in, index_state& st, size_t bytes_left_limit) {
if (serde::peek_version(in) > index_state::ondisk_version) {
- st = serde::read_nested<index_state>(in, bytes_left_limit);
+ serde::read_nested<index_state>(in, st, bytes_left_limit);
return;
}
Signed-off-by: Noah Watkins <[email protected]>
tchaikov
added a commit
to tchaikov/redpanda
that referenced
this pull request
Apr 16, 2022
This picks up the commit of 21d665aebf7b74371e4527299441812de5eea6c2,
which instantiate the template specialization of
template future<> sleep_abortable<manual_clock>(typename manual_clock::duration, abort_source&)
which is used by
net::connection_rate<seastar::manual_clock>::spawn_updating_fiber_if_needed()
in src/v/net/connection_rate.h
without this change, we'd run into following FTBFS:
src/v/net/tests/CMakeFiles/net_connection_rate_rpunit.dir/connection_rate_test.cc.o: in function `net::connection_rate<seastar::manual_clock>::spawn_updating_fiber_if_needed(seastar::lw_shared_ptr<net::connection_rate_counter<seastar::manual_clock> >)::{lambda()redpanda-data#1}::operator()()':
connection_rate_test.cc:(.text._ZZN3net15connection_rateIN7seastar12manual_clockEE30spawn_updating_fiber_if_neededENS1_13lw_shared_ptrINS_23connection_rate_counterIS2_EEEEENUlvE_clEv[_ZZN3net15connection_rateIN7seastar12manual_clockEE30spawn_updating_fiber_if_neededENS1_13lw_shared_ptrINS_23connection_rate_counterIS2_EEEEENUlvE_clEv]+0x271): undefined reference to `seastar::future<void> seastar::sleep_abortable<seastar::manual_clock>(seastar::manual_clock::duration, seastar::abort_source&)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Signed-off-by: Kefu Chai <[email protected]>
tchaikov
added a commit
to tchaikov/redpanda
that referenced
this pull request
Apr 17, 2022
This picks up the commit of 21d665aebf7b74371e4527299441812de5eea6c2,
which instantiate the template specialization of
template future<> sleep_abortable<manual_clock>(typename manual_clock::duration, abort_source&)
which is used by
net::connection_rate<seastar::manual_clock>::spawn_updating_fiber_if_needed()
in src/v/net/connection_rate.h
without this change, we'd run into following FTBFS:
src/v/net/tests/CMakeFiles/net_connection_rate_rpunit.dir/connection_rate_test.cc.o: in function `net::connection_rate<seastar::manual_clock>::spawn_updating_fiber_if_needed(seastar::lw_shared_ptr<net::connection_rate_counter<seastar::manual_clock> >)::{lambda()redpanda-data#1}::operator()()':
connection_rate_test.cc:(.text._ZZN3net15connection_rateIN7seastar12manual_clockEE30spawn_updating_fiber_if_neededENS1_13lw_shared_ptrINS_23connection_rate_counterIS2_EEEEENUlvE_clEv[_ZZN3net15connection_rateIN7seastar12manual_clockEE30spawn_updating_fiber_if_neededENS1_13lw_shared_ptrINS_23connection_rate_counterIS2_EEEEENUlvE_clEv]+0x271): undefined reference to `seastar::future<void> seastar::sleep_abortable<seastar::manual_clock>(seastar::manual_clock::duration, seastar::abort_source&)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Signed-off-by: Kefu Chai <[email protected]>
tchaikov
added a commit
to tchaikov/redpanda
that referenced
this pull request
Apr 17, 2022
This picks up the commit of 21d665aebf7b74371e4527299441812de5eea6c2,
which instantiate the template specialization of
template future<> sleep_abortable<manual_clock>(typename manual_clock::duration, abort_source&)
which is used by
net::connection_rate<seastar::manual_clock>::spawn_updating_fiber_if_needed()
in src/v/net/connection_rate.h
without this change, we'd run into following FTBFS:
src/v/net/tests/CMakeFiles/net_connection_rate_rpunit.dir/connection_rate_test.cc.o: in function `net::connection_rate<seastar::manual_clock>::spawn_updating_fiber_if_needed(seastar::lw_shared_ptr<net::connection_rate_counter<seastar::manual_clock> >)::{lambda()redpanda-data#1}::operator()()':
connection_rate_test.cc:(.text._ZZN3net15connection_rateIN7seastar12manual_clockEE30spawn_updating_fiber_if_neededENS1_13lw_shared_ptrINS_23connection_rate_counterIS2_EEEEENUlvE_clEv[_ZZN3net15connection_rateIN7seastar12manual_clockEE30spawn_updating_fiber_if_neededENS1_13lw_shared_ptrINS_23connection_rate_counterIS2_EEEEENUlvE_clEv]+0x271): undefined reference to `seastar::future<void> seastar::sleep_abortable<seastar::manual_clock>(seastar::manual_clock::duration, seastar::abort_source&)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Signed-off-by: Kefu Chai <[email protected]>
ajfabbri
pushed a commit
that referenced
this pull request
Aug 25, 2022
Three integration tests that verify behavior of max_compactable_offset: - Randomized batch / messsage generaton with a limited number of keys. Verify that compaction only deletes messages less than this max. - Single key #1: fixed message test that checks that the boundaries of the compacted area are precisely as expected. - Single key #2: same as above, but with max_compactible_offset unset, we validate that compactions works as before.
ajfabbri
pushed a commit
that referenced
this pull request
Sep 6, 2022
Three integration tests that verify behavior of max_compactable_offset: - Randomized batch / messsage generaton with a limited number of keys. Verify that compaction only deletes messages less than this max. - Single key #1: fixed message test that checks that the boundaries of the compacted area are precisely as expected. - Single key #2: same as above, but with max_compactible_offset unset, we validate that compactions works as before.
ballard26
referenced
this pull request
in ballard26/redpanda
Sep 27, 2022
Three integration tests that verify behavior of max_compactable_offset: - Randomized batch / messsage generaton with a limited number of keys. Verify that compaction only deletes messages less than this max. - Single key #1: fixed message test that checks that the boundaries of the compacted area are precisely as expected. - Single key redpanda-data#2: same as above, but with max_compactible_offset unset, we validate that compactions works as before.
dotnwat
pushed a commit
that referenced
this pull request
Feb 2, 2023
Without this change there is a data race that can result in a
heap-use-after-free.
```
==3190349==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000257420 at pc 0x14995faf26e4 bp 0x7ffd3e04b710 sp 0x7ffd3e04b708
READ of size 8 at 0x60b000257420 thread T0
#0 0x14995faf26e3 in profile_flush_file_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:468:24
#1 0x14995faf0cef in profile_close_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:559:14
#2 0x14995fb0c086 in profile_release /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:514:13
#3 0x14995fa924c6 in k5_os_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:506:9
#4 0x14995f938b87 in krb5_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:294:5
#5 0x14995ffc41c3 in krb5_gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/delete_sec_context.c:87:9
#6 0x14995ff399a6 in gssint_delete_internal_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_glue.c:606:15
#7 0x14995ff25450 in gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_delete_sec_context.c:91:11
#8 0x55e258b9c641 in security::gss::ctx_id::reset() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:170:13
#9 0x55e258b9c408 in security::gss::ctx_id::~ctx_id() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:165:17
#10 0x55e258b9c37c in security::gssapi_authenticator::impl::~impl() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:116:29
#11 0x55e258b9c23c in std::__1::default_delete<security::gssapi_authenticator::impl>::operator()(security::gssapi_authenticator::impl*) const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:54:5
#12 0x55e258b65324 in std::__1::unique_ptr<security::gssapi_authenticator::impl, std::__1::default_delete<security::gssapi_authenticator::impl> >::reset(security::gssapi_authenticator::impl*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:315:7
#13 0x55e258b5f302 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:202:15
#14 0x55e258b11830 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9
#15 0x55e258b11315 in seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:78:20
#16 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14
#17 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9
#18 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9
#19 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16
#20 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31
#21 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12
#22 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16
#23 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16
#24 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#25 0x14995cc29e3f in __libc_start_main csu/../csu/libc-start.c:392:3
#26 0x55e246fc1844 in _start (/home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/bin/redpanda+0x3a678844) (BuildId: 0a81be2927716d279ef7fc96d20cc5d5dfbd1cb2)
0x60b000257420 is located 0 bytes inside of 104-byte region [0x60b000257420,0x60b000257488)
freed by thread T0 here:
#0 0x55e2470443e2 in free /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
#1 0x14995faf3ed4 in profile_free_file_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:552:5
#2 0x14995faf3860 in profile_dereference_data_locked /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:509:9
#3 0x14995faf0b74 in profile_dereference_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:502:5
#4 0x14995faf3fc1 in profile_free_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:523:5
#5 0x14995faf0d38 in profile_close_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:562:5
#6 0x14995fb0c086 in profile_release /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:514:13
#7 0x14995fa924c6 in k5_os_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:506:9
#8 0x14995f938b87 in krb5_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:294:5
#9 0x14995ffc41c3 in krb5_gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/delete_sec_context.c:87:9
#10 0x14995ff399a6 in gssint_delete_internal_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_glue.c:606:15
#11 0x14995ff25450 in gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_delete_sec_context.c:91:11
#12 0x55e258b9c641 in security::gss::ctx_id::reset() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:170:13
#13 0x55e258b9c408 in security::gss::ctx_id::~ctx_id() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:165:17
#14 0x55e258b9c37c in security::gssapi_authenticator::impl::~impl() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:116:29
#15 0x55e258b9c23c in std::__1::default_delete<security::gssapi_authenticator::impl>::operator()(security::gssapi_authenticator::impl*) const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:54:5
#16 0x55e258b65324 in std::__1::unique_ptr<security::gssapi_authenticator::impl, std::__1::default_delete<security::gssapi_authenticator::impl> >::reset(security::gssapi_authenticator::impl*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:315:7
#17 0x55e258b5f302 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:202:15
#18 0x55e258b11830 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9
#19 0x55e258b11315 in seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:78:20
#20 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14
#21 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9
#22 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9
#23 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16
#24 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31
#25 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12
#26 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16
#27 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16
#28 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
previously allocated by thread T4 here:
#0 0x55e24704468e in malloc /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
#1 0x14995faee48d in profile_make_prf_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:150:9
#2 0x14995faf0360 in profile_open_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:233:12
#3 0x14995fb0ab1e in profile_init_flags /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:190:22
#4 0x14995fa90bc0 in os_init_paths /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:387:18
#5 0x14995fa9090e in k5_os_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:438:18
#6 0x14995f936ece in krb5_init_context_profile /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:209:14
#7 0x14995f936871 in krb5_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:139:12
#8 0x14995fffc70b in krb5_gss_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/init_sec_context.c:1047:12
#9 0x14995ffa0719 in kg_accept_krb5 /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:694:12
#10 0x14995ff9e686 in krb5_gss_accept_sec_context_ext /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:1311:12
#11 0x14995ffaaf9c in krb5_gss_accept_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:1340:12
#12 0x14995ff05dbd in gss_accept_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_accept_sec_context.c:266:15
#13 0x55e258b2b157 in security::gssapi_authenticator::impl::more(std::__1::basic_string_view<unsigned char, std::__1::char_traits<unsigned char> >) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:292:25
#14 0x55e258b2946c in security::gssapi_authenticator::impl::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:218:16
#15 0x55e258b3b562 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>)::$_0::operator()() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:195:25
#16 0x55e258b3a96a in ssx::impl::worker_task<security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>)::$_0>::process(seastar::alien::instance&, unsigned int) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:67:37
#17 0x55e24764d1bc in ssx::impl::thread_worker::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:159:20
#18 0x55e24764ca9f in ssx::impl::thread_worker::start()::'lambda'()::operator()() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:104:49
#19 0x55e24764c910 in decltype(static_cast<ssx::impl::thread_worker::start()::'lambda'()>(fp)()) std::__1::__invoke<ssx::impl::thread_worker::start()::'lambda'()>(ssx::impl::thread_worker::start()::'lambda'()&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/type_traits:3640:23
#20 0x55e24764c808 in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()>&, std::__1::__tuple_indices<>) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:282:5
#21 0x55e24764ad39 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()> >(void*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:293:5
#22 0x14995cc94b42 in start_thread nptl/./nptl/pthread_create.c:442:8
Thread T4 created by T0 here:
#0 0x55e24702db0c in pthread_create /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_interceptors.cpp:208:3
#1 0x55e24764ab0c in std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__threading_support:375:10
#2 0x55e24764a106 in std::__1::thread::thread<ssx::impl::thread_worker::start()::'lambda'(), void>(ssx::impl::thread_worker::start()::'lambda'()&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:309:16
#3 0x55e247606d40 in ssx::impl::thread_worker::start() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:104:19
#4 0x55e247305329 in ssx::thread_worker::start() (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:199:15
#5 0x55e24756b680 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<void>::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9
#6 0x55e24756b1a5 in seastar::internal::coroutine_traits_base<void>::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:120:20
#7 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14
#8 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9
#9 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9
#10 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16
#11 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31
#12 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12
#13 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16
#14 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16
#15 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
SUMMARY: AddressSanitizer: heap-use-after-free /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:468:24 in profile_flush_file_data
Shadow bytes around the buggy address:
0x0c1680042e30: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa
0x0c1680042e40: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c1680042e50: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c1680042e60: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fd fd
0x0c1680042e70: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa
=>0x0c1680042e80: fa fa fa fa[fd]fd fd fd fd fd fd fd fd fd fd fd
0x0c1680042e90: fd fa fa fa fa fa fa fa fa fa fd fd fd fd fd fd
0x0c1680042ea0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c1680042eb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
0x0c1680042ec0: fa fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd
0x0c1680042ed0: fd fd fd fd fa fa fa fa fa fa fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==3190349==ABORTING
```
Signed-off-by: Ben Pope <[email protected]>
abhijat
pushed a commit
to abhijat/redpanda
that referenced
this pull request
Apr 10, 2023
This segfault is triggering as the test does something innocuous like logging a few string: ``` #0 0x7f875b0bbc1f in std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) (/vectorized/lib/libc++.so.1+0x65c1f) redpanda-data#1 0x5597c3979afc in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) /vectorized/llvm/bin/../include/c++/v1/ostream:722:57 redpanda-data#2 0x5597c4079e44 in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<<<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, std::__1::basic_string_view<char, std::__1::char_traits<char> >) /vectorized/llvm/bin/../include/c++/v1/ostream:1057:12 redpanda-data#3 0x5597c602c9f3 in seastar::logger::do_log(seastar::log_level, seastar::logger::log_writer&) /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/log.cc:342:15 redpanda-data#4 0x5597c3c4ad20 in void seastar::logger::log<char const*, int, seastar::basic_sstring<char, unsigned int, 15u, true>&>(seastar::log_level, seastar::logger::format_info, char const*&&, int&&, seastar::basic_sstring<char, unsigned int, 15u, true>&) /vectorized/include/seastar/util/log.hh:231:17 redpanda-data#5 0x5597c3f23c51 in void seastar::logger::trace<char const*, int, seastar::basic_sstring<char, unsigned int, 15u, true>&>(seastar::logger::format_info, char const*&&, int&&, seastar::basic_sstring<char, unsigned int, 15u, true>&) /vectorized/include/seastar/util/log.hh:385:9 redpanda-data#6 0x5597c3f14b95 in cloud_roles::signature_v4::sign_header(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&, std::__1::basic_string_view<char, std::__1::char_traits<char> >) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/signature.cc:382:5 redpanda-data#7 0x5597c3bf1dfb in cloud_roles::apply_aws_credentials::add_auth(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/apply_aws_credentials.cc:83:23 redpanda-data#8 0x5597c3b8f88d in cloud_roles::apply_credentials::add_auth(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/apply_credentials.h:47:23 redpanda-data#9 0x5597c3b8c578 in test_aws_headers::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/tests/credential_tests.cc:50:17 ``` What is suspicious is that this test is using seastar's logger from a non-seastar reactor thread. Taking a look we do see some things like this_shard_id() being called, but other than that it doesn't look like it should be problematic. But, it's unclear if there is more lurking there. Changed the test to run things that log using seastar logger on a seastar thread. Signed-off-by: Noah Watkins <[email protected]>
dotnwat
added a commit
that referenced
this pull request
Apr 11, 2023
The tee_wrapper helper was being used to set the ostream pointer for global logger to be an ostream instance that lived only on the stack, and not resetting it to something global like std::cerr after the test finished. This resulted in the following segfault in other tests that ran after the change but didn't make their own change to ostream pointer: ``` #0 0x7f875b0bbc1f in std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) (/vectorized/lib/libc++.so.1+0x65c1f) #1 0x5597c3979afc in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) /vectorized/llvm/bin/../include/c++/v1/ostream:722:57 #2 0x5597c4079e44 in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<<<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, std::__1::basic_string_view<char, std::__1::char_traits<char> >) /vectorized/llvm/bin/../include/c++/v1/ostream:1057:12 #3 0x5597c602c9f3 in seastar::logger::do_log(seastar::log_level, seastar::logger::log_writer&) /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/log.cc:342:15 #4 0x5597c3c4ad20 in void seastar::logger::log<char const*, int, seastar::basic_sstring<char, unsigned int, 15u, true>&>(seastar::log_level, seastar::logger::format_info, char const*&&, int&&, seastar::basic_sstring<char, unsigned int, 15u, true>&) /vectorized/include/seastar/util/log.hh:231:17 #5 0x5597c3f23c51 in void seastar::logger::trace<char const*, int, seastar::basic_sstring<char, unsigned int, 15u, true>&>(seastar::logger::format_info, char const*&&, int&&, seastar::basic_sstring<char, unsigned int, 15u, true>&) /vectorized/include/seastar/util/log.hh:385:9 #6 0x5597c3f14b95 in cloud_roles::signature_v4::sign_header(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&, std::__1::basic_string_view<char, std::__1::char_traits<char> >) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/signature.cc:382:5 #7 0x5597c3bf1dfb in cloud_roles::apply_aws_credentials::add_auth(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/apply_aws_credentials.cc:83:23 #8 0x5597c3b8f88d in cloud_roles::apply_credentials::add_auth(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/apply_credentials.h:47:23 #9 0x5597c3b8c578 in test_aws_headers::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/tests/credential_tests.cc:50:17 ``` This change resets the ostream pointer in the wrapper destructor. Signed-off-by: Noah Watkins <[email protected]>
ballard26
referenced
this pull request
in ballard26/redpanda
May 9, 2023
The tee_wrapper helper was being used to set the ostream pointer for global logger to be an ostream instance that lived only on the stack, and not resetting it to something global like std::cerr after the test finished. This resulted in the following segfault in other tests that ran after the change but didn't make their own change to ostream pointer: ``` #0 0x7f875b0bbc1f in std::__1::basic_ostream<char, std::__1::char_traits<char> >::sentry::sentry(std::__1::basic_ostream<char, std::__1::char_traits<char> >&) (/vectorized/lib/libc++.so.1+0x65c1f) #1 0x5597c3979afc in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) /vectorized/llvm/bin/../include/c++/v1/ostream:722:57 redpanda-data#2 0x5597c4079e44 in std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::operator<<<char, std::__1::char_traits<char> >(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, std::__1::basic_string_view<char, std::__1::char_traits<char> >) /vectorized/llvm/bin/../include/c++/v1/ostream:1057:12 redpanda-data#3 0x5597c602c9f3 in seastar::logger::do_log(seastar::log_level, seastar::logger::log_writer&) /v/build/v_deps_build/seastar-prefix/src/seastar/src/util/log.cc:342:15 redpanda-data#4 0x5597c3c4ad20 in void seastar::logger::log<char const*, int, seastar::basic_sstring<char, unsigned int, 15u, true>&>(seastar::log_level, seastar::logger::format_info, char const*&&, int&&, seastar::basic_sstring<char, unsigned int, 15u, true>&) /vectorized/include/seastar/util/log.hh:231:17 redpanda-data#5 0x5597c3f23c51 in void seastar::logger::trace<char const*, int, seastar::basic_sstring<char, unsigned int, 15u, true>&>(seastar::logger::format_info, char const*&&, int&&, seastar::basic_sstring<char, unsigned int, 15u, true>&) /vectorized/include/seastar/util/log.hh:385:9 redpanda-data#6 0x5597c3f14b95 in cloud_roles::signature_v4::sign_header(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&, std::__1::basic_string_view<char, std::__1::char_traits<char> >) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/signature.cc:382:5 redpanda-data#7 0x5597c3bf1dfb in cloud_roles::apply_aws_credentials::add_auth(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/apply_aws_credentials.cc:83:23 redpanda-data#8 0x5597c3b8f88d in cloud_roles::apply_credentials::add_auth(boost::beast::http::header<true, boost::beast::http::basic_fields<std::__1::allocator<char> > >&) const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/apply_credentials.h:47:23 redpanda-data#9 0x5597c3b8c578 in test_aws_headers::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-00e717c5e253e2891-1/redpanda/vtools/src/v/cloud_roles/tests/credential_tests.cc:50:17 ``` This change resets the ostream pointer in the wrapper destructor. Signed-off-by: Noah Watkins <[email protected]>
7 tasks
7 tasks
Merged
7 tasks
This was referenced Dec 14, 2023
nvartolomei
added a commit
to nvartolomei/redpanda
that referenced
this pull request
Apr 25, 2024
Segmentation fault on shard 0.
Backtrace:
0x107c353
0x10d0deb
/lib/x86_64-linux-gnu/libc.so.6+0x3c04f
0x9e48ff
0x9e4ee9
0x1074fb8
0x1119eeb
0x111defb
0x1078118
0x1075256
0xf91780
0xf8fb78
0x4e690d
0x102a38f
/lib/x86_64-linux-gnu/libc.so.6+0x89133
/lib/x86_64-linux-gnu/libc.so.6+0x1097db
Backtrace:
[Backtrace #0]
void seastar::backtrace<seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)redpanda-data#1}>(seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/backtrace.hh:64
(inlined by) seastar::backtrace_buffer::append_backtrace() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:824
(inlined by) seastar::print_with_backtrace(seastar::backtrace_buffer&, bool) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:854
(inlined by) seastar::print_with_backtrace(char const*, bool) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:866
seastar::sigsegv_action() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4150
(inlined by) seastar::install_oneshot_signal_handler<11, (void (*)())(&seastar::sigsegv_action)>()::{lambda(int, siginfo_t*, void*)redpanda-data#1}::operator()(int, siginfo_t*, void*) const at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4131
(inlined by) seastar::install_oneshot_signal_handler<11, (void (*)())(&seastar::sigsegv_action)>()::{lambda(int, siginfo_t*, void*)redpanda-data#1}::__invoke(int, siginfo_t*, void*) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4127
/lib/x86_64-linux-gnu/libc.so.6: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=82ce4e6e4ef08fa58a3535f7437bd3e592db5ac0, for GNU/Linux 3.2.0, stripped
__GI___sigaction at :?
std::__1::unique_ptr<storage::log_reader, std::__1::default_delete<storage::log_reader> >::operator->[abi:v160004]() const at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:274
(inlined by) storage::readers_cache::maybe_evict()::$_0::operator()(storage::readers_cache::entry&) const at /home/nv/redpanda/src/v/storage/readers_cache.cc:351
(inlined by) seastar::future<void> storage::readers_cache::evict_if<storage::readers_cache::maybe_evict()::$_0>(storage::readers_cache::maybe_evict()::$_0) at /home/nv/redpanda/src/v/storage/readers_cache.h:179
(inlined by) storage::readers_cache::maybe_evict() at /home/nv/redpanda/src/v/storage/readers_cache.cc:350
storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}::operator()() const at /home/nv/redpanda/src/v/storage/readers_cache.cc:41
(inlined by) seastar::future<void> seastar::futurize<seastar::future<void> >::invoke<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/future.hh:2006
(inlined by) auto seastar::futurize_invoke<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/future.hh:2037
(inlined by) auto seastar::internal::invoke_func_with_gate<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate::holder&&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/gate.hh:309
(inlined by) auto seastar::try_with_gate<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/gate.hh:350
(inlined by) auto ssx::spawn_with_gate_then<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/src/v/ssx/include/ssx/future-util.h:318
(inlined by) void ssx::spawn_with_gate<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/src/v/ssx/include/ssx/future-util.h:332
(inlined by) storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const at /home/nv/redpanda/src/v/storage/readers_cache.cc:40
(inlined by) seastar::noncopyable_function<void ()>::direct_vtable_for<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0>::call(seastar::noncopyable_function<void ()> const*) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/util/noncopyable_function.hh:129
seastar::noncopyable_function<void ()>::operator()() const at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/noncopyable_function.hh:215
(inlined by) void seastar::reactor::complete_timers<seastar::timer_set<seastar::timer<std::__1::chrono::steady_clock>, &seastar::timer<std::__1::chrono::steady_clock>::_link>, boost::intrusive::list<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::member_hook<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::list_member_hook<>, &seastar::timer<std::__1::chrono::steady_clock>::_link> >, seastar::reactor::service_highres_timer()::$_0>(seastar::timer_set<seastar::timer<std::__1::chrono::steady_clock>, &seastar::timer<std::__1::chrono::steady_clock>::_link>&, boost::intrusive::list<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::member_hook<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::list_member_hook<>, &seastar::timer<std::__1::chrono::steady_clock>::_link> >&, seastar::reactor::service_highres_timer()::$_0&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:1621
(inlined by) seastar::reactor::service_highres_timer() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3256
seastar::hrtimer_aio_completion::complete_with(long) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor_backend.cc:411
seastar::reactor_backend_aio::await_events(int, __sigset_t const*) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor_backend.cc:518
(inlined by) seastar::reactor_backend_aio::reap_kernel_completions() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor_backend.cc:550
seastar::reactor::poll_once() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3497
(inlined by) seastar::reactor::do_run()::$_7::operator()() const at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3390
(inlined by) seastar::reactor::do_run() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3418
seastar::reactor::run() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3265
seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:276
seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:167
seastar::testing::test_runner::start_thread(int, char**)::$_0::operator()() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/testing/test_runner.cc:75
(inlined by) decltype (((std::declval<seastar::testing::test_runner::start_thread(int, char**)::$_0&>)())()) std::__1::__invoke[abi:v160004]<seastar::testing::test_runner::start_thread(int, char**)::$_0&>(seastar::testing::test_runner::start_thread(int, char**)::$_0&) at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/invoke.h:394
(inlined by) void std::__1::__invoke_void_return_wrapper<void, true>::__call<seastar::testing::test_runner::start_thread(int, char**)::$_0&>(seastar::testing::test_runner::start_thread(int, char**)::$_0&) at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/invoke.h:487
(inlined by) std::__1::__function::__alloc_func<seastar::testing::test_runner::start_thread(int, char**)::$_0, std::__1::allocator<seastar::testing::test_runner::start_thread(int, char**)::$_0>, void ()>::operator()[abi:v160004]() at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:185
(inlined by) std::__1::__function::__func<seastar::testing::test_runner::start_thread(int, char**)::$_0, std::__1::allocator<seastar::testing::test_runner::start_thread(int, char**)::$_0>, void ()>::operator()() at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:356
std::__1::__function::__value_func<void ()>::operator()[abi:v160004]() const at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:510
(inlined by) std::__1::function<void ()>::operator()() const at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:1156
(inlined by) seastar::posix_thread::start_routine(void*) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:90
start_thread at ./nptl/./nptl/pthread_create.c:442
clone3 at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
nvartolomei
added a commit
to nvartolomei/redpanda
that referenced
this pull request
Apr 25, 2024
```
Segmentation fault on shard 0.
Backtrace:
0x107c353
0x10d0deb
/lib/x86_64-linux-gnu/libc.so.6+0x3c04f
0x9e48ff
0x9e4ee9
0x1074fb8
0x1119eeb
0x111defb
0x1078118
0x1075256
0xf91780
0xf8fb78
0x4e690d
0x102a38f
/lib/x86_64-linux-gnu/libc.so.6+0x89133
/lib/x86_64-linux-gnu/libc.so.6+0x1097db
Backtrace:
[Backtrace #0]
void seastar::backtrace<seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)redpanda-data#1}>(seastar::backtrace_buffer::append_backtrace()::{lambda(seastar::frame)redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/backtrace.hh:64
(inlined by) seastar::backtrace_buffer::append_backtrace() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:824
(inlined by) seastar::print_with_backtrace(seastar::backtrace_buffer&, bool) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:854
(inlined by) seastar::print_with_backtrace(char const*, bool) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:866
seastar::sigsegv_action() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4150
(inlined by) seastar::install_oneshot_signal_handler<11, (void (*)())(&seastar::sigsegv_action)>()::{lambda(int, siginfo_t*, void*)redpanda-data#1}::operator()(int, siginfo_t*, void*) const at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4131
(inlined by) seastar::install_oneshot_signal_handler<11, (void (*)())(&seastar::sigsegv_action)>()::{lambda(int, siginfo_t*, void*)redpanda-data#1}::__invoke(int, siginfo_t*, void*) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:4127
/lib/x86_64-linux-gnu/libc.so.6: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=82ce4e6e4ef08fa58a3535f7437bd3e592db5ac0, for GNU/Linux 3.2.0, stripped
__GI___sigaction at :?
std::__1::unique_ptr<storage::log_reader, std::__1::default_delete<storage::log_reader> >::operator->[abi:v160004]() const at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:274
(inlined by) storage::readers_cache::maybe_evict()::$_0::operator()(storage::readers_cache::entry&) const at /home/nv/redpanda/src/v/storage/readers_cache.cc:351
(inlined by) seastar::future<void> storage::readers_cache::evict_if<storage::readers_cache::maybe_evict()::$_0>(storage::readers_cache::maybe_evict()::$_0) at /home/nv/redpanda/src/v/storage/readers_cache.h:179
(inlined by) storage::readers_cache::maybe_evict() at /home/nv/redpanda/src/v/storage/readers_cache.cc:350
storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}::operator()() const at /home/nv/redpanda/src/v/storage/readers_cache.cc:41
(inlined by) seastar::future<void> seastar::futurize<seastar::future<void> >::invoke<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/future.hh:2006
(inlined by) auto seastar::futurize_invoke<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/future.hh:2037
(inlined by) auto seastar::internal::invoke_func_with_gate<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate::holder&&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/gate.hh:309
(inlined by) auto seastar::try_with_gate<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/core/gate.hh:350
(inlined by) auto ssx::spawn_with_gate_then<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/src/v/ssx/include/ssx/future-util.h:318
(inlined by) void ssx::spawn_with_gate<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}>(seastar::gate&, storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const::{lambda()redpanda-data#1}&&) at /home/nv/redpanda/src/v/ssx/include/ssx/future-util.h:332
(inlined by) storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0::operator()() const at /home/nv/redpanda/src/v/storage/readers_cache.cc:40
(inlined by) seastar::noncopyable_function<void ()>::direct_vtable_for<storage::readers_cache::readers_cache(model::ntp, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> >, config::binding<unsigned long>)::$_0>::call(seastar::noncopyable_function<void ()> const*) at /home/nv/redpanda/vbuild/release/clang/rp_deps_install/include/seastar/util/noncopyable_function.hh:129
seastar::noncopyable_function<void ()>::operator()() const at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/include/seastar/util/noncopyable_function.hh:215
(inlined by) void seastar::reactor::complete_timers<seastar::timer_set<seastar::timer<std::__1::chrono::steady_clock>, &seastar::timer<std::__1::chrono::steady_clock>::_link>, boost::intrusive::list<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::member_hook<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::list_member_hook<>, &seastar::timer<std::__1::chrono::steady_clock>::_link> >, seastar::reactor::service_highres_timer()::$_0>(seastar::timer_set<seastar::timer<std::__1::chrono::steady_clock>, &seastar::timer<std::__1::chrono::steady_clock>::_link>&, boost::intrusive::list<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::member_hook<seastar::timer<std::__1::chrono::steady_clock>, boost::intrusive::list_member_hook<>, &seastar::timer<std::__1::chrono::steady_clock>::_link> >&, seastar::reactor::service_highres_timer()::$_0&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:1621
(inlined by) seastar::reactor::service_highres_timer() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3256
seastar::hrtimer_aio_completion::complete_with(long) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor_backend.cc:411
seastar::reactor_backend_aio::await_events(int, __sigset_t const*) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor_backend.cc:518
(inlined by) seastar::reactor_backend_aio::reap_kernel_completions() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor_backend.cc:550
seastar::reactor::poll_once() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3497
(inlined by) seastar::reactor::do_run()::$_7::operator()() const at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3390
(inlined by) seastar::reactor::do_run() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3418
seastar::reactor::run() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:3265
seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:276
seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:167
seastar::testing::test_runner::start_thread(int, char**)::$_0::operator()() at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/testing/test_runner.cc:75
(inlined by) decltype (((std::declval<seastar::testing::test_runner::start_thread(int, char**)::$_0&>)())()) std::__1::__invoke[abi:v160004]<seastar::testing::test_runner::start_thread(int, char**)::$_0&>(seastar::testing::test_runner::start_thread(int, char**)::$_0&) at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/invoke.h:394
(inlined by) void std::__1::__invoke_void_return_wrapper<void, true>::__call<seastar::testing::test_runner::start_thread(int, char**)::$_0&>(seastar::testing::test_runner::start_thread(int, char**)::$_0&) at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/invoke.h:487
(inlined by) std::__1::__function::__alloc_func<seastar::testing::test_runner::start_thread(int, char**)::$_0, std::__1::allocator<seastar::testing::test_runner::start_thread(int, char**)::$_0>, void ()>::operator()[abi:v160004]() at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:185
(inlined by) std::__1::__function::__func<seastar::testing::test_runner::start_thread(int, char**)::$_0, std::__1::allocator<seastar::testing::test_runner::start_thread(int, char**)::$_0>, void ()>::operator()() at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:356
std::__1::__function::__value_func<void ()>::operator()[abi:v160004]() const at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:510
(inlined by) std::__1::function<void ()>::operator()() const at /home/nv/redpanda/vbuild/llvm/install/bin/../include/c++/v1/__functional/function.h:1156
(inlined by) seastar::posix_thread::start_routine(void*) at /home/nv/redpanda/vbuild/release/clang/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:90
start_thread at ./nptl/./nptl/pthread_create.c:442
clone3 at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
```
7 tasks
dotnwat
added a commit
that referenced
this pull request
Sep 26, 2024
=================================================================
==524==ERROR: AddressSanitizer: container-overflow on address 0x52100028a100 at pc 0x7f29c0bf51fd bp 0x7ffda75397d0 sp 0x7ffda75397c8
READ of size 4 at 0x52100028a100 thread T0
#0 0x7f29c0bf51fc in util::mem_tracker::pretty_print_ascii() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tracking_allocator.cc:75:31
#1 0x55be1ac7eb7b in mem_tracker_pretty_printing::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:82:5
#2 0x55be1ac7d651 in mem_tracker_pretty_printing_invoker() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:50:1
Signed-off-by: Noah Watkins <[email protected]>
vbotbuildovich
pushed a commit
to vbotbuildovich/redpanda
that referenced
this pull request
Sep 26, 2024
=================================================================
==524==ERROR: AddressSanitizer: container-overflow on address 0x52100028a100 at pc 0x7f29c0bf51fd bp 0x7ffda75397d0 sp 0x7ffda75397c8
READ of size 4 at 0x52100028a100 thread T0
#0 0x7f29c0bf51fc in util::mem_tracker::pretty_print_ascii() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tracking_allocator.cc:75:31
redpanda-data#1 0x55be1ac7eb7b in mem_tracker_pretty_printing::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:82:5
redpanda-data#2 0x55be1ac7d651 in mem_tracker_pretty_printing_invoker() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:50:1
Signed-off-by: Noah Watkins <[email protected]>
(cherry picked from commit 71c352c)
vbotbuildovich
pushed a commit
to vbotbuildovich/redpanda
that referenced
this pull request
Sep 26, 2024
=================================================================
==524==ERROR: AddressSanitizer: container-overflow on address 0x52100028a100 at pc 0x7f29c0bf51fd bp 0x7ffda75397d0 sp 0x7ffda75397c8
READ of size 4 at 0x52100028a100 thread T0
#0 0x7f29c0bf51fc in util::mem_tracker::pretty_print_ascii() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tracking_allocator.cc:75:31
redpanda-data#1 0x55be1ac7eb7b in mem_tracker_pretty_printing::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:82:5
redpanda-data#2 0x55be1ac7d651 in mem_tracker_pretty_printing_invoker() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:50:1
Signed-off-by: Noah Watkins <[email protected]>
(cherry picked from commit 71c352c)
vbotbuildovich
pushed a commit
to vbotbuildovich/redpanda
that referenced
this pull request
Sep 26, 2024
=================================================================
==524==ERROR: AddressSanitizer: container-overflow on address 0x52100028a100 at pc 0x7f29c0bf51fd bp 0x7ffda75397d0 sp 0x7ffda75397c8
READ of size 4 at 0x52100028a100 thread T0
#0 0x7f29c0bf51fc in util::mem_tracker::pretty_print_ascii() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tracking_allocator.cc:75:31
redpanda-data#1 0x55be1ac7eb7b in mem_tracker_pretty_printing::test_method() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:82:5
redpanda-data#2 0x55be1ac7d651 in mem_tracker_pretty_printing_invoker() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-024f8676eb7b139e9-1/redpanda/vtools/src/v/utils/tests/tracking_allocator_tests.cc:50:1
Signed-off-by: Noah Watkins <[email protected]>
(cherry picked from commit 71c352c)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.