Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI Failure (ASAN Stack overflow) in id_allocator_stm_test_rpunit.stm_monotonicity_test #13491

Closed
VladLazar opened this issue Sep 18, 2023 · 3 comments · Fixed by #13544
Closed
Assignees
Labels
ci-failure kind/bug Something isn't working rpunit unit test ci-failure (not ducktape)

Comments

@VladLazar
Copy link
Contributor

https://buildkite.com/redpanda/redpanda/builds/37157

Entering test module "Master Test Suite"
--
  | /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/cluster/tests/id_allocator_stm_test.cc(58): Entering test case "stm_monotonicity_test"
  | WARNING: debug mode. Not for benchmarking or production
  | WARN  2023-09-18 11:35:28,653 seastar - Seastar compiled with default allocator, --memory option won't take effect
  | DEBUG 2023-09-18 11:35:28,653 seastar - smp::count: 4
  | DEBUG 2023-09-18 11:35:28,653 seastar - latency_goal: 0.00075
  | DEBUG 2023-09-18 11:35:28,724 seastar - Not a cgroups-v2-only system
  | DEBUG 2023-09-18 11:35:28,725 seastar - Assign CPU0 to NUMA0
  | DEBUG 2023-09-18 11:35:28,725 seastar - Assign CPU6 to NUMA0
  | DEBUG 2023-09-18 11:35:28,725 seastar - Assign CPU12 to NUMA0
  | DEBUG 2023-09-18 11:35:28,725 seastar - Assign CPU18 to NUMA0
  | DEBUG 2023-09-18 11:35:28,725 seastar - Auto-configure 1 IO groups
  | INFO  2023-09-18 11:35:28,726 seastar - Reactor backend: linux-aio
  | DEBUG 2023-09-18 11:35:28,734 [shard 1:main] seastar - generate_config dev_id: 0
  | INFO  2023-09-18 11:35:28,735 [shard 1:main] seastar - Created fair group io-queue-0 for 4 queues, capacity rate 2147483:2147483, limit 12582912, rate 16777216 (factor 1), threshold 2000, per tick grab 3145728
  | INFO  2023-09-18 11:35:28,735 [shard 1:main] seastar - IO queue uses 0.75ms latency goal for device 0
  | INFO  2023-09-18 11:35:28,735 [shard 1:main] seastar - Created io group dev(0), length limit 4194304:4194304, rate 2147483647:2147483647
  | DEBUG 2023-09-18 11:35:28,735 [shard 1:main] seastar - allocate 0 IO group with 4 queues, dev 0
  | DEBUG 2023-09-18 11:35:28,735 [shard 1:main] seastar - attached 1 queue to 0 IO group, dev 0
  | DEBUG 2023-09-18 11:35:28,735 [shard 2:main] seastar - attached 2 queue to 0 IO group, dev 0
  | INFO  2023-09-18 11:35:28,736 [shard 0:main] seastar - Created io queue dev(0) capacities: 512:2000:2000 1024:3000:3000 2048:5000:5000 4096:9000:9000 8192:17000:17000 16384:33000:33000 32768:65000:65000 65536:129000:129000 131072:257000:257000
  | DEBUG 2023-09-18 11:35:28,736 [shard 0:main] seastar - attached 0 queue to 0 IO group, dev 0
  | DEBUG 2023-09-18 11:35:28,736 [shard 3:main] seastar - attached 3 queue to 0 IO group, dev 0
  | random-seed=1790443234
  | ==681==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases!
  | AddressSanitizer:DEADLYSIGNAL
  | =================================================================
  | ==681==ERROR: AddressSanitizer: stack-overflow on address 0x7f65f5822e48 (pc 0x565430d121b2 bp 0x7f65f5823690 sp 0x7f65f5822e50 T1)
  | #0 0x565430d121b2 in __asan_memcpy /v/llvm/llvm/src/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:22:3
  | #1 0x565430dd6459 in fmt::v8::appender fmt::v8::detail::write<char, fmt::v8::appender, long long, 0>(fmt::v8::appender, long long) /vectorized/include/fmt/format.h:2099:13
  | #2 0x565430dce5ae in fmt::v8::appender fmt::v8::detail::default_arg_formatter<char>::operator()<long long>(long long) /vectorized/include/fmt/format.h:2197:12
  | #3 0x565430dc9423 in decltype(fp(0)) fmt::v8::visit_format_arg<fmt::v8::detail::default_arg_formatter<char>, fmt::v8::basic_format_context<fmt::v8::appender, char>>(fmt::v8::detail::default_arg_formatter<char>&&, fmt::v8::basic_format_arg<fmt::v8::basic_format_context<fmt::v8::appender, char>> const&) /vectorized/include/fmt/core.h:1597:12
  | #4 0x565430dc9423 in void fmt::v8::detail::vformat_to<char>(fmt::v8::detail::buffer<char>&, fmt::v8::basic_string_view<char>, fmt::v8::basic_format_args<fmt::v8::basic_format_context<std::__1::conditional<std::is_same<fmt::v8::type_identity<char>::type, char>::value, fmt::v8::appender, std::__1::back_insert_iterator<fmt::v8::detail::buffer<fmt::v8::type_identity<char>::type>>>::type, fmt::v8::type_identity<char>::type>>, fmt::v8::detail::locale_ref) /vectorized/include/fmt/format.h:2931:5
  | #5 0x565430ecba40 in void fmt::v8::vprint<char>(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, fmt::v8::basic_string_view<char>, fmt::v8::basic_format_args<fmt::v8::basic_format_context<std::__1::conditional<std::is_same<fmt::v8::type_identity<char>::type, char>::value, fmt::v8::appender, std::__1::back_insert_iterator<fmt::v8::detail::buffer<fmt::v8::type_identity<char>::type>>>::type, fmt::v8::type_identity<char>::type>>) /vectorized/include/fmt/ostream.h:113:3
  | #6 0x565430ecb2ee in void fmt::v8::print<char [3], long long, char>(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, char const (&) [3], long long&&) /vectorized/include/fmt/ostream.h:130:3
  | #7 0x565430eca306 in std::operator<<(std::__1::basic_ostream<char, std::__1::char_traits<char>>&, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000000000l>> const&) /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/utils/to_string.h:46:5
  | #8 0x565431045fba in void fmt::v8::detail::format_value<char, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>>(fmt::v8::detail::buffer<char>&, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>> const&, fmt::v8::detail::locale_ref) /vectorized/include/fmt/ostream.h:77:10
  | #9 0x565431044f80 in fmt::v8::appender fmt::v8::detail::fallback_formatter<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>, char, void>::format<fmt::v8::appender>(std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>> const&, fmt::v8::basic_format_context<fmt::v8::appender, char>&) /vectorized/include/fmt/ostream.h:92:5
  | #10 0x56543104496f in void fmt::v8::detail::value<fmt::v8::basic_format_context<fmt::v8::appender, char>>::format_custom_arg<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>, fmt::v8::detail::fallback_formatter<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>, char, void>>(void*, fmt::v8::basic_format_parse_context<char, fmt::v8::detail::error_handler>&, fmt::v8::basic_format_context<fmt::v8::appender, char>&) /vectorized/include/fmt/core.h:1282:22
  | #11 0x565430e2a309 in fmt::v8::basic_format_arg<fmt::v8::basic_format_context<fmt::v8::appender, char>>::handle::format(fmt::v8::basic_format_parse_context<char, fmt::v8::detail::error_handler>&, fmt::v8::basic_format_context<fmt::v8::appender, char>&) const /vectorized/include/fmt/core.h:1558:7
  | #12 0x565430dd11dc in fmt::v8::detail::default_arg_formatter<char>::operator()(fmt::v8::basic_format_arg<fmt::v8::basic_format_context<fmt::v8::appender, char>>::handle) /vectorized/include/fmt/format.h:2202:7
  | #13 0x565430dcab50 in decltype(fp(0)) fmt::v8::visit_format_arg<fmt::v8::detail::default_arg_formatter<char>, fmt::v8::basic_format_context<fmt::v8::appender, char>>(fmt::v8::detail::default_arg_formatter<char>&&, fmt::v8::basic_format_arg<fmt::v8::basic_format_context<fmt::v8::appender, char>> const&) /vectorized/include/fmt/core.h:1622:12
  | #14 0x565430dcab50 in void fmt::v8::detail::vformat_to<char>(fmt::v8::detail::buffer<char>&, fmt::v8::basic_string_view<char>, fmt::v8::basic_format_args<fmt::v8::basic_format_context<std::__1::conditional<std::is_same<fmt::v8::type_identity<char>::type, char>::value, fmt::v8::appender, std::__1::back_insert_iterator<fmt::v8::detail::buffer<fmt::v8::type_identity<char>::type>>>::type, fmt::v8::type_identity<char>::type>>, fmt::v8::detail::locale_ref) /vectorized/include/fmt/format.h:2931:5
  | #15 0x7f65fbd5981e in fmt::v8::vformat(fmt::v8::basic_string_view<char>, fmt::v8::basic_format_args<fmt::v8::basic_format_context<fmt::v8::appender, char>>) /v/build/v_deps_build/fmt-prefix/src/fmt/include/fmt/format-inl.h:2588:3
  | #16 0x7f660aaec8b8 in std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> fmt::v8::format<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>&>(fmt::v8::basic_format_string<char, fmt::v8::type_identity<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>&>::type>, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>&) /vectorized/include/fmt/core.h:3119:10
  | #17 0x7f660aaec8b8 in config::bounded_property<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>, config::numeric_integral_bounds, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>>::generate_example() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/config/bounded_property.h:288:16
  | #18 0x7f660a851429 in config::bounded_property<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>, config::numeric_integral_bounds, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>>::bounded_property(config::config_store&, std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::basic_string_view<char, std::__1::char_traits<char>>, config::base_property::metadata, std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>, config::numeric_integral_bounds<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>>, std::__1::optional<config::legacy_default<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l>>>>) /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/config/bounded_property.h:216:18
  | #19 0x7f660a7e46a0 in config::configuration::configuration() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/config/configuration.cc:206:5
  | #20 0x7f660a846b9b in config::shard_local_cfg() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/config/configuration.cc:2508:39
  | #21 0x565431134c8e in simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()::operator()() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/raft/tests/simple_raft_fixture.h:46:13
  | #22 0x56543113466d in seastar::future<void> seastar::futurize<void>::invoke<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&) /vectorized/include/seastar/core/future.hh:2003:13
  | #23 0x565431134367 in seastar::futurize<std::__1::invoke_result<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>::type>::type seastar::smp::submit_to<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(unsigned int, seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&) /vectorized/include/seastar/core/smp.hh:378:28
  | #24 0x5654311340ae in seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)::operator()(unsigned int) const /vectorized/include/seastar/core/smp.hh:433:20
  | #25 0x565431133e5a in seastar::future<void> seastar::futurize<seastar::future<void>>::invoke<seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int), unsigned int>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&, unsigned int&&) /vectorized/include/seastar/core/future.hh:2006:20
  | #26 0x565431133a1f in auto seastar::futurize_invoke<seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int), unsigned int>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&, unsigned int&&) /vectorized/include/seastar/core/future.hh:2037:12
  | #27 0x565431132eaa in seastar::future<void> seastar::parallel_for_each<boost::range_detail::integer_iterator<unsigned int>, boost::range_detail::integer_iterator<unsigned int>, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'(), boost::range_detail::integer_iterator<unsigned int>, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&) /vectorized/include/seastar/core/loop.hh:575:18
  | #28 0x565431132a19 in seastar::future<void> seastar::internal::parallel_for_each_impl<boost::integer_range<unsigned int>, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&) /vectorized/include/seastar/core/loop.hh:628:12
  | #29 0x56543113b77f in seastar::future<void> seastar::futurize<seastar::future<void>>::invoke<seastar::future<void> (*&)(boost::integer_range<unsigned int>&&, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&), boost::integer_range<unsigned int>, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&, boost::integer_range<unsigned int>&&, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&) /vectorized/include/seastar/core/future.hh:2006:20
  | #30 0x565431132b6b in auto seastar::futurize_invoke<seastar::future<void> (*&)(boost::integer_range<unsigned int>&&, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&), boost::integer_range<unsigned int>, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&, boost::integer_range<unsigned int>&&, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&) /vectorized/include/seastar/core/future.hh:2037:12
  | #31 0x56543113238a in seastar::future<void> seastar::parallel_for_each<boost::integer_range<unsigned int>, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&, seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&)::'lambda'(unsigned int)&&) /vectorized/include/seastar/core/loop.hh:643:12
  | #32 0x565431131c49 in seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(seastar::smp_submit_to_options, simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&) /vectorized/include/seastar/core/smp.hh:432:16
  | #33 0x56543112c081 in seastar::future<void> seastar::smp::invoke_on_all<simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()>(simple_raft_fixture::create_raft(storage::ntp_config::default_overrides)::'lambda'()&&) /vectorized/include/seastar/core/smp.hh:447:16
  | #34 0x565431127abd in simple_raft_fixture::create_raft(storage::ntp_config::default_overrides) /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/raft/tests/simple_raft_fixture.h:43:9
  | #35 0x565430d5c8bb in id_allocator_stm_fixture::create_stm_and_start_raft() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/cluster/tests/id_allocator_stm_test.cc:44:9
  | #36 0x565430d510b0 in id_allocator_stm_fixture_stm_monotonicity_test::fixture_test() /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/cluster/tests/id_allocator_stm_test.cc:59:5
  | #37 0x565430d5079f in stm_monotonicity_test::do_run_test_case() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/cluster/tests/id_allocator_stm_test.cc:58:1
  | #38 0x565430d80a1f in stm_monotonicity_test::run_test_case() const::'lambda'()::operator()() const /var/lib/buildkite-agent/builds/buildkite-amd64-builders-i-0a388a7e354f944a6-1/redpanda/redpanda/src/v/cluster/tests/id_allocator_stm_test.cc:58:1
  | #39 0x565430d809b0 in decltype(std::declval<stm_monotonicity_test::run_test_case() const::'lambda'()>()()) std::__1::__invoke[abi:v160004]<stm_monotonicity_test::run_test_case() const::'lambda'()>(stm_monotonicity_test::run_test_case() const::'lambda'()&&) /vectorized/llvm/bin/../include/c++/v1/__functional/invoke.h:394:23
  | #40 0x565430d80954 in decltype(auto) std::__1::__apply_tuple_impl[abi:v160004]<stm_monotonicity_test::run_test_case() const::'lambda'(), std::__1::tuple<>>(stm_monotonicity_test::run_test_case() const::'lambda'()&&, std::__1::tuple<>&&, std::__1::__tuple_indices<>) /vectorized/llvm/bin/../include/c++/v1/tuple:1816:1
  | #41 0x565430d8082a in decltype(auto) std::__1::apply[abi:v160004]<stm_monotonicity_test::run_test_case() const::'lambda'(), std::__1::tuple<>>(stm_monotonicity_test::run_test_case() const::'lambda'()&&, std::__1::tuple<>&&) /vectorized/llvm/bin/../include/c++/v1/tuple:1825:1
  | #42 0x565430d80449 in seastar::future<void> seastar::futurize<void>::apply<stm_monotonicity_test::run_test_case() const::'lambda'()>(stm_monotonicity_test::run_test_case() const::'lambda'()&&, std::__1::tuple<>&&) /vectorized/include/seastar/core/future.hh:1970:13
  | #43 0x565430d802d4 in seastar::futurize<std::__1::invoke_result<stm_monotonicity_test::run_test_case() const::'lambda'()>::type>::type seastar::async<stm_monotonicity_test::run_test_case() const::'lambda'()>(seastar::thread_attributes, stm_monotonicity_test::run_test_case() const::'lambda'()&&)::'lambda'()::operator()() const /vectorized/include/seastar/core/thread.hh:262:13
  | #44 0x565430d800f8 in seastar::noncopyable_function<void ()>::direct_vtable_for<seastar::futurize<std::__1::invoke_result<stm_monotonicity_test::run_test_case() const::'lambda'()>::type>::type seastar::async<stm_monotonicity_test::run_test_case() const::'lambda'()>(seastar::thread_attributes, stm_monotonicity_test::run_test_case() const::'lambda'()&&)::'lambda'()>::call(seastar::noncopyable_function<void ()> const*) /vectorized/include/seastar/util/noncopyable_function.hh:129:20
  | #45 0x7f664197365d in seastar::noncopyable_function<void ()>::operator()() const /vectorized/include/seastar/util/noncopyable_function.hh:215:16
  | #46 0x7f660136b694 in seastar::thread_context::main() /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/thread.cc:311:9
  |  
  | SUMMARY: AddressSanitizer: stack-overflow /v/llvm/llvm/src/compiler-rt/lib/asan/asan_interceptors_memintrinsics.cpp:22:3 in __asan_memcpy
  | Thread T1 created by T0 here:
  | #0 0x565430cfb18a in pthread_create /v/llvm/llvm/src/compiler-rt/lib/asan/asan_interceptors.cpp:208:3
  | #1 0x7f6600acc956 in seastar::posix_thread::posix_thread(seastar::posix_thread::attr, std::__1::function<void ()>) /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:130:9
  | #2 0x7f6600acc286 in seastar::posix_thread::posix_thread(std::__1::function<void ()>) /v/build/v_deps_build/seastar-prefix/src/seastar/src/core/posix.cc:95:7
  | #3 0x7f660d654fbb in std::__1::__unique_if<seastar::posix_thread>::__unique_single std::__1::make_unique[abi:v160004]<seastar::posix_thread, seastar::testing::test_runner::start_thread(int, char**)::$_0>(seastar::testing::test_runner::start_thread(int, char**)::$_0&&) /vectorized/llvm/bin/../include/c++/v1/__memory/unique_ptr.h:686:30
  | #4 0x7f660d654a71 in seastar::testing::test_runner::start_thread(int, char**) /v/build/v_deps_build/seastar-prefix/src/seastar/src/testing/test_runner.cc:68:15
  | #5 0x7f660d6555ce in seastar::testing::test_runner::run_sync(std::__1::function<seastar::future<void> ()>) /v/build/v_deps_build/seastar-prefix/src/seastar/src/testing/test_runner.cc:122:9
  | #6 0x7f660d62f7ee in seastar::testing::seastar_test::run() /v/build/v_deps_build/seastar-prefix/src/seastar/src/testing/seastar_test.cc:50:26
  | #7 0x7f660d63848f in seastar::testing::seastar_test::seastar_test(char const*, char const*, int, boost::unit_test::decorator::collector_t&)::$_0::operator()() const /v/build/v_deps_build/seastar-prefix/src/seastar/src/testing/seastar_test.cc:61:59
  | #8 0x7f660d637978 in boost::detail::function::void_function_obj_invoker0<seastar::testing::seastar_test::seastar_test(char const*, char const*, int, boost::unit_test::decorator::collector_t&)::$_0, void>::invoke(boost::detail::function::function_buffer&) /vectorized/include/boost/function/function_template.hpp:158:11
  | #9 0x7f660cfde6c2 in boost::function0<void>::operator()() const /v/build/v_deps_build/boost-prefix/src/boost/libs/function/include/boost/function/function_template.hpp:771:14
  | #10 0x7f660cfde38f in boost::detail::forward::operator()() /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/execution_monitor.ipp:1388:32
  | #11 0x7f660cfdd768 in boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /v/build/v_deps_build/boost-prefix/src/boost/libs/function/include/boost/function/function_template.hpp:137:18
  | #12 0x7f660cfcb442 in boost::function0<int>::operator()() const /v/build/v_deps_build/boost-prefix/src/boost/libs/function/include/boost/function/function_template.hpp:771:14
  | #13 0x7f660cfc72b3 in int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()>>(boost::shared_ptr<boost::detail::translator_holder_base> const&, boost::function<int ()> const&) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/execution_monitor.ipp:301:30
  | #14 0x7f660cfc23f4 in boost::execution_monitor::catch_signals(boost::function<int ()> const&) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/execution_monitor.ipp:903:16
  | #15 0x7f660cfc2be5 in boost::execution_monitor::execute(boost::function<int ()> const&) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/execution_monitor.ipp:1301:16
  | #16 0x7f660cfc5d30 in boost::execution_monitor::vexecute(boost::function<void ()> const&) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/execution_monitor.ipp:1397:5
  | #17 0x7f660d221997 in boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned long) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/unit_test_monitor.ipp:49:9
  | #18 0x7f660d01c3c3 in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/framework.ipp:815:44
  | #19 0x7f660d018dff in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/framework.ipp:740:54
  | #20 0x7f660cff22ec in boost::unit_test::framework::run(unsigned long, bool) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/framework.ipp:1722:29
  | #21 0x7f660d211521 in boost::unit_test::unit_test_main(bool (*)(), int, char**) /v/build/v_deps_build/boost-prefix/src/boost/libs/test/include/boost/test/impl/unit_test_main.ipp:250:9
  | #22 0x7f660d62e92a in seastar::testing::entry_point(int, char**) /v/build/v_deps_build/seastar-prefix/src/seastar/src/testing/entry_point.cc:63:33
  | #23 0x7f6673f5139a in main /vectorized/include/seastar/testing/seastar_test.hh:75:12
  | #24 0x7f65fb2e658f in __libc_start_call_main (/lib64/libc.so.6+0x2d58f) (BuildId: 6e7b96dfb83f0bdcb6a410469b82f86415e5ada3)


@VladLazar VladLazar added kind/bug Something isn't working ci-failure rpunit unit test ci-failure (not ducktape) labels Sep 18, 2023
@VladLazar VladLazar changed the title CI Failure (ASAN Stack overflow) in id_allocator_stm_test_rpunit.stm_monotonicity_test CI Failure (ASAN Stack overflow) in id_allocator_stm_test_rpunit.stm_monotonicity_test Sep 18, 2023
@VladLazar
Copy link
Contributor Author

This fails when attempting to init the thread_local configuration object in shard_local_cfg(). Not quite sure what's going on, but it's a bit weird that this test uses a separate configuration object.

@mmaslankaprv
Copy link
Member

mmaslankaprv commented Sep 19, 2023

this failure was actually introduced by the scrubbing PR which is very weird

@mmaslankaprv mmaslankaprv self-assigned this Sep 19, 2023
mmaslankaprv added a commit to mmaslankaprv/redpanda that referenced this issue Sep 19, 2023
All other Raft timeouts are configurable in runtime. Made election
timeout runtime configurable as well.

Fixes: redpanda-data#13491

Signed-off-by: Michal Maslanka <[email protected]>
@VladLazar
Copy link
Contributor Author

this failure was actually introduced by the scrubbing PR which is very weird

Hmm. I'll take a look then.

mmaslankaprv added a commit to mmaslankaprv/redpanda that referenced this issue Sep 20, 2023
All other Raft timeouts are configurable in runtime. Made election
timeout runtime configurable as well.

Fixes: redpanda-data#13491

Signed-off-by: Michal Maslanka <[email protected]>
mmaslankaprv added a commit to mmaslankaprv/redpanda that referenced this issue Oct 4, 2023
All other Raft timeouts are configurable in runtime. Made election
timeout runtime configurable as well.

Fixes: redpanda-data#13491

Signed-off-by: Michal Maslanka <[email protected]>
(cherry picked from commit fbc1130)
mmaslankaprv added a commit to mmaslankaprv/redpanda that referenced this issue Oct 5, 2023
All other Raft timeouts are configurable in runtime. Made election
timeout runtime configurable as well.

Fixes: redpanda-data#13491

Signed-off-by: Michal Maslanka <[email protected]>
(cherry picked from commit fbc1130)
mmaslankaprv added a commit to mmaslankaprv/redpanda that referenced this issue Oct 5, 2023
All other Raft timeouts are configurable in runtime. Made election
timeout runtime configurable as well.

Fixes: redpanda-data#13491

Signed-off-by: Michal Maslanka <[email protected]>
(cherry picked from commit fbc1130)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-failure kind/bug Something isn't working rpunit unit test ci-failure (not ducktape)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants