[Bug] heap-use-after-free when using type array<string>

[eldenmoon]

Search before asking

• I had searched in the issues and found no similar issues.

Version

master

What's Wrong?

after enable asan,

heap-use-after-free when using type array<string>

==3608077==ERROR: AddressSanitizer: heap-use-after-free on address 0x6060001ed238 at pc 0x55555b1edd39 bp 0x7fffdf05fdc0 sp 0x7fffdf05fdb0
READ of size 8 at 0x6060001ed238 thread T94 (MemTableFlushTh)
#0 0x55555b1edd38 in doris::Slice::get_size() const /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/slice.h:87
#1 0x55555b77b6f2 in doris::segment_v2::BinaryDictPageBuilder::add(unsigned char const*, unsigned long*) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/binary_dict_page.cpp:73
#2 0x55556005acf5 in doris::segment_v2::ScalarColumnWriter::append_data_in_current_page(unsigned char const**, unsigned long*) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:364
#3 0x55556005a82c in doris::segment_v2::ScalarColumnWriter::append_data(unsigned char const**, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:352
#4 0x555560056ecd in doris::segment_v2::ColumnWriter::append_nullable(unsigned char const*, void const*, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:213
#5 0x555560064dbb in doris::segment_v2::ColumnWriter::append(bool, void*) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.h:102
#6 0x55556005f606 in doris::segment_v2::ArrayColumnWriter::append_data(unsigned char const**, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:600
#7 0x555560057794 in doris::segment_v2::ColumnWriter::append_nullable(unsigned char const*, unsigned char const**, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:242
#8 0x555560057afb in doris::segment_v2::ColumnWriter::append(unsigned char const*, void const*, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/column_writer.cpp:254
#9 0x55555ff07391 in doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:168
#10 0x55555b81cf72 in doris::BetaRowsetWriter::_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_deletedoris::segment_v2::SegmentWriter >) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/beta_rowset_writer.cpp:142
#11 0x55555b820394 in doris::BetaRowsetWriter::flush_single_memtable(doris::vectorized::Block const) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/beta_rowset_writer.cpp:278
#12 0x55555b614592 in doris::MemTable::_do_flush(long&) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable.cpp:354
#13 0x55555b6132e0 in doris::MemTable::flush() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable.cpp:319
#14 0x55555b2e9dc4 in doris::FlushToken::_flush_memtable(std::shared_ptrdoris::MemTable, long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable_flush_executor.cpp:70
#15 0x55555b2f04e9 in void std::__invoke_impl<void, void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>(std::__invoke_memfun_deref, void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9c4e9)
#16 0x55555b2f0108 in std::__invoke_result<void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>::type std::__invoke<void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>(void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9c108)
#17 0x55555b2efe13 in void std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>::__call<void, , 0ul, 1ul, 2ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul>) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9be13)
#18 0x55555b2efaf4 in void std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>::operator()<, void>() /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:503
#19 0x55555b2ef52d in void std::__invoke_impl<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>(std::__invoke_other, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:61
#20 0x55555b2eefab in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>(std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:111
#21 0x55555b2eea68 in std::_Function_handler<void (), std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)> >::_M_invoke(std::_Any_data const&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:291
#22 0x55555b9bb0b9 in std::function<void ()>::operator()() const /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:560
#23 0x55555c0704b5 in doris::FunctionRunnable::run() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:45
#24 0x55555c06b762 in doris::ThreadPool::dispatch_thread() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:540
#25 0x55555c08bfe3 in void std::__invoke_impl<void, void (doris::ThreadPool::&)(), doris::ThreadPool&>(std::__invoke_memfun_deref, void (doris::ThreadPool::&)(), doris::ThreadPool&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:74
#26 0x55555c08b882 in std::__invoke_result<void (doris::ThreadPool::&)(), doris::ThreadPool&>::type std::__invoke<void (doris::ThreadPool::&)(), doris::ThreadPool&>(void (doris::ThreadPool::&)(), doris::ThreadPool&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:96
#27 0x55555c08ac21 in void std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:420
#28 0x55555c089724 in void std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>::operator()<, void>() /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:503
#29 0x55555c08628b in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:61
#30 0x55555c083baf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&>(std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:111
#31 0x55555c07ef00 in std::_Function_handler<void (), std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()> >::_M_invoke(std::_Any_data const&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:291
#32 0x55555b9bb0b9 in std::function<void ()>::operator()() const /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:560
#33 0x55555c0504bb in doris::Thread::supervise_thread(void*) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/thread.cpp:408
#34 0x7ffff6e4d1ce in start_thread (/lib64/libpthread.so.0+0x81ce)
#35 0x7ffff709ed82 in clone (/lib64/libc.so.6+0x39d82)

0x6060001ed238 is located 24 bytes inside of 64-byte region [0x6060001ed220,0x6060001ed260)
freed by thread T94 (MemTableFlushTh) here:
#0 0x55555aa49888 in realloc (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x54f5888)
#1 0x55555af32239 in Allocator<false, false>::realloc(void*, unsigned long, unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/allocator.h:125
#2 0x55555b34bbb3 in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::realloc<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:147
#3 0x55555b342682 in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::reserve<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:213
#4 0x55555caeedda in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::resize<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:219
#5 0x555560e5e8ec in doris::vectorized::OlapBlockDataConvertor::OlapColumnDataConvertorVarChar::set_source_column(doris::vectorized::ColumnWithTypeAndName const&, unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/olap/olap_data_convertor.cpp:413
#6 0x555560e6183a in doris::vectorized::OlapBlockDataConvertor::OlapColumnDataConvertorArray::convert_to_olap() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/olap/olap_data_convertor.cpp:667
#7 0x555560e5c091 in doris::vectorized::OlapBlockDataConvertor::convert_column_data(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/olap/olap_data_convertor.cpp:116
#8 0x55555ff070ab in doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:161
#9 0x55555b81cf72 in doris::BetaRowsetWriter::_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_deletedoris::segment_v2::SegmentWriter >) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/beta_rowset_writer.cpp:142
#10 0x55555b820394 in doris::BetaRowsetWriter::flush_single_memtable(doris::vectorized::Block const) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/beta_rowset_writer.cpp:278
#11 0x55555b614592 in doris::MemTable::_do_flush(long&) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable.cpp:354
#12 0x55555b6132e0 in doris::MemTable::flush() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable.cpp:319
#13 0x55555b2e9dc4 in doris::FlushToken::_flush_memtable(std::shared_ptrdoris::MemTable, long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable_flush_executor.cpp:70
#14 0x55555b2f04e9 in void std::__invoke_impl<void, void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>(std::__invoke_memfun_deref, void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9c4e9)
#15 0x55555b2f0108 in std::__invoke_result<void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>::type std::__invoke<void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>(void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9c108)
#16 0x55555b2efe13 in void std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>::__call<void, , 0ul, 1ul, 2ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul>) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9be13)
#17 0x55555b2efaf4 in void std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>::operator()<, void>() /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:503
#18 0x55555b2ef52d in void std::__invoke_impl<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>(std::__invoke_other, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:61
#19 0x55555b2eefab in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>(std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:111
#20 0x55555b2eea68 in std::_Function_handler<void (), std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)> >::_M_invoke(std::_Any_data const&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:291
#21 0x55555b9bb0b9 in std::function<void ()>::operator()() const /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:560
#22 0x55555c0704b5 in doris::FunctionRunnable::run() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:45
#23 0x55555c06b762 in doris::ThreadPool::dispatch_thread() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:540
#24 0x55555c08bfe3 in void std::__invoke_impl<void, void (doris::ThreadPool::&)(), doris::ThreadPool&>(std::__invoke_memfun_deref, void (doris::ThreadPool::&)(), doris::ThreadPool&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:74
#25 0x55555c08b882 in std::__invoke_result<void (doris::ThreadPool::&)(), doris::ThreadPool&>::type std::__invoke<void (doris::ThreadPool::&)(), doris::ThreadPool&>(void (doris::ThreadPool::&)(), doris::ThreadPool&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:96
#26 0x55555c08ac21 in void std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:420
#27 0x55555c089724 in void std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>::operator()<, void>() /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:503
#28 0x55555c08628b in void std::__invoke_impl<void, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&>(std::__invoke_other, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:61
#29 0x55555c083baf in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&>(std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:111

previously allocated by thread T94 (MemTableFlushTh) here:
#0 0x55555aa494d7 in malloc (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x54f54d7)
#1 0x55555af1d87d in Allocator<false, false>::alloc_no_track(unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/allocator.h:223
#2 0x55555af0dfde in Allocator<false, false>::alloc(unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/allocator.h:104
#3 0x55555b354aa3 in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::alloc<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:120
#4 0x55555b34baec in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::realloc<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:139
#5 0x55555b342682 in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::reserve<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:213
#6 0x55555caeedda in void doris::vectorized::PODArrayBase<16ul, 4096ul, Allocator<false, false>, 15ul, 16ul>::resize<>(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/common/pod_array.h:219
#7 0x555560e5e8ec in doris::vectorized::OlapBlockDataConvertor::OlapColumnDataConvertorVarChar::set_source_column(doris::vectorized::ColumnWithTypeAndName const&, unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/olap/olap_data_convertor.cpp:413
#8 0x555560e6183a in doris::vectorized::OlapBlockDataConvertor::OlapColumnDataConvertorArray::convert_to_olap() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/olap/olap_data_convertor.cpp:667
#9 0x555560e5c091 in doris::vectorized::OlapBlockDataConvertor::convert_column_data(unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/vec/olap/olap_data_convertor.cpp:116
#10 0x55555ff070ab in doris::segment_v2::SegmentWriter::append_block(doris::vectorized::Block const*, unsigned long, unsigned long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:161
#11 0x55555b81cf72 in doris::BetaRowsetWriter::_add_block(doris::vectorized::Block const*, std::unique_ptr<doris::segment_v2::SegmentWriter, std::default_deletedoris::segment_v2::SegmentWriter >) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/beta_rowset_writer.cpp:142
#12 0x55555b820394 in doris::BetaRowsetWriter::flush_single_memtable(doris::vectorized::Block const) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/rowset/beta_rowset_writer.cpp:278
#13 0x55555b614592 in doris::MemTable::_do_flush(long&) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable.cpp:354
#14 0x55555b6132e0 in doris::MemTable::flush() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable.cpp:319
#15 0x55555b2e9dc4 in doris::FlushToken::_flush_memtable(std::shared_ptrdoris::MemTable, long) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable_flush_executor.cpp:70
#16 0x55555b2f04e9 in void std::__invoke_impl<void, void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>(std::__invoke_memfun_deref, void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9c4e9)
#17 0x55555b2f0108 in std::__invoke_result<void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>::type std::__invoke<void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&>(void (doris::FlushToken::&)(std::shared_ptrdoris::MemTable, long), doris::FlushToken&, std::shared_ptrdoris::MemTable&, long&) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9c108)
#18 0x55555b2efe13 in void std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>::__call<void, , 0ul, 1ul, 2ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul, 2ul>) (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5d9be13)
#19 0x55555b2efaf4 in void std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>::operator()<, void>() /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:503
#20 0x55555b2ef52d in void std::__invoke_impl<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>(std::__invoke_other, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:61
#21 0x55555b2eefab in std::enable_if<is_invocable_r_v<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>, void>::type std::__invoke_r<void, std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&>(std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)>&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:111
#22 0x55555b2eea68 in std::_Function_handler<void (), std::_Bind<void (doris::FlushToken::(doris::FlushToken, std::shared_ptrdoris::MemTable, long))(std::shared_ptrdoris::MemTable, long)> >::_M_invoke(std::_Any_data const&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:291
#23 0x55555b9bb0b9 in std::function<void ()>::operator()() const /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/std_function.h:560
#24 0x55555c0704b5 in doris::FunctionRunnable::run() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:45
#25 0x55555c06b762 in doris::ThreadPool::dispatch_thread() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:540
#26 0x55555c08bfe3 in void std::__invoke_impl<void, void (doris::ThreadPool::&)(), doris::ThreadPool&>(std::__invoke_memfun_deref, void (doris::ThreadPool::&)(), doris::ThreadPool&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:74
#27 0x55555c08b882 in std::__invoke_result<void (doris::ThreadPool::&)(), doris::ThreadPool&>::type std::__invoke<void (doris::ThreadPool::&)(), doris::ThreadPool&>(void (doris::ThreadPool::&)(), doris::ThreadPool&) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/bits/invoke.h:96
#28 0x55555c08ac21 in void std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>::__call<void, , 0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:420
#29 0x55555c089724 in void std::_Bind<void (doris::ThreadPool::(doris::ThreadPool))()>::operator()<, void>() /mnt/disk1/lihangyu/ldb_toolchain/include/c++/11/functional:503

Thread T94 (MemTableFlushTh) created by T0 here:
#0 0x55555a9ed771 in __interceptor_pthread_create (/mnt/disk1/lihangyu/eldenmoon/doris/test_env/be/lib/palo_be+0x5499771)
#1 0x55555c04f87f in doris::Thread::start_thread(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::function<void ()> const&, unsigned long, scoped_refptrdoris::Thread) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/thread.cpp:362
#2 0x55555c074997 in doris::Status doris::Thread::create<void (doris::ThreadPool::)(), doris::ThreadPool*>(std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, std::__cxx11::basic_string<char, std::char_traits, std::allocator > const&, void (doris::ThreadPool::* const&)(), doris::ThreadPool* const&, scoped_refptrdoris::Thread) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/thread.h:54
#3 0x55555c06cf82 in doris::ThreadPool::create_thread() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:603
#4 0x55555c06671c in doris::ThreadPool::init() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:266
#5 0x55555c062fd6 in doris::ThreadPoolBuilder::build(std::unique_ptr<doris::ThreadPool, std::default_deletedoris::ThreadPool >) const /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/threadpool.cpp:77
#6 0x55555b2ea7d2 in doris::MemTableFlushExecutor::init(std::vector<doris::DataDir*, std::allocatordoris::DataDir* > const&) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/memtable_flush_executor.cpp:93
#7 0x55555b00c7f8 in doris::StorageEngine::_open() /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/storage_engine.cpp:205
#8 0x55555b008324 in doris::StorageEngine::open(doris::EngineOptions const&, doris::StorageEngine**) /mnt/disk1/lihangyu/eldenmoon/doris/be/src/olap/storage_engine.cpp:108
#9 0x55555aa93aac in main /mnt/disk1/lihangyu/eldenmoon/doris/be/src/service/doris_main.cpp:391
#10 0x7ffff709fca2 in __libc_start_main (/lib64/libc.so.6+0x3aca2)

SUMMARY: AddressSanitizer: heap-use-after-free /mnt/disk1/lihangyu/eldenmoon/doris/be/src/util/slice.h:87 in doris::Slice::get_size() const
Shadow bytes around the buggy address:
0x0c0c800359f0: 00 00 00 00 00 00 00 00 fa fa fa fa fd fd fd fd
0x0c0c80035a00: fd fd fd fd fa fa fa fa fd fd fd fd fd fd fd fa
0x0c0c80035a10: fa fa fa fa 00 00 00 00 00 00 00 00 fa fa fa fa
0x0c0c80035a20: fd fd fd fd fd fd fd fa fa fa fa fa 00 00 00 00
0x0c0c80035a30: 00 00 00 00 fa fa fa fa fd fd fd fd fd fd fd fd
=>0x0c0c80035a40: fa fa fa fa fd fd fd[fd]fd fd fd fd fa fa fa fa
0x0c0c80035a50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0c80035a60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0c80035a70: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0c80035a80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0c80035a90: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==3608077==ABORTING

What You Expected?

array<string> works

How to Reproduce?

mysql> create table test_array_string (k1 INT, k2 INT, k3 array<string>) ENGINE=olap DUPLICATE KEY(k1, k2) PARTITION BY RANGE (k1) (partition `p1` values less than ("1000"), partition `p2` values less than ("2000"),partition `p3` values less than ("3000"))  DISTRIBUTED BY HASH(k2) BUCKETS 3 PROPERTIES("replication_num" = "1");
Query OK, 0 rows affected (0.02 sec)

mysql> set enable_vectorized_engine = true;
Query OK, 0 rows affected (0.00 sec)

mysql> insert into test_array_string  values(1, 2, ["a", "b", "c"]),(1, 2, ["a", "b"]),(1, 2, ["a", "xxqwdqw", "c"]),(1, 2, ["a", "b"]),(1, 2, ["a", "b", "c"]),(1, 2, ["a", "b", "c"]),(1, 2, ["a", "b", "c"]),(1, 2, ["a", "b"]),(1, 2, ["a", "b", "c", "d", "e"]),(1, 2, ["a", "b", "c"]), (1, 2, ["a", "b", "cdsdasd"]),(1, 2, [ "dwdjwo"]),(1, 2, ["a", "b", "c"]),(1, 2, ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j"]), (1, 2, ["a", "cdsdasd"]), (1, 2, ["cdsdasd", "lgyu2", "3", "4", "5"]);

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[adonis0147]

I will investigate this issue.

[adonis0147]

Hi @eldenmoon , I have reproduced this issue. That is a good catch!

[adonis0147]

Hi @eldenmoon , I just submitted a pr #10127 to resolve this issue, PTAL.

[eldenmoon]

thanks a lot @adonis0147 ， I will test it as soon as posible