[fix](es catalog) fix issue with select and insert from es catalog core #24318
Conversation
qidaye
changed the title
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run beut |
|
TeamCity be ut coverage result: |
|
(From new machine)TeamCity pipeline, clickbench performance test result: |
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
…re (#24318) Issue Number: close #24315 The root cause of this issue is that Elasticsearch's long type allows inserting floats and strings. Doris did not handle these cases when doing type conversion. The current strategy is to take the integer before the decimal point if a float or string is found.
…alog core (#39585) ## Proposed changes Issue Number: pick #24318 ``` (gdb) #0 0x00007fb77df2b387 in raise () from /lib64/libc.so.6 #1 0x00007fb77df2ca78 in abort () from /lib64/libc.so.6 #2 0x000056189ef8e1d9 in ?? () #3 0x000056189ef837ed in google::LogMessage::Fail() () #4 0x000056189ef85d29 in google::LogMessage::SendToLog() () #5 0x000056189ef83356 in google::LogMessage::Flush() () #6 0x000056189ef86399 in google::LogMessageFatal::~LogMessageFatal() () #7 0x000056189ac09e08 in doris::vectorized::IColumn::append_data_by_selector_impl<doris::vectorized::ColumnNullable> (selector=..., res=..., this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/ext/new_allocator.h:89 #8 doris::vectorized::ColumnNullable::append_data_by_selector (this=<optimized out>, res=..., selector=...) at /data/TCHouse-D-1.2/be/src/vec/columns/column_nullable.h:201 #9 0x000056189acab325 in doris::vectorized::Block::append_block_by_selector (this=this@entry=0x7fb6a9e3a510, columns=..., selector=...) at /var/local/ldb-toolchain/include/c++/11/bits/stl_vector.h:1043 #10 0x000056189e90e9a8 in doris::stream_load::VNodeChannel::add_block (this=0x56198399b090, block=0x7fb6a9e3a510, payload=...) at /data/TCHouse-D-1.2/be/src/vec/core/block.h:421 #11 0x000056189e914840 in doris::stream_load::VOlapTableSink::send (this=<optimized out>, state=<optimized out>, input_block=<optimized out>) at /data/TCHouse-D-1.2/be/src/vec/sink/vtablet_sink.cpp:608 #12 0x0000561899b13261 in doris::PlanFragmentExecutor::open_vectorized_internal (this=this@entry=0x56199fdaaf80) at /data/TCHouse-D-1.2/be/src/runtime/plan_fragment_executor.cpp:322 #13 0x0000561899b1423e in doris::PlanFragmentExecutor::open (this=this@entry=0x56199fdaaf80) at /data/TCHouse-D-1.2/be/src/runtime/plan_fragment_executor.cpp:261 #14 0x0000561899aec9a0 in doris::FragmentExecState::execute (this=0x56199fdaaf00) at /data/TCHouse-D-1.2/be/src/runtime/fragment_mgr.cpp:260 #15 0x0000561899aeffae in doris::FragmentMgr::_exec_actual(std::shared_ptr<doris::FragmentExecState>, std::function<void (doris::PlanFragmentExecutor*)>) (this=this@entry=0x5618a9b11400, exec_state=..., cb=...) at /var/local/ldb-toolchain/include/c++/11/bits/shared_ptr_base.h:1290 #16 0x0000561899af04e2 in operator() (__closure=<optimized out>) at /data/TCHouse-D-1.2/be/src/runtime/fragment_mgr.cpp:737 #17 std::__invoke_impl<void, doris::FragmentMgr::exec_plan_fragment(const doris::TExecPlanFragmentParams&, doris::FragmentMgr::FinishCallback)::<lambda()>&> (__f=...) at /var/local/ldb-toolchain/include/c++/11/bits/invoke.h:61 #18 std::__invoke_r<void, doris::FragmentMgr::exec_plan_fragment(const doris::TExecPlanFragmentParams&, doris::FragmentMgr::FinishCallback)::<lambda()>&> (__fn=...) at /var/local/ldb-toolchain/include/c++/11/bits/invoke.h:111 #19 std::_Function_handler<void(), doris::FragmentMgr::exec_plan_fragment(const doris::TExecPlanFragmentParams&, doris::FragmentMgr::FinishCallback)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:291 #20 0x0000561899d9fcb5 in std::function<void ()>::operator()() const (this=<optimized out>) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:560 #21 doris::FunctionRunnable::run (this=<optimized out>) at /data/TCHouse-D-1.2/be/src/util/threadpool.cpp:46 #22 doris::ThreadPool::dispatch_thread (this=0x5618ab4e0700) at /data/TCHouse-D-1.2/be/src/util/threadpool.cpp:535 #23 0x0000561899d9510f in std::function<void ()>::operator()() const (this=0x5618a9e02ef8) at /var/local/ldb-toolchain/include/c++/11/bits/std_function.h:560 #24 doris::Thread::supervise_thread (arg=0x5618a9e02ee0) at /data/TCHouse-D-1.2/be/src/util/thread.cpp:454 #25 0x00007fb77dce0ea5 in start_thread () from /lib64/libpthread.so.0 #26 0x00007fb77dff39fd in clone () from /lib64/libc.so.6 ``` <!--Describe your changes.-->
Proposed changes
Issue Number: close #24315
The root cause of this issue is that Elasticsearch's long type allows inserting floats and strings. Doris did not handle these cases when doing type conversion. The current strategy is to take the integer before the decimal point if a float or string is found.
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...