Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry pick bug fixes for PODArray #1357

Merged
merged 1 commit into from
Jan 17, 2021

Conversation

JaySon-Huang
Copy link
Contributor

@JaySon-Huang JaySon-Huang commented Jan 15, 2021

What problem does this PR solve?

Issue Number: maybe related to pingcap/tidb#22184

Problem Summary: TiFlash crashes when inserting data.

What is changed and how it works?

Cherry-pick two commits in Clickhouse to fix bugs for PODArray

Related changes

  • PR to update pingcap/docs/pingcap/docs-cn:
  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test

Side effects

  • N/A

Release note

  • Fix the bug that TiFlash may crash when reading data

@JaySon-Huang JaySon-Huang added needs-cherry-pick-release-4.0 PR which needs to be cherry-picked to release-4.0 type/bugfix This PR fixes a bug. labels Jan 15, 2021
@JaySon-Huang
Copy link
Contributor Author

/run-all-tests

@JaySon-Huang
Copy link
Contributor Author

/run-all-tests

Copy link
Contributor

@flowbehappy flowbehappy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 16, 2021
@JaySon-Huang JaySon-Huang merged commit eefb963 into pingcap:master Jan 17, 2021
@ti-srebot
Copy link
Collaborator

cherry pick to release-4.0 in PR #1358

lidezhu pushed a commit that referenced this pull request Jan 18, 2021
@JaySon-Huang JaySon-Huang deleted the pick_podarray_fix branch January 26, 2021 09:01
@JaySon-Huang
Copy link
Contributor Author

JaySon-Huang commented Jan 26, 2021

The stack frame of coredump is likely to be as following:

2021.01.05 09:40:41.562453 [ 365 ] <Error> DAGDriver: void DB::DAGDriver<batch>::execute() [with bool batch = false]: DB Exception: Allocator: Cannot realloc from 1.00 MiB1048576 to 0.00 B0., errno: 11, strerror: Resource temporarily unavailable
0. bin/tiflash/tiflash(StackTrace::StackTrace()+0x15) [0x3381595]
1. bin/tiflash/tiflash(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x1e) [0x337823e]
2. bin/tiflash/tiflash(DB::throwFromErrno(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int, int)+0x195) [0x337df75]
3. bin/tiflash/tiflash(Allocator<false>::realloc(void*, unsigned long, unsigned long, unsigned long)+0x2b2) [0x339a1d2]
4. bin/tiflash/tiflash() [0x66c6cbc]
5. bin/tiflash/tiflash(DB::ColumnString::popBack(unsigned long)+0x55) [0x66cbcf5]
6. bin/tiflash/tiflash(DB::ColumnNullable::popBack(unsigned long)+0x19) [0x66bfa59]
7. bin/tiflash/tiflash(DB::DM::DMHandleFilterBlockInputStream<true>::read()+0x199) [0x67e5b49]
8. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<0>::initNextBlock()+0x2e) [0x6cbab9e]
9. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<0>::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x1860) [0x6cbedb0]
10. bin/tiflash/tiflash(DB::DM::DMSegmentThreadInputStream::readImpl(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x88) [0x67c0838]
11. bin/tiflash/tiflash(DB::DM::DMSegmentThreadInputStream::readImpl()+0x1a) [0x67b51aa]
12. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x3bc) [0x5ee9e7c]
13. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read()+0x16) [0x5eea066]
14. bin/tiflash/tiflash(DB::ExpressionBlockInputStream::readImpl()+0x1a) [0x6ab65da]
15. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x3bc) [0x5ee9e7c]
16. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read()+0x16) [0x5eea066]
17. bin/tiflash/tiflash(DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::atomic<bool>*)+0x5c) [0x6b6d6cc]
18. bin/tiflash/tiflash(DB::DAGDriver<false>::execute()+0x4bd) [0x6e8834d]
19. bin/tiflash/tiflash(DB::CoprocessorHandler::execute()+0x63f) [0x6eb19ef]
20. bin/tiflash/tiflash(DB::FlashService::Coprocessor(grpc_impl::ServerContext*, coprocessor::Request const*, coprocessor::Response*)+0x63b) [0x6e70c1b]
21. bin/tiflash/tiflash(std::_Function_handler<grpc::Status (tikvpb::Tikv::Service*, grpc_impl::ServerContext*, coprocessor::Request const*, coprocessor::Response*), std::_Mem_fn<grpc::Status (tikvpb::Tikv::Service::*)(grpc_impl::ServerContext*, coprocessor::Request const*, coprocessor::Response*)> >::_M_invoke(std::_Any_data const&, tikvpb::Tikv::Service*&&, gr
pc_impl::ServerContext*&&, coprocessor::Request const*&&, coprocessor::Response*&&)+0x38) [0x7605308]
22. bin/tiflash/tiflash(grpc::Status grpc::internal::CatchingFunctionHandler<grpc::internal::RpcMethodHandler<tikvpb::Tikv::Service, coprocessor::Request, coprocessor::Response>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)::{lambda()#1}>(grpc::internal::RpcMethodHandler<tikvpb::Tikv::Service, coprocessor::Request, coprocessor::Response>::R
unHandler(grpc::internal::MethodHandler::HandlerParameter const&)::{lambda()#1}&&)+0x53) [0x765b933]
23. bin/tiflash/tiflash(grpc::internal::RpcMethodHandler<tikvpb::Tikv::Service, coprocessor::Request, coprocessor::Response>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)+0x1fd) [0x7692f7d]
24. bin/tiflash/tiflash(grpc_impl::Server::SyncRequest::CallData::ContinueRunAfterInterception()+0x160) [0x76e4900]
25. bin/tiflash/tiflash(grpc_impl::Server::SyncRequestThreadManager::DoWork(void*, bool, bool)+0x42f) [0x76e5ebf]
26. bin/tiflash/tiflash(grpc::ThreadManager::MainWorkLoop()+0x9a) [0x76ebafa]
27. bin/tiflash/tiflash(grpc::ThreadManager::WorkerThread::Run()+0xb) [0x76ebbdb]
28. bin/tiflash/tiflash() [0x7a62c32]
29. /lib64/libpthread.so.0(+0x7e64) [0x7feaaa422e64]
30. /lib64/libc.so.6(clone+0x6c) [0x7feaa9e4988c]
2021.01.05 14:51:27.838206 [ 2565 ] <Error> DAGDriver: void DB::DAGDriver<batch>::execute() [with bool batch = false]: DB Exception: Parameters start = 0, length = 27646 are out of bound in ColumnVector<T>::insertRangeFrom method (data.size() = 3533).
0. bin/tiflash/tiflash(StackTrace::StackTrace()+0x15) [0x3381595]
1. bin/tiflash/tiflash(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x1e) [0x337823e]
2. bin/tiflash/tiflash(DB::ColumnVector<unsigned char>::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long)+0x1ca) [0x66ebb4a]
3. bin/tiflash/tiflash(DB::ColumnNullable::insertRangeFrom(DB::IColumn const&, unsigned long, unsigned long)+0x23) [0x66bfb03]
4. bin/tiflash/tiflash(DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueSpace::Snapshot, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false>::writeCurStableBlock(std::vector<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn>, std::allocator<COWPtr<DB::IColumn>::mutable_ptr<DB::IColumn> > >&, unsigned long&)+0x156) [0x67f7026]
5. bin/tiflash/tiflash(DB::DM::DeltaMergeBlockInputStream<DB::DM::DeltaValueSpace::Snapshot, DB::DM::DTCompactedEntries<55ul, 20ul, 3ul>::Iterator, false>::read()+0x665) [0x67f7785]
6. bin/tiflash/tiflash(DB::DM::DMHandleFilterBlockInputStream<true>::read()+0x5a) [0x67e5a0a]
7. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<0>::initNextBlock()+0x2e) [0x6cbab9e]
8. bin/tiflash/tiflash(DB::DM::DMVersionFilterBlockInputStream<0>::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x15c7) [0x6cbeb17]
9. bin/tiflash/tiflash(DB::DM::DMSegmentThreadInputStream::readImpl(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x88) [0x67c0838]
10. bin/tiflash/tiflash(DB::DM::DMSegmentThreadInputStream::readImpl()+0x1a) [0x67b51aa]
11. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x3bc) [0x5ee9e7c]
12. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read()+0x16) [0x5eea066]
13. bin/tiflash/tiflash(DB::ExpressionBlockInputStream::readImpl()+0x1a) [0x6ab65da]
14. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read(DB::PODArray<unsigned char, 4096ul, Allocator<false>, 15ul, 16ul>*&, bool)+0x3bc) [0x5ee9e7c]
15. bin/tiflash/tiflash(DB::IProfilingBlockInputStream::read()+0x16) [0x5eea066]
16. bin/tiflash/tiflash(DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::atomic<bool>*)+0x5c) [0x6b6d6cc]
17. bin/tiflash/tiflash(DB::DAGDriver<false>::execute()+0x4bd) [0x6e8834d]
18. bin/tiflash/tiflash(DB::CoprocessorHandler::execute()+0x63f) [0x6eb19ef]
19. bin/tiflash/tiflash(DB::FlashService::Coprocessor(grpc_impl::ServerContext*, coprocessor::Request const*, coprocessor::Response*)+0x63b) [0x6e70c1b]
20. bin/tiflash/tiflash(std::_Function_handler<grpc::Status (tikvpb::Tikv::Service*, grpc_impl::ServerContext*, coprocessor::Request const*, coprocessor::Response*), std::_Mem_fn<grpc::Status (tikvpb::Tikv::Service::*)(grpc_impl::ServerContext*, coprocessor::Request const*, coprocessor::Response*)> >::_M_invoke(std::_Any_data const&, tikvpb::Tikv::Service*&&, grpc_impl::ServerContext*&&, coprocessor::Request const*&&, coprocessor::Response*&&)+0x38) [0x7605308]
21. bin/tiflash/tiflash(grpc::Status grpc::internal::CatchingFunctionHandler<grpc::internal::RpcMethodHandler<tikvpb::Tikv::Service, coprocessor::Request, coprocessor::Response>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)::{lambda()#1}>(grpc::internal::RpcMethodHandler<tikvpb::Tikv::Service, coprocessor::Request, coprocessor::Response>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)::{lambda()#1}&&)+0x53) [0x765b933]
22. bin/tiflash/tiflash(grpc::internal::RpcMethodHandler<tikvpb::Tikv::Service, coprocessor::Request, coprocessor::Response>::RunHandler(grpc::internal::MethodHandler::HandlerParameter const&)+0x1fd) [0x7692f7d]
23. bin/tiflash/tiflash(grpc_impl::Server::SyncRequest::CallData::ContinueRunAfterInterception()+0x160) [0x76e4900]
24. bin/tiflash/tiflash(grpc_impl::Server::SyncRequestThreadManager::DoWork(void*, bool, bool)+0x42f) [0x76e5ebf]
25. bin/tiflash/tiflash(grpc::ThreadManager::MainWorkLoop()+0x9a) [0x76ebafa]
26. bin/tiflash/tiflash(grpc::ThreadManager::WorkerThread::Run()+0xb) [0x76ebbdb]
27. bin/tiflash/tiflash() [0x7a62c32]
28. /lib64/libpthread.so.0(+0x7e64) [0x7f9b82730e64]
29. /lib64/libc.so.6(clone+0x6c) [0x7f9b8215788c]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-cherry-pick-release-4.0 PR which needs to be cherry-picked to release-4.0 status/LGT1 Indicates that a PR has LGTM 1. type/bugfix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants