Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ThreadSanitizer: data race of DeltaIndex #5947

Closed
JinheLin opened this issue Sep 19, 2022 · 0 comments · Fixed by #5948
Closed

ThreadSanitizer: data race of DeltaIndex #5947

JinheLin opened this issue Sep 19, 2022 · 0 comments · Fixed by #5948
Assignees
@JinheLin
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. component/storage severity/moderate type/bug The issue is confirmed as a bug.

Comments

@JinheLin
Copy link
Contributor

DeltaIndex::toString is not thread-safe. Member variable delta_tree (std::shared_ptr<DefaultDeltaTree>) can be modifed concurrently.
image

[2022-09-18T18:13:13.415Z] WARNING: ThreadSanitizer: data race (pid=23885)
[2022-09-18T18:13:13.415Z]   Write of size 8 at 0x7b1c001232a0 by thread T53 (mutexes: write M590669861329449664):
[2022-09-18T18:13:13.415Z]     #0 std::__1::enable_if<(is_move_constructible<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists>*>::value) && (is_move_assignable<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists>*>::value), void>::type std::__1::swap<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists>*>(DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists>*&, DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists>*&) /usr/local/bin/../include/c++/v1/__utility/swap.h:40:7 (gtests_dbms+0xba4f0a2)
[2022-09-18T18:13:13.415Z]     #1 std::__1::shared_ptr<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists> >::swap(std::__1::shared_ptr<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists> >&) /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:1045:5 (gtests_dbms+0xba4f0a2)
[2022-09-18T18:13:13.415Z]     #2 std::__1::shared_ptr<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists> >::operator=(std::__1::shared_ptr<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists> > const&) /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:967:21 (gtests_dbms+0xba4f0a2)
[2022-09-18T18:13:13.415Z]     #3 DB::DM::DeltaIndex::updateIfAdvanced(DB::DM::DeltaIndex const&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaIndex.h:191:24 (gtests_dbms+0xba4f0a2)
[2022-09-18T18:13:13.415Z]     #4 DB::DM::Segment::getReadInfo(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, unsigned long) const /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:1651:63 (gtests_dbms+0xba4f0a2)
[2022-09-18T18:13:13.415Z]     #5 DB::DM::Segment::getInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:388:22 (gtests_dbms+0xba4d334)
[2022-09-18T18:13:13.415Z]     #6 DB::DM::SegmentReadTaskPool::buildInputStream(std::__1::shared_ptr<DB::DM::SegmentReadTask>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:110:23 (gtests_dbms+0xba8ad2d)
[2022-09-18T18:13:13.415Z]     #7 DB::DM::MergedTask::initOnce() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:50:24 (gtests_dbms+0xbbbad32)
[2022-09-18T18:13:13.415Z]     #8 DB::DM::MergedTask::readBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:31:5 (gtests_dbms+0xbbbabb9)
[2022-09-18T18:13:13.415Z]     #9 DB::DM::SegmentReader::readSegments() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:101:39 (gtests_dbms+0xbbc6812)
[2022-09-18T18:13:13.415Z]     #10 DB::DM::SegmentReader::run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:152:13 (gtests_dbms+0xbbc61d7)
[2022-09-18T18:13:13.416Z]     #11 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xbbc77c3)
[2022-09-18T18:13:13.416Z]     #12 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xbbc77c3)
[2022-09-18T18:13:13.416Z]     #13 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xbbc77c3)
[2022-09-18T18:13:13.416Z] 
[2022-09-18T18:13:13.416Z]   Previous read of size 8 at 0x7b1c001232a0 by thread T54:
[2022-09-18T18:13:13.416Z]     #0 std::__1::shared_ptr<DB::DM::DeltaTree<DB::DM::EmptyValueSpace, 55ul, 20ul, 3ul, DB::ArenaWithFreeLists> >::operator->() const /usr/local/bin/../include/c++/v1/__memory/shared_ptr.h:589:16 (gtests_dbms+0xba67839)
[2022-09-18T18:13:13.416Z]     #1 DB::DM::DeltaIndex::toString() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/DeltaIndex.h:151:28 (gtests_dbms+0xba67839)
[2022-09-18T18:13:13.416Z]     #2 DB::DM::Segment::ensurePlace(DB::DM::DMContext const&, std::__1::shared_ptr<DB::DM::StableValueSpace::Snapshot> const&, std::__1::shared_ptr<DB::DM::DeltaValueReader> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, unsigned long) const /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:1815:5 (gtests_dbms+0xba67839)
[2022-09-18T18:13:13.416Z]     #3 DB::DM::Segment::getReadInfo(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, unsigned long) const /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:1639:44 (gtests_dbms+0xba4e9a7)
[2022-09-18T18:13:13.416Z]     #4 DB::DM::Segment::getInputStream(DB::DM::DMContext const&, std::__1::vector<DB::DM::ColumnDefine, std::__1::allocator<DB::DM::ColumnDefine> > const&, std::__1::shared_ptr<DB::DM::SegmentSnapshot> const&, std::__1::vector<DB::DM::RowKeyRange, std::__1::allocator<DB::DM::RowKeyRange> > const&, std::__1::shared_ptr<DB::DM::RSOperator> const&, unsigned long, unsigned long) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/Segment.cpp:388:22 (gtests_dbms+0xba4d334)
[2022-09-18T18:13:13.416Z]     #5 DB::DM::SegmentReadTaskPool::buildInputStream(std::__1::shared_ptr<DB::DM::SegmentReadTask>&) /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/SegmentReadTaskPool.cpp:110:23 (gtests_dbms+0xba8ad2d)
[2022-09-18T18:13:13.416Z]     #6 DB::DM::MergedTask::initOnce() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:50:24 (gtests_dbms+0xbbbad32)
[2022-09-18T18:13:13.416Z]     #7 DB::DM::MergedTask::readBlock() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/MergedTask.cpp:31:5 (gtests_dbms+0xbbbabb9)
[2022-09-18T18:13:13.416Z]     #8 DB::DM::SegmentReader::readSegments() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:101:39 (gtests_dbms+0xbbc6812)
[2022-09-18T18:13:13.416Z]     #9 DB::DM::SegmentReader::run() /home/jenkins/agent/workspace/tiflash-sanitizer-daily/tiflash/dbms/src/Storages/DeltaMerge/ReadThread/SegmentReader.cpp:152:13 (gtests_dbms+0xbbc61d7)
[2022-09-18T18:13:13.416Z]     #10 decltype(*(static_cast<DB::DM::SegmentReader*>(fp0)).*fp()) std::__1::__invoke<void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, void>(void (DB::DM::SegmentReader::*&&)(), DB::DM::SegmentReader*&&) /usr/local/bin/../include/c++/v1/type_traits:3859:1 (gtests_dbms+0xbbc77c3)
[2022-09-18T18:13:13.417Z]     #11 void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*>&, std::__1::__tuple_indices<2ul>) /usr/local/bin/../include/c++/v1/thread:280:5 (gtests_dbms+0xbbc77c3)
[2022-09-18T18:13:13.417Z]     #12 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (DB::DM::SegmentReader::*)(), DB::DM::SegmentReader*> >(void*) /usr/local/bin/../include/c++/v1/thread:291:5 (gtests_dbms+0xbbc77c3)
@JinheLin JinheLin added type/bug The issue is confirmed as a bug. severity/moderate component/storage affects-6.1 This bug affects the 6.1.x(LTS) versions. labels Sep 19, 2022
@JinheLin JinheLin self-assigned this Sep 19, 2022
@JinheLin JinheLin mentioned this issue Sep 19, 2022
Merged
12 tasks
ti-chi-bot pushed a commit that referenced this issue Sep 19, 2022
@JinheLin
Fix data race of DeltaIndex::toString (#5948)
f01f4b1 
close #5947
@ti-chi-bot ti-chi-bot mentioned this issue Sep 19, 2022
Merged
12 tasks
ti-chi-bot added a commit that referenced this issue Oct 10, 2022
@ti-chi-bot
Fix data race of DeltaIndex::toString (#5948) (#5954)
ca39013 
close #5947
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JinheLin JinheLin

Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. component/storage severity/moderate type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix data race of DeltaIndex::toString JinheLin/tiflash-1
1 participant
@JinheLin