Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved snapshot creation logic. #2318

Merged
merged 8 commits into from
Sep 4, 2020
Merged

Improved snapshot creation logic. #2318

merged 8 commits into from
Sep 4, 2020

Conversation

bright-starry-sky
Copy link
Contributor

Unable to reproduce on development machine, added some diagnostic information for snapshot_command_test.
Please ignore this PR, I will close it when I find the failure reason.

@CLAassistant
Copy link

CLAassistant commented Sep 2, 2020

CLA assistant check
All committers have signed the CLA.

@codecov-commenter
Copy link

Codecov Report

Merging #2318 into master will decrease coverage by 0.29%.
The diff coverage is 65.50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2318      +/-   ##
==========================================
- Coverage   86.46%   86.17%   -0.30%     
==========================================
  Files         649      648       -1     
  Lines       64374    64770     +396     
==========================================
+ Hits        55662    55815     +153     
- Misses       8712     8955     +243     
Impacted Files Coverage Δ
src/common/hdfs/HdfsCommandHelper.cpp 0.00% <0.00%> (-75.00%) ⬇️
src/graph/DownloadExecutor.cpp 0.00% <0.00%> (ø)
src/graph/DropSpaceExecutor.h 0.00% <ø> (ø)
src/graph/FetchEdgesExecutor.cpp 79.32% <0.00%> (-0.27%) ⬇️
src/graph/FetchVerticesExecutor.cpp 78.21% <0.00%> (-0.18%) ⬇️
src/graph/FindPathExecutor.h 50.00% <ø> (ø)
src/graph/GoExecutor.h 91.30% <ø> (-1.56%) ⬇️
src/graph/IngestExecutor.cpp 0.00% <0.00%> (ø)
src/graph/LookupExecutor.cpp 67.89% <0.00%> (-0.15%) ⬇️
src/graph/test/TestBase.h 70.00% <ø> (ø)
... and 82 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b548c74...fca17ef. Read the comment docs.

@bright-starry-sky bright-starry-sky changed the title Only used to analyze unstable test cases on CI systems Improved snapshot creation logic. Sep 3, 2020
@bright-starry-sky
Copy link
Contributor Author

bright-starry-sky commented Sep 3, 2020

1, Solved problem of unstable test cases of snapshot_command_test. Problem is caused by a deadlock. createSnapshot(get read space_lock) --> updateHostInfo ( get write space_lock) -> cancelWriteBlocking end of createSnapshot (get read space_lock) . read_lock -> write_lock -> read_lock so deadlock occurred. Thanks @critical27.
2, Allowed to retry when create snapshot.
3, Write Blocking all partitions, whether they are leaders or followers.

if (fs::FileUtils::exist(checkpointPath)) {
LOG(ERROR) << "The snapshot file already exists: " << checkpointPath;
return ResultCode::ERR_CHECKPOINT_ERROR;
if (fs::FileUtils::exist(checkpointPath) &&
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not return success?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not return success?

Because we don't have a way to verify the validity of the snapshot that has been created. For data security, I need to delete and recreate it at here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fine.

@@ -620,6 +620,13 @@ bool FileBasedWal::linkCurrentWAL(const char* newPath) {
LOG(INFO) << idStr_ << "No wal files found, skip link";
return true;
}

if (fs::FileUtils::exist(newPath) &&
!fs::FileUtils::remove(newPath, true)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor

@dangleptr dangleptr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Well done.

@@ -13,7 +13,6 @@ namespace nebula {
namespace meta {

void CreateSnapshotProcessor::process(const cpp2::CreateSnapshotReq&) {
folly::SharedMutex::ReadHolder rHolder(LockUtils::spaceLock());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, 👍

@dangleptr dangleptr merged commit dcf96d9 into vesoft-inc:master Sep 4, 2020
xuguruogu pushed a commit to xuguruogu/nebula that referenced this pull request Sep 12, 2020
tong-hao pushed a commit to tong-hao/nebula that referenced this pull request Jun 1, 2021
yixinglu pushed a commit to yixinglu/nebula that referenced this pull request Jan 31, 2023
The dump_syms tool path should be match with third party version.

Co-authored-by: George <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants