Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the same rocksdb snapshot when sending snapshot #3846

Merged
merged 2 commits into from
Mar 10, 2022

Conversation

critical27
Copy link
Contributor

@critical27 critical27 commented Jan 29, 2022

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

This is some prepare work for later data balance test, mainly two changes:

  1. Follow up of Add snapshot for prefix iterate. #3717(In that pr we use snapshot when doing data scan, but the commit log id/term is not taken care). So in this PR, we create a rocksdb snapshot at first, read commit log id/term, and use the same snapshot to scan data.
  2. We can't simply skip the special raft command, because if we skip the special log after rebooting, the state of raft peer will be wrong.
  3. Add some metrics in raft, remove the SlowOpTracker.

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

not related

SuperYoko
SuperYoko previously approved these changes Jan 31, 2022
src/kvstore/NebulaSnapshotManager.cpp Outdated Show resolved Hide resolved
src/kvstore/Part.cpp Show resolved Hide resolved
wenhaocs
wenhaocs previously approved these changes Feb 9, 2022
Copy link
Contributor

@wenhaocs wenhaocs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Want to make sure deleting SlowOpTracker is also intended.

@critical27
Copy link
Contributor Author

LGTM. Want to make sure deleting SlowOpTracker is also intended.

Yeah, SlowOpTracker is only used in raft, and after log level is adjusted, SlowOpTracker still use LOG(INFO) which is not appropriate. Since we could add more metrics, just unify them and delete the SlowOpTracker.

wenhaocs
wenhaocs previously approved these changes Feb 10, 2022
wenhaocs
wenhaocs previously approved these changes Feb 21, 2022
Copy link
Contributor

@panda-sheep panda-sheep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job!

@Shinji-IkariG Shinji-IkariG merged commit b88c1e1 into vesoft-inc:master Mar 10, 2022
@critical27 critical27 deleted the fix branch March 21, 2022 10:00
@critical27 critical27 mentioned this pull request Apr 1, 2022
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready for review ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants