Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rocksdb): Use original CreateCheckpoint and read meta data from meta CF #535

Merged
merged 6 commits into from
Jun 30, 2020

Conversation

acelyc111
Copy link
Member

@acelyc111 acelyc111 commented May 14, 2020

What problem does this PR solve?

Ref: #468
We have store meta data(Pegasus data version, last manual compact timestamp, decree) both in RocksDB's manifest and meta column family in version 2.0.0, now we are going to read meta data from meta CF instead of manifest.
Main changes of this PR:

  • Validate meta CF existence when DB exist (means only allowed to upgrade from 2.0)
  • Read meta data from meta CF
  • Do checkpoint by rocksdb's original APIs.

NOTE: Since meta data is still wrote into manifest, so there is no compatibility problems with 2.0.0.

What is changed and how it works?

No user faced changes.

Check List

Tests

  • Manual test (add detailed scripts or steps below)
  1. Upgrade from 1.x.x
    Will fail, only allowed from version 2.0.x
  2. Upgrade from 2.0.0
  • Deploy a cluster in version 2.0.0
  • Write some data
  • Upgrade to this commit's version
  • Read these data successfully

Code changes

  • Has exported function/method change
    No
  • Has exported variable/fields change
    No
  • Has interface methods change
    No
  • Has persistent data change
    No

Side effects

  • Possible performance regression
    No
  • Increased code complexity
    No
  • Breaking backward compatibility
    No

Related changes

  • Need to cherry-pick to the release branch
    Yes
  • Need to update the documentation
    Yes
  • Need to be included in the release note
    Yes

@acelyc111 acelyc111 changed the title [rocksdb] use original rocksdb feat(rocksdb): Use original rocksdb May 15, 2020
@acelyc111 acelyc111 changed the title feat(rocksdb): Use original rocksdb feat(rocksdb): Use original CreateCheckpoint and read meta data from meta CF Jun 29, 2020
@acelyc111 acelyc111 marked this pull request as ready for review June 29, 2020 08:00
src/server/meta_store.cpp Show resolved Hide resolved
src/server/pegasus_server_impl.cpp Outdated Show resolved Hide resolved
src/server/pegasus_server_impl.cpp Show resolved Hide resolved
hycdong
hycdong previously approved these changes Jun 30, 2020
@acelyc111 acelyc111 merged commit ef31d13 into apache:master Jun 30, 2020
@hycdong hycdong mentioned this pull request Aug 17, 2020
hycdong pushed a commit to hycdong/pegasus that referenced this pull request Aug 24, 2020
@hycdong hycdong added the type/incompatible Changes that introduced incompatibility to Pegasus. label Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.1.0 type/incompatible Changes that introduced incompatibility to Pegasus.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants