Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Subtask] Add the version information in storage layout #663

Closed
jerryshao opened this issue Nov 3, 2023 · 3 comments · Fixed by #701
Closed

[Subtask] Add the version information in storage layout #663

jerryshao opened this issue Nov 3, 2023 · 3 comments · Fixed by #701
Assignees
@yuqi1129
Milestone
Gravitino 0.3.0

Comments

@jerryshao
Copy link
Contributor

Describe the subtask

We should add version information in our storage layout to show the version of the current layout. So that we can evolve our format and keep code compatibility.

Parent issue

#603
@yuqi1129
Copy link
Contributor

yuqi1129 commented Nov 6, 2023

@jerryshao, could you provide more information about this issue? I'm afraid I can't fully understand your idea.
Let me guess what you mean.

  • Add a separate key to display the current storage layout version , such as 'current_version --> 0.2.0'
  • Add the version information as the prefixes/suffixes of keys, such as if the origin key-value pair is key --> value then the the changed key-value pair is 0.2.0_key --> value.
  • Other implementations include putting the version information on the value of key-value pair.

Can you explain the issue more clearly and what your implementation entails?

@jerryshao
Copy link
Contributor Author

I think the 1st point is what I think of. We should have a version number written into the storage. And based on this version, we could have the ability to choose the right code path.

This is to support compatibility if we later on have a different storage layout design.

The 2nd point is mainly related to MMVC design, this is unrelated to what I mentioned here.

@yuqi1129
Copy link
Contributor

yuqi1129 commented Nov 7, 2023

I think the 1st point is what I think of. We should have a version number written into the storage. And based on this version, we could have the ability to choose the right code path.

This is to support compatibility if we later on have a different storage layout design.

The 2nd point is mainly related to MMVC design, this is unrelated to what I mentioned here.

Got it.

@yuqi1129 yuqi1129 mentioned this issue Nov 7, 2023
Merged
@jerryshao jerryshao added this to the Gravitino 0.3.0 milestone Nov 20, 2023
jerryshao pushed a commit that referenced this issue Nov 20, 2023
@yuqi1129
[#663] improvement(core): Add storage layout version information and …
d3d5aeb 
…store it in kv store (#701)

### What changes were proposed in this pull request?

Save the KV layout version information into KV storage.

### Why are the changes needed?

We require storage layout version information to determine the code path
we should use.


Fix: #663 

### Does this PR introduce _any_ user-facing change?

No

### How was this patch tested?

New UTs in `TestKvEntityStore`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@yuqi1129 yuqi1129

Labels
None yet
Projects
None yet
Milestone
Gravitino 0.3.0
2 participants
@jerryshao @yuqi1129