Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: implement value_schema_manager::get_value_schema #739

Merged
merged 7 commits into from
Jun 1, 2021

Conversation

levy5307
Copy link
Contributor

What problem does this PR solve?

proposal https://github.com/apache/incubator-pegasus/blob/master/rfcs/2020-10-09-data-version-v3.md

implement value_schema_manager::get_value_schema

Checklist

Tests
  • Unit test

@hycdong
Copy link
Contributor

hycdong commented May 31, 2021

The old value_schema is a table level concept defined in data_version in meta column family. Now, value_schema version 2 updates it, value_schema becomes a value level concept, a table is permitted having different value schema. However, it will raise two problems.
The first problem is the compatibility problem. New server version won't be downgraded because the old server can not figure out value_schema version 2 data. I think it is okay for new server can not downgrade, but our recent releases are almost all not-downgraded, should we merge this commit into master branch?
The second problem is the data_version in meta cf. This value is actually useless for new server, but it can not be removed for version0 and version1, would you have any solution to remove this value in future?

@levy5307
Copy link
Contributor Author

levy5307 commented May 31, 2021

The old value_schema is a table level concept defined in data_version in meta column family. Now, value_schema version 2 updates it, value_schema becomes a value level concept, a table is permitted having different value schema. However, it will raise two problems.
The first problem is the compatibility problem. New server version won't be downgraded because the old server can not figure out value_schema version 2 data. I think it is okay for new server can not downgrade, but our recent releases are almost all not-downgraded, should we merge this commit into master branch?
The second problem is the data_version in meta cf. This value is actually useless for new server, but it can not be removed for version0 and version1, would you have any solution to remove this value in future?

I don't think so. Because value_schema version 2 will not updates data version in meta column family. It only reads from it.

hycdong
hycdong previously approved these changes Jun 1, 2021
@hycdong hycdong merged commit b5e73d4 into apache:master Jun 1, 2021
@hycdong hycdong mentioned this pull request Sep 23, 2021
acelyc111 pushed a commit that referenced this pull request Jun 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants