Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix some rocksdb api will fail in PlainTable #4254

Closed
wants to merge 4 commits into from

Conversation

critical27
Copy link
Contributor

What type of PR is this?

  • bug
  • feature
  • enhancement

What problem(s) does this PR solve?

Issue(s) number:

Description:

This problem is found in nebula-ng. Some of our rocksdb api is not well verified in PlainTable, for example, whether the data is in sst will effect the api result. The main problem is that:

  1. Since PlainTable only support prefix-based Seek, we need to specify prefix_same_as_start not only in prefix, but also range and rangeWithPrefix.
  2. And the length of prefix_extractor need to be modified. Because within PlainTable, if the prefix bloom filter is not inserted, you can't be read by prefix any more. So to make sure every data could be read, we use the minimum length we use in prefix, which is 4.

This is a simple way to fix the problem. In nebula-ng, I will consider refactor it a bit.

How do you solve it?

Special notes for your reviewer, ex. impact of this fix, design document, etc:

Checklist:

Tests:

  • Unit test(positive and negative cases)
  • Function test
  • Performance test
  • N/A

Affects:

  • Documentation affected (Please add the label if documentation needs to be modified.)
  • Incompatibility (If it breaks the compatibility, please describe it and add the label.)
  • If it's needed to cherry-pick (If cherry-pick to some branches is required, please label the destination version(s).)
  • Performance impacted: Consumes more CPU/Memory

Release notes:

Please confirm whether to be reflected in release notes and how to describe:
Not related.

if (!isPlainTable_) {
options.total_order_seek = FLAGS_enable_rocksdb_prefix_filtering;
} else {
options.prefix_same_as_start = true;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So prefix_same_as_start & extractorLen_ together decide the behavior? And only in plaintable?

Copy link
Contributor Author

@critical27 critical27 May 16, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PlainTable must use prefix-based Seek, so we must set prefix_same_as_start = true;
And for BlockBasedTable, if we have prefix extractor, we just use total_order_seek for simplicity, otherwise we need to judge whether the parameter is longer than extractor length (the way we use in prefix function)

Copy link
Contributor

@SuperYoko SuperYoko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Copy link
Contributor

@wenhaocs wenhaocs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GJ

@critical27
Copy link
Contributor Author

Don't merge it plz, the funny part is that this will fail in ubuntu2004-clang, but other env will pass. I need to check it later

@critical27 critical27 marked this pull request as draft May 17, 2022 04:24
@Sophie-Xie Sophie-Xie added this to the v3.2.0 milestone May 20, 2022
@Sophie-Xie Sophie-Xie removed this from the v3.2.0 milestone Jun 8, 2022
@critical27 critical27 closed this Jul 8, 2022
@critical27 critical27 deleted the fix branch July 8, 2022 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-for-testing PR: ready for the CI test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants