Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix HBase PrefixFilter bug #2364

Merged
merged 6 commits into from
Nov 29, 2023
Merged

fix HBase PrefixFilter bug #2364

merged 6 commits into from
Nov 29, 2023

Conversation

haohao0103
Copy link
Contributor

@haohao0103 haohao0103 commented Nov 27, 2023 •
edited by javeme
Loading

Purpose of the PR

Main Changes

Verifying these changes

  • Trivial rework / code cleanup without any test coverage. (No Need)
  • Already covered by existing tests, such as (please modify tests here).
  • Need tests and can be verified as follows:
    • xxx

Does this PR potentially affect the following parts?

  • Nope
  • Dependencies (add/update license info)
  • Modify configurations
  • The public API
  • Other affects (typed here)

Documentation Status

  • Doc - TODO
  • Doc - Done
  • Doc - No Need

@javeme javeme changed the title fix #2177 fix HBase PrefixFilter bug Nov 27, 2023
javeme
javeme reviewed Nov 27, 2023
View reviewed changes
Copy link
Contributor

@javeme javeme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution

...er/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseSessions.java Outdated Show resolved Hide resolved
@codecov Codecov
Copy link

codecov bot commented Nov 27, 2023 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (c0ea21e) 65.88% compared to head (90780fc) 65.87%.
Report is 2 commits behind head on master.

Additional details and impacted files 
@@             Coverage Diff              @@
##             master    #2364      +/-   ##
============================================
- Coverage     65.88%   65.87%   -0.02%     
  Complexity      826      826              
============================================
  Files           507      507              
  Lines         42073    42076       +3     
  Branches       5831     5832       +1     
============================================
- Hits          27720    27716       -4     
- Misses        11595    11608      +13     
+ Partials       2758     2752       -6

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@JackyYangPassion JackyYangPassion mentioned this pull request Nov 27, 2023
Closed
JackyYangPassion
JackyYangPassion approved these changes Nov 27, 2023
View reviewed changes
...er/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseSessions.java Outdated
*/
default R scan(String table, byte[] prefix) {
return this.scan(table, prefix, true, prefix);
final Scan scan = new Scan().setRowPrefixFilter(prefix);
Copy link
Contributor

@JackyYangPassion JackyYangPassion Nov 27, 2023 •
edited by imbajin
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

    public Scan setRowPrefixFilter(byte[] rowPrefix) {
        if (rowPrefix == null) {
            this.setStartRow(HConstants.EMPTY_START_ROW);
            this.setStopRow(HConstants.EMPTY_END_ROW);
        } else {
            this.setStartRow(rowPrefix);
            this.setStopRow(this.calculateTheClosestNextRowKeyForPrefix(rowPrefix));
        }

        return this;
    }

calculateTheClosestNextRowKeyForPrefix(rowPrefix)

The core of this method accurately calculates stop row, thereby solving the problem of edge table query seek HFile filtering invalid key-value cells, thus reducing subgraph query p99.

Copy link
Member

@imbajin imbajin Nov 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculateTheClosestNextRowKeyForPrefix

calculateTheClosestNextRowKeyForPrefix() doesn't include for now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setRowPrefixFilter will call calculateTheClosestNextRowKeyForPrefix method

javeme
javeme previously approved these changes Nov 27, 2023
View reviewed changes
Copy link
Contributor

@javeme javeme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

JackyYangPassion
JackyYangPassion requested changes Nov 28, 2023
View reviewed changes
Copy link
Contributor

@JackyYangPassion JackyYangPassion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some ci error need to fix

JackyYangPassion
JackyYangPassion approved these changes Nov 29, 2023
View reviewed changes
Copy link
Contributor

@JackyYangPassion JackyYangPassion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JackyYangPassion
Copy link
Contributor

JackyYangPassion commented Nov 29, 2023 •
edited
Loading

ci-cassandra failure is not related to this PR.

log

Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [/127.0.0.1:9042] Cannot connect))

@simon824 simon824 merged commit d376b02 into apache:master Nov 29, 2023
17 of 21 checks passed
VGalaxies pushed a commit to VGalaxies/incubator-hugegraph that referenced this pull request Jan 12, 2024
@haohao0103 @VGalaxies
fix HBase PrefixFilter bug (apache#2364)
422a8cd 
* apache#2177

* apache#2177

* apache#2177

* apache#2177

* apache#2177

* apache#2177
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@imbajin imbajin imbajin left review comments

@simon824 simon824 simon824 approved these changes

@JackyYangPassion JackyYangPassion JackyYangPassion approved these changes

@javeme javeme javeme left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug] Due to a bug in HBase PrefixFilter, the performance of xhop subgraph query has been degraded.
5 participants
@haohao0103 @JackyYangPassion @javeme @imbajin @simon824