Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix HBase PrefixFilter bug #2364

Merged
merged 6 commits into from
Nov 29, 2023
Merged

fix HBase PrefixFilter bug #2364

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
c80d7b9
#2177
haohao0103 Nov 27, 2023
f002452
#2177
haohao0103 Nov 27, 2023
9b52d39
#2177
haohao0103 Nov 27, 2023
4c5d119
#2177
haohao0103 Nov 27, 2023
6bbf5d2
#2177
haohao0103 Nov 28, 2023
90780fc
#2177
haohao0103 Nov 29, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 4 additions & 10 deletions ...hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseSessions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,10 @@
* Scan records by rowkey prefix from a table
*/
default R scan(String table, byte[] prefix) {
return this.scan(table, prefix, true, prefix);
// TODO: setRowPrefixFilter deprecated since HBase 2.5.0,
// will be removed in 4.0.0,setStartStopRowForPrefixScan(byte[]) instead.
haohao0103 marked this conversation as resolved.
Show resolved Hide resolved
final Scan scan = new Scan().setRowPrefixFilter(prefix);
Copy link
Contributor

@JackyYangPassion JackyYangPassion Nov 27, 2023 •
edited by imbajin
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

    public Scan setRowPrefixFilter(byte[] rowPrefix) {
        if (rowPrefix == null) {
            this.setStartRow(HConstants.EMPTY_START_ROW);
            this.setStopRow(HConstants.EMPTY_END_ROW);
        } else {
            this.setStartRow(rowPrefix);
            this.setStopRow(this.calculateTheClosestNextRowKeyForPrefix(rowPrefix));
        }

        return this;
    }

calculateTheClosestNextRowKeyForPrefix(rowPrefix)

The core of this method accurately calculates stop row, thereby solving the problem of edge table query seek HFile filtering invalid key-value cells, thus reducing subgraph query p99.

Copy link
Member

@imbajin imbajin Nov 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculateTheClosestNextRowKeyForPrefix

calculateTheClosestNextRowKeyForPrefix() doesn't include for now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

setRowPrefixFilter will call calculateTheClosestNextRowKeyForPrefix method

return this.scan(table, scan);

Check warning on line 397 in hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseSessions.java

View check run for this annotation

Codecov / codecov/patch

hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseSessions.java#L396-L397 

Added lines #L396 - L397 were not covered by tests
}

/**
Expand All @@ -413,15 +416,6 @@
return this.scan(table, scan);
}

/**
* Scan records by rowkey start and prefix from a table
*/
default R scan(String table, byte[] startRow, boolean inclusiveStart,
byte[] prefix) {
Scan scan = new Scan().withStartRow(startRow, inclusiveStart)
.setFilter(new PrefixFilter(prefix));
return this.scan(table, scan);
}

/**
* Scan records by rowkey range from a table
Expand Down
3 changes: 1 addition & 2 deletions ...er/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseTable.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,8 +230,7 @@

protected <R> R queryByPrefix(HbaseSessions.HbaseSession<R> session,
IdPrefixQuery query) {
return session.scan(this.table(), query.start().asBytes(),
query.inclusiveStart(), query.prefix().asBytes());
return session.scan(this.table(), query.prefix().asBytes());

Check warning on line 233 in hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseTable.java

View check run for this annotation

Codecov / codecov/patch

hugegraph-server/hugegraph-hbase/src/main/java/org/apache/hugegraph/backend/store/hbase/HbaseTable.java#L233 

Added line #L233 was not covered by tests
}

protected <R> R queryByRange(HbaseSessions.HbaseSession<R> session, IdRangeQuery query) {
Expand Down
Loading