-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HBASE-22625 documet use scan snapshot feature #496
Conversation
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
=== TableSnapshotScanner | ||
|
||
TableSnapshotScanner provide a way to do single client side scan over snapshot files. | ||
When use TableSnapshotScanner, we must specify a temporary directory to copy the snapshot files into. Current user should have write permissions to this directory, and this should not be a subdirectory of rootdir. The scanner deletes the contents of the directory once the scanner is closed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
copy the snapshot files into. -> restore the snapshot
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rootdir -> hbase.rootdir
:source-language: java | ||
|
||
In HBase, scan a table costs many CPU, memory... resources. Luckily, HBase provides a TableSnapshotScanner and TableSnapshotInputFormat (introduced by link:https://issues.apache.org/jira/browse/HBASE-8369[HBASE-8369]), which performs a scan over snapshot files. | ||
By this way, we can bypasse HBase servers, and access the underlying files directly to provide maximum performance. And can also be used with offline HBase with in-place or exported snapshot files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bypasse -> bypass
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
files -> HFiles
:toc: left | ||
:source-language: java | ||
|
||
In HBase, scan a table costs many CPU, memory... resources. Luckily, HBase provides a TableSnapshotScanner and TableSnapshotInputFormat (introduced by link:https://issues.apache.org/jira/browse/HBASE-8369[HBASE-8369]), which performs a scan over snapshot files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scan a table costs many CPU, memory... resources -> scanning a big table will put tremendous pressure on the cluster.
[source,java] | ||
---- | ||
Job job = new Job(conf); | ||
Path restoreDir = new Path("XX"); // restore dir should not be a subdirectory HBase rootdir |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
a subdirectory of hbase.rootdir
|
||
==== HDFS ACLs | ||
|
||
HDFS ACLs supports an "access ACL", which defines the rules to enforce during permission checks, and a "default ACL", which defines the ACL entries that new child files or sub-directories receive automatically during creation. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Provide a link referring to the document of HDFS ACL ?
hbase.acl.sync.to.hdfs.enable=true | ||
---- | ||
|
||
* Modify table scheme to enable this feature for a specified table, this config is false by default for every table, this means the HBase granted acls will not synced to HDFS |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will not synced -> will not be synced
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
Addendum adding Zheng Hu feedback.
I pushed @openinx review edits as an addendum. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
D'oh always late to seeing things. It seems there may be one or two misspellings?
|
||
==== Basic idea | ||
|
||
The HBase files are orginazed as the following ways: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
orginazed->organized
hbase.acl.sync.to.hdfs.enable=true | ||
---- | ||
|
||
* Modify table scheme to enable this feature for a specified table, this config is false by default for every table, this means the HBase granted acls will not synced to HDFS |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
scheme->schema (or CONFIGURATION
)?
Fix feedback from Clay Baenziger. Signed-off-by: Clay Baenziger <[email protected]>
No description provided.