Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-23055 Alter hbase:meta #646

Closed
wants to merge 1 commit into from

Conversation

saintstack
Copy link
Contributor

Make it so hbase:meta can be altered. TableState for hbase:meta
is kept in Master. State is in-memory transient so if Master
fails, hbase:meta is ENABLED again. hbase:meta schema will be
bootstrapped from the filesystem. Changes to filesystem schema
are atomic so we should be ok if Master fails mid-edit (TBD)
Undoes a bunch of guards that prevented our being able to edit
hbase:meta. At minimmum, need to add in a bunch of WARNING.

TODO: Tests, more clarity around hbase:meta table state, and undoing
references to hard-coded hbase:meta regioninfo.

M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
Throw illegal access exception if you try to use MetaTableAccessor
getting state of the hbase:meta table.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
TODO: getTableState needs work in Connection implemetnations. Presumes
state is in meta table for all tables. Uses MetaTableAccessor.
TODO: More cleanup in here and more cleanup in async versions.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
Change isTableDisabled/Enabled implementation to ask the Master instead.
This will give the Master's TableStateManager's opinion rather than
client figuring it for themselves reading meta table direct.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
TODO: Cleanup in here. Go to master for state, not to meta.

M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
Logging cleanup.

M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
Shutdown access.

M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
Just cleanup.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
Add state holder for hbase:meta.
Removed unused methods.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
Shut down access.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
Allow hbase:meta to be disabled.

M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
Allow hbase:meta to be enabled.

@@ -2057,6 +2058,9 @@ public NonceGenerator getNonceGenerator() {

@Override
public TableState getTableState(TableName tableName) throws IOException {
// TODO: This doesn't work if tablename is hbase:meta. Need to ask Master.
// Other problems with this implementation are that it presumes state is
// available in Master. Would be good to hide how state is kept.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/hbase:meta/Master/

}
// TODO: This doesn't work if tablename is hbase:meta. Need to ask Master.
// Other problems with this implementation are that it presumes state is
// available in Master. Would be good to hide how state is kept.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Master/hbase:meta/

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 1m 8s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💛 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ branch-2 Compile Tests _
💙 mvndep 0m 13s Maven dependency ordering for branch
💚 mvninstall 5m 39s branch-2 passed
💚 compile 2m 2s branch-2 passed
💚 checkstyle 2m 50s branch-2 passed
💚 shadedjars 4m 45s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 33s branch-2 passed
💙 spotbugs 0m 36s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 6m 6s branch-2 passed
_ Patch Compile Tests _
💙 mvndep 0m 14s Maven dependency ordering for patch
💚 mvninstall 5m 22s the patch passed
💚 compile 2m 2s the patch passed
💚 javac 2m 2s the patch passed
💔 checkstyle 0m 27s hbase-common: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
💔 checkstyle 0m 41s hbase-client: The patch generated 6 new + 169 unchanged - 4 fixed = 175 total (was 173)
💔 checkstyle 1m 28s hbase-server: The patch generated 2 new + 50 unchanged - 11 fixed = 52 total (was 61)
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 4m 42s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 16m 59s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 javadoc 1m 41s the patch passed
💚 findbugs 6m 36s the patch passed
_ Other Tests _
💚 unit 3m 6s hbase-common in the patch passed.
💚 unit 2m 29s hbase-client in the patch passed.
💚 unit 0m 52s hbase-zookeeper in the patch passed.
💔 unit 235m 18s hbase-server in the patch failed.
💚 asflicense 1m 33s The patch does not generate ASF License warnings.
315m 54s
Reason Tests
Failed junit tests hadoop.hbase.client.TestAdmin2
hadoop.hbase.client.TestAsyncTableAdminApi3
hadoop.hbase.TestSplitMerge
hadoop.hbase.client.TestAsyncTableAdminApi2
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/Dockerfile
GITHUB PR #646
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 23dc5114e7cc 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-646/out/precommit/personality/provided.sh
git revision branch-2 / 0ed11fe
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/diff-checkstyle-hbase-common.txt
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/diff-checkstyle-hbase-client.txt
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/diff-checkstyle-hbase-server.txt
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/testReport/
Max. process+thread count 4523 (vs. ulimit of 10000)
modules C: hbase-common hbase-client hbase-zookeeper hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/1/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

    Make it so hbase:meta can be altered. TableState for hbase:meta
    is kept in Master. State is in-memory transient so if  Master
    fails, hbase:meta is ENABLED again. hbase:meta schema will be
    bootstrapped from the filesystem. Changes to filesystem schema
    are atomic so we should be ok if Master fails mid-edit (TBD)
    Undoes a bunch of guards that prevented our being able to edit
    hbase:meta. At minimmum, need to add in a bunch of WARNING.

    TODO: Tests, more clarity around hbase:meta table state, and undoing
    references to hard-coded hbase:meta regioninfo.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/MetaTableAccessor.java
     Throw illegal access exception if you try to use MetaTableAccessor
     getting state of the hbase:meta table.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ConnectionImplementation.java
     For table state, go to master rather than go to meta direct. Going
     to meta won't work for hbase;meta state. Puts load on Master.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/HBaseAdmin.java
     Change isTableDisabled/Enabled implementation to ask the Master instead.
     This will give the Master's TableStateManager's opinion rather than
     client figuring it for themselves reading meta table direct.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/RawAsyncHBaseAdmin.java
     TODO: Cleanup in here. Go to master for state, not to meta.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/client/ZKAsyncRegistry.java
     Logging cleanup.

    M hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/ZNodePaths.java
     Shutdown access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/TableDescriptors.java
     Just cleanup.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/TableStateManager.java
     Add state holder for hbase:meta.
     Removed unused methods.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/assignment/RegionStateStore.java
     Shut down access.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/DisableTableProcedure.java
     Allow hbase:meta to be disabled.

    M hbase-server/src/main/java/org/apache/hadoop/hbase/master/procedure/EnableTableProcedure.java
     Allow hbase:meta to be enabled.
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
💙 reexec 1m 35s Docker mode activated.
_ Prechecks _
💚 dupname 0m 0s No case conflicting files found.
💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
💚 @author 0m 0s The patch does not contain any @author tags.
💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-2 Compile Tests _
💙 mvndep 0m 13s Maven dependency ordering for branch
💚 mvninstall 5m 44s branch-2 passed
💚 compile 2m 0s branch-2 passed
💚 checkstyle 2m 56s branch-2 passed
💚 shadedjars 4m 38s branch has no errors when building our shaded downstream artifacts.
💚 javadoc 1m 32s branch-2 passed
💙 spotbugs 0m 34s Used deprecated FindBugs config; considering switching to SpotBugs.
💚 findbugs 6m 4s branch-2 passed
_ Patch Compile Tests _
💙 mvndep 0m 13s Maven dependency ordering for patch
💚 mvninstall 5m 15s the patch passed
💚 compile 1m 59s the patch passed
💚 javac 1m 59s the patch passed
💔 checkstyle 0m 25s hbase-common: The patch generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
💔 checkstyle 0m 43s hbase-client: The patch generated 20 new + 349 unchanged - 5 fixed = 369 total (was 354)
💔 checkstyle 1m 35s hbase-server: The patch generated 6 new + 372 unchanged - 14 fixed = 378 total (was 386)
💚 whitespace 0m 0s The patch has no whitespace issues.
💚 shadedjars 4m 37s patch has no errors when building our shaded downstream artifacts.
💚 hadoopcheck 17m 20s Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
💚 javadoc 1m 30s the patch passed
💔 findbugs 1m 24s hbase-client generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
_ Other Tests _
💚 unit 2m 47s hbase-common in the patch passed.
💚 unit 3m 34s hbase-client in the patch passed.
💚 unit 0m 48s hbase-zookeeper in the patch passed.
💔 unit 29m 35s hbase-server in the patch failed.
💚 asflicense 0m 53s The patch does not generate ASF License warnings.
109m 50s
Reason Tests
FindBugs module:hbase-client
Call to org.apache.hadoop.hbase.shaded.protobuf.generated.HBaseProtos$TableState.equals(org.apache.hadoop.hbase.client.TableState$State) in org.apache.hadoop.hbase.client.RawAsyncHBaseAdmin.lambda$null$32(TableState$State, MasterProtos$GetTableStateResponse) At RawAsyncHBaseAdmin.java:MasterProtos$GetTableStateResponse) At RawAsyncHBaseAdmin.java:[line 683]
Failed junit tests hadoop.hbase.client.TestAsyncAccessControlAdminApi
Subsystem Report/Notes
Docker Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/Dockerfile
GITHUB PR #646
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux a6021653900c 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24 10:55:24 UTC 2019 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-646/out/precommit/personality/provided.sh
git revision branch-2 / 6fe7063
Default Java 1.8.0_181
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/diff-checkstyle-hbase-common.txt
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/diff-checkstyle-hbase-client.txt
checkstyle https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/diff-checkstyle-hbase-server.txt
findbugs https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/new-findbugs-hbase-client.html
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/testReport/
Max. process+thread count 680 (vs. ulimit of 10000)
modules C: hbase-common hbase-client hbase-zookeeper hbase-server U: .
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-646/2/console
versions git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by Apache Yetus 0.11.0 https://yetus.apache.org

This message was automatically generated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants