HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem #461

infraio · 2019-08-07T09:10:52Z

No description provided.

Apache-HBase · 2019-08-07T15:31:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	98	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1	mvninstall	368	master passed
+1	compile	53	master passed
+1	checkstyle	77	master passed
+1	shadedjars	265	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	34	master passed
0	spotbugs	235	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	231	master passed
		_ Patch Compile Tests _
+1	mvninstall	294	the patch passed
+1	compile	54	the patch passed
+1	javac	54	the patch passed
+1	checkstyle	73	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	268	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	993	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	38	the patch passed
+1	findbugs	245	the patch passed
		_ Other Tests _
-1	unit	18709	hbase-server in the patch failed.
+1	asflicense	50	The patch does not generate ASF License warnings.
		22304

Reason	Tests
Failed junit tests	hadoop.hbase.client.TestSnapshotTemporaryDirectory
	hadoop.hbase.master.TestAssignmentManagerMetrics
	hadoop.hbase.replication.TestReplicationDisableInactivePeer
	hadoop.hbase.client.TestSnapshotDFSTemporaryDirectory
	hadoop.hbase.tool.TestSecureBulkLoadHFiles
	hadoop.hbase.replication.TestReplicationSmallTests
	hadoop.hbase.client.TestFromClientSide
	hadoop.hbase.tool.TestBulkLoadHFiles

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/1/artifact/out/Dockerfile
GITHUB PR	#461
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 4ee2576da3c1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-461/out/precommit/personality/provided.sh
git revision	master / `07c6bb1`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/1/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/1/testReport/
Max. process+thread count	5109 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/1/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

the-sakthi · 2019-08-07T17:09:45Z

hbase-server/src/main/java/org/apache/hadoop/hbase/master/HbckChore.java

        HbckRegionInfo hri = regionInfoMap.get(encodedRegionName);
        if (hri == null) {
          orphanRegionsOnFS.add(encodedRegionName);
          continue;
        }
-        HbckRegionInfo.HdfsEntry hdfsEntry =


Now that we aren't using this constructor anymore, do we need to still keep that?

Let me try it.

the-sakthi

Okay the fs.listStatus(tableDir.getPath()) lists .tmp & .tabledesc. Have verified that the added UT fails without the fix. A small nit.

Apache-HBase · 2019-08-07T17:20:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	41	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1	mvninstall	339	master passed
+1	compile	55	master passed
+1	checkstyle	76	master passed
+1	shadedjars	284	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	40	master passed
0	spotbugs	241	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	239	master passed
		_ Patch Compile Tests _
+1	mvninstall	313	the patch passed
+1	compile	56	the patch passed
+1	javac	56	the patch passed
+1	checkstyle	77	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	285	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	969	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	37	the patch passed
+1	findbugs	251	the patch passed
		_ Other Tests _
-1	unit	8514	hbase-server in the patch failed.
+1	asflicense	34	The patch does not generate ASF License warnings.
		12024

Reason	Tests
Failed junit tests	hadoop.hbase.regionserver.TestHRegionWithInMemoryFlush

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/2/artifact/out/Dockerfile
GITHUB PR	#461
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 03eefb1e84bd 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23 09:17:01 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-461/out/precommit/personality/provided.sh
git revision	master / `a9773ec`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/2/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/2/testReport/
Max. process+thread count	4591 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/2/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

the-sakthi

Okay the fs.listStatus(tableDir.getPath()) lists .tmp & .tabledesc. Have verified that the added UT fails without the fix. A small nit.

Apache-HBase · 2019-08-07T21:06:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	42	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1	mvninstall	342	master passed
+1	compile	54	master passed
+1	checkstyle	72	master passed
+1	shadedjars	277	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	37	master passed
0	spotbugs	235	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	233	master passed
		_ Patch Compile Tests _
+1	mvninstall	311	the patch passed
+1	compile	54	the patch passed
+1	javac	54	the patch passed
+1	checkstyle	74	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	284	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	971	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	40	the patch passed
+1	findbugs	279	the patch passed
		_ Other Tests _
-1	unit	8886	hbase-server in the patch failed.
+1	asflicense	34	The patch does not generate ASF License warnings.
		12398

Reason	Tests
Failed junit tests	hadoop.hbase.replication.TestReplicationDroppedTables

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/3/artifact/out/Dockerfile
GITHUB PR	#461
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux 1c4ab41866be 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23 09:17:01 UTC 2019 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-461/out/precommit/personality/provided.sh
git revision	master / `49839e4`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/3/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/3/testReport/
Max. process+thread count	4485 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/3/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

infraio · 2019-08-08T00:06:09Z

Okay the fs.listStatus(tableDir.getPath()) lists .tmp & .tabledesc. Have verified that the added UT fails without the fix. A small nit.

Yes. Thanks for the verify.

) Signed-off-by: Sakthi <[email protected]>

Apache-HBase · 2019-08-08T04:41:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	67	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	hbaseanti	0	Patch does not have any anti-patterns.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+1	mvninstall	386	master passed
+1	compile	54	master passed
+1	checkstyle	74	master passed
+1	shadedjars	273	branch has no errors when building our shaded downstream artifacts.
+1	javadoc	37	master passed
0	spotbugs	248	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	246	master passed
		_ Patch Compile Tests _
+1	mvninstall	300	the patch passed
+1	compile	55	the patch passed
+1	javac	55	the patch passed
+1	checkstyle	77	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedjars	278	patch has no errors when building our shaded downstream artifacts.
+1	hadoopcheck	936	Patch does not cause any errors with Hadoop 2.8.5 2.9.2 or 3.1.2.
+1	javadoc	37	the patch passed
+1	findbugs	253	the patch passed
		_ Other Tests _
-1	unit	17041	hbase-server in the patch failed.
+1	asflicense	27	The patch does not generate ASF License warnings.
		20511

Reason	Tests
Failed junit tests	hadoop.hbase.regionserver.TestSplitTransactionOnCluster
	hadoop.hbase.client.TestFromClientSide
	hadoop.hbase.client.TestSnapshotTemporaryDirectory
	hadoop.hbase.master.assignment.TestOpenRegionProcedureHang
	hadoop.hbase.client.TestAsyncTableBatch

Subsystem	Report/Notes
Docker	Client=19.03.1 Server=19.03.1 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/4/artifact/out/Dockerfile
GITHUB PR	#461
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname	Linux d0fca3344f76 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 GNU/Linux
Build tool	maven
Personality	/home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-461/out/precommit/personality/provided.sh
git revision	master / `1b168cd`
Default Java	1.8.0_181
unit	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/4/artifact/out/patch-unit-hbase-server.txt
Test Results	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/4/testReport/
Max. process+thread count	4992 (vs. ulimit of 10000)
modules	C: hbase-server U: hbase-server
Console output	https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-461/4/console
versions	git=2.11.0 maven=2018-06-17T18:33:14Z) findbugs=3.1.11
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

…pache#461) Signed-off-by: Sakthi <[email protected]>

…pache#461) Signed-off-by: Sakthi <[email protected]> (cherry picked from commit 6ec251c) Change-Id: I7c629180bf99058afa5d59bce3e6bfb77ee9cfee

the-sakthi reviewed Aug 7, 2019

View reviewed changes

the-sakthi approved these changes Aug 7, 2019

View reviewed changes

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem

a5589a9

infraio force-pushed the HBASE-22807 branch from 750b6ab to a5589a9 Compare August 8, 2019 00:16

infraio merged commit 547cec4 into apache:master Aug 8, 2019

infraio deleted the HBASE-22807 branch August 8, 2019 00:17

asfgit pushed a commit that referenced this pull request Aug 8, 2019

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461

ee40d47

) Signed-off-by: Sakthi <[email protected]>

asfgit pushed a commit that referenced this pull request Aug 8, 2019

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461

6ec251c

) Signed-off-by: Sakthi <[email protected]>

asfgit pushed a commit that referenced this pull request Aug 8, 2019

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461

5cec62f

) Signed-off-by: Sakthi <[email protected]>

asfgit pushed a commit that referenced this pull request Aug 8, 2019

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (#461

43b4059

) Signed-off-by: Sakthi <[email protected]>

infraio added a commit to infraio/hbase that referenced this pull request Aug 17, 2020

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem (a…

abe16f2

…pache#461) Signed-off-by: Sakthi <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem #461

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem #461

infraio commented Aug 7, 2019

Apache-HBase commented Aug 7, 2019

the-sakthi Aug 7, 2019

infraio Aug 8, 2019

the-sakthi left a comment

Apache-HBase commented Aug 7, 2019

the-sakthi left a comment

Apache-HBase commented Aug 7, 2019

infraio commented Aug 8, 2019

Apache-HBase commented Aug 8, 2019

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem #461

HBASE-22807 HBCK Report showed wrong orphans regions on FileSystem #461

Conversation

infraio commented Aug 7, 2019

Apache-HBase commented Aug 7, 2019

the-sakthi Aug 7, 2019

Choose a reason for hiding this comment

infraio Aug 8, 2019

Choose a reason for hiding this comment

the-sakthi left a comment

Choose a reason for hiding this comment

Apache-HBase commented Aug 7, 2019

the-sakthi left a comment

Choose a reason for hiding this comment

Apache-HBase commented Aug 7, 2019

infraio commented Aug 8, 2019

Apache-HBase commented Aug 8, 2019