HBASE-27556 Reuse Zookeeper session of Master in LogCleaner #4946

petersomogyi · 2023-01-06T09:56:32Z

Backport of HBASE-23340
hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB))

Backport of HBASE-23340 hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB)) Signed-off-by: Duo Zhang <[email protected]> Signed-off-by: Pankaj Kumar<[email protected]>

Apache-HBase · 2023-01-06T10:38:22Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 58s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	2m 35s	branch-2.4 passed
+1 💚	compile	2m 17s	branch-2.4 passed
+1 💚	checkstyle	0m 33s	branch-2.4 passed
+1 💚	spotless	0m 40s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 20s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 14s	the patch passed
+1 💚	compile	2m 19s	the patch passed
+1 💚	javac	2m 19s	the patch passed
+1 💚	checkstyle	0m 31s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	15m 42s	Patch does not cause any errors with Hadoop 2.10.2 or 3.1.4 3.2.4 3.3.4.
+1 💚	spotless	0m 36s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 26s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		33m 5s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4946
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 692eb72ca510 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `9cae2bb`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-06T13:08:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 53s	Docker mode activated.
-0 ⚠️	yetus	0m 5s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.4 Compile Tests _
-1 ❌	mvninstall	2m 1s	root in branch-2.4 failed.
+1 💚	compile	0m 32s	branch-2.4 passed
+1 💚	shadedjars	4m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	1m 55s	the patch passed
+1 💚	compile	0m 32s	the patch passed
+1 💚	javac	0m 32s	the patch passed
+1 💚	shadedjars	4m 9s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	164m 44s	hbase-server in the patch passed.
		183m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4946
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a0a60e83b14c 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `9cae2bb`
Default Java	Temurin-1.8.0_352-b08
mvninstall	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/artifact/yetus-jdk8-hadoop2-check/output/branch-mvninstall-root.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/testReport/
Max. process+thread count	2980 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-06T13:11:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 28s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	3m 59s	branch-2.4 passed
+1 💚	compile	0m 56s	branch-2.4 passed
+1 💚	shadedjars	4m 50s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 4s	the patch passed
+1 💚	compile	0m 51s	the patch passed
+1 💚	javac	0m 51s	the patch passed
+1 💚	shadedjars	4m 52s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	161m 13s	hbase-server in the patch passed.
		186m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4946
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux d44682cc4bb8 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `9cae2bb`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/testReport/
Max. process+thread count	3086 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-06T14:02:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 17s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	3m 7s	branch-2.4 passed
+1 💚	compile	3m 0s	branch-2.4 passed
+1 💚	checkstyle	0m 40s	branch-2.4 passed
+1 💚	spotless	0m 46s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 42s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 7s	the patch passed
+1 💚	compile	3m 3s	the patch passed
+1 💚	javac	3m 3s	the patch passed
+1 💚	checkstyle	0m 39s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	21m 38s	Patch does not cause any errors with Hadoop 2.10.2 or 3.1.4 3.2.4 3.3.4.
+1 💚	spotless	0m 51s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 58s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		43m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4946
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 526b6e676cf4 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `9cae2bb`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-06T16:20:43Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 53s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	1m 53s	branch-2.4 passed
+1 💚	compile	0m 31s	branch-2.4 passed
+1 💚	shadedjars	4m 11s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	1m 50s	the patch passed
+1 💚	compile	0m 35s	the patch passed
+1 💚	javac	0m 35s	the patch passed
+1 💚	shadedjars	4m 11s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	163m 36s	hbase-server in the patch passed.
		182m 5s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#4946
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 9ba8fff265b4 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `9cae2bb`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/testReport/
Max. process+thread count	2515 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-06T16:24:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 55s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	2m 21s	branch-2.4 passed
+1 💚	compile	0m 41s	branch-2.4 passed
+1 💚	shadedjars	4m 28s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 20s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	4m 25s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
+1 💚	unit	165m 6s	hbase-server in the patch passed.
		185m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4946
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux c4ee3019e825 5.4.0-1088-aws #96~18.04.1-Ubuntu SMP Mon Oct 17 02:57:48 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `9cae2bb`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/testReport/
Max. process+thread count	2940 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4946/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

) Backport of HBASE-23340 hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB)) Signed-off-by: Wellington Ramos Chevreuil <[email protected]> Signed-off-by: Pankaj Kumar<[email protected]> (cherry picked from commit 695dce5) Change-Id: Id47c83dc38b54c62a57138cfd839279b6c26caa1

petersomogyi mentioned this pull request Jan 6, 2023

HBASE-23340 hmaster /hbase/replication/rs session expired (hbase replication default value is true, we don't use ) causes logcleaner can not clean oldWALs, which resulits in oldWALs too large (more than 2TB) #4944

Closed

wchevreuil approved these changes Jan 6, 2023

View reviewed changes

pankaj72981 approved these changes Jan 6, 2023

View reviewed changes

petersomogyi merged commit 695dce5 into apache:branch-2.4 Jan 6, 2023

petersomogyi deleted the HBASE-27556 branch January 6, 2023 19:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27556 Reuse Zookeeper session of Master in LogCleaner #4946

HBASE-27556 Reuse Zookeeper session of Master in LogCleaner #4946

petersomogyi commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

HBASE-27556 Reuse Zookeeper session of Master in LogCleaner #4946

HBASE-27556 Reuse Zookeeper session of Master in LogCleaner #4946

Conversation

petersomogyi commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023

Apache-HBase commented Jan 6, 2023