HBASE-28750 Region normalizer should work in off peak if config #6113

1458451310 · 2024-07-23T11:38:37Z

Region normalizer involves the splitting and merging of regions, which can cause jitter in online services, especially when there are many region normalizer plans. We should run this task during off peak hours if config.

Apache-HBase · 2024-07-23T12:26:20Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	3m 2s		master passed
+1 💚	compile	0m 59s		master passed
+1 💚	javadoc	0m 31s		master passed
+1 💚	shadedjars	5m 23s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 55s		the patch passed
+1 💚	compile	0m 58s		the patch passed
+1 💚	javac	0m 58s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed
+1 💚	shadedjars	5m 20s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	13m 26s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		35m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6113
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 9effec4ca0f6 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `aec62bc`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/testReport/
Max. process+thread count	1669 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-23T12:26:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 4s		master passed
+1 💚	compile	2m 56s		master passed
+1 💚	checkstyle	0m 36s		master passed
+1 💚	spotbugs	1m 34s		master passed
+1 💚	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 51s		the patch passed
+1 💚	compile	2m 54s		the patch passed
+1 💚	javac	2m 54s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 37s		the patch passed
+1 💚	spotbugs	1m 39s		the patch passed
+1 💚	hadoopcheck	10m 32s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 44s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 12s		The patch does not generate ASF License warnings.
		35m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6113
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 0e746eb74a97 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `aec62bc`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

wchevreuil

This is a nice addition. Is it possible add test for this condition in TestRegionNormalizerWorker?

wchevreuil · 2024-07-23T15:50:16Z

...e-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java

@@ -199,6 +207,21 @@ public void run() {
    }
  }

+  private void sleepToNextHour() {


nit: Add an info log saying offpeak hours is configured and we'll wait for offpeak hours to continue normalising.

1458451310 · 2024-07-26T09:16:05Z

This is a nice addition. Is it possible add test for this condition in TestRegionNormalizerWorker?

It is ok.

Apache-HBase · 2024-07-26T09:56:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 5s		master passed
+1 💚	compile	2m 56s		master passed
+1 💚	checkstyle	0m 38s		master passed
+1 💚	spotbugs	1m 34s		master passed
+1 💚	spotless	0m 46s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 56s		the patch passed
+1 💚	compile	2m 55s		the patch passed
+1 💚	javac	2m 55s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 38s	/results-checkstyle-hbase-server.txt	hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	spotbugs	1m 40s		the patch passed
+1 💚	hadoopcheck	10m 43s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 ❌	spotless	0m 39s		patch has 23 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
+1 💚	asflicense	0m 12s		The patch does not generate ASF License warnings.
		35m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6113
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux f5cf9d63837f 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `37b8af7`
Default Java	Eclipse Adoptium-17.0.11+9
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-26T13:38:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 36s		Docker mode activated.
-0 ⚠️	yetus	0m 2s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	3m 4s		master passed
+1 💚	compile	0m 59s		master passed
+1 💚	javadoc	0m 31s		master passed
+1 💚	shadedjars	5m 25s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 58s		the patch passed
+1 💚	compile	0m 58s		the patch passed
+1 💚	javac	0m 58s		the patch passed
+1 💚	javadoc	0m 30s		the patch passed
+1 💚	shadedjars	5m 17s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
-1 ❌	unit	232m 20s	/patch-unit-hbase-server.txt	hbase-server in the patch failed.
		257m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6113
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux ab85b4fb8cf5 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `37b8af7`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/testReport/
Max. process+thread count	5251 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/2/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-28T07:59:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	4m 2s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	2m 56s		master passed
+1 💚	compile	2m 56s		master passed
+1 💚	checkstyle	0m 37s		master passed
+1 💚	spotbugs	1m 32s		master passed
+1 💚	spotless	0m 43s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 53s		the patch passed
+1 💚	compile	2m 55s		the patch passed
+1 💚	javac	2m 55s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 37s	/results-checkstyle-hbase-server.txt	hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	spotbugs	1m 39s		the patch passed
+1 💚	hadoopcheck	10m 33s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
-1 ❌	spotless	0m 39s		patch has 23 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
+1 💚	asflicense	0m 10s		The patch does not generate ASF License warnings.
		38m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6113
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 008f7cf66619 5.4.0-186-generic #206-Ubuntu SMP Fri Apr 26 12:31:10 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e066a29`
Default Java	Eclipse Adoptium-17.0.11+9
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-07-28T11:29:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	2m 57s		master passed
+1 💚	compile	0m 58s		master passed
+1 💚	javadoc	0m 30s		master passed
+1 💚	shadedjars	5m 23s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 0s		the patch passed
+1 💚	compile	0m 59s		the patch passed
+1 💚	javac	0m 59s		the patch passed
+1 💚	javadoc	0m 29s		the patch passed
+1 💚	shadedjars	5m 23s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	223m 35s		hbase-server in the patch passed.
		248m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6113
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 42adfebb16c2 5.4.0-182-generic #202-Ubuntu SMP Fri Apr 26 12:29:36 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `e066a29`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/testReport/
Max. process+thread count	5576 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6113/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

1458451310 · 2024-08-30T07:43:25Z

@wchevreuil hello, If you have time, please take a review again.

wchevreuil reviewed Jul 23, 2024

View reviewed changes

1458451310 closed this Jul 24, 2024

1458451310 reopened this Jul 24, 2024

1458451310 force-pushed the HBASE-28750 branch from aec62bc to 37b8af7 Compare July 26, 2024 09:13

HBASE-28750 Region normalizer should work in off peak if config

e066a29

1458451310 force-pushed the HBASE-28750 branch from 37b8af7 to e066a29 Compare July 28, 2024 07:06

1458451310 requested a review from wchevreuil July 28, 2024 07:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28750 Region normalizer should work in off peak if config #6113

HBASE-28750 Region normalizer should work in off peak if config #6113

1458451310 commented Jul 23, 2024

Apache-HBase commented Jul 23, 2024

Apache-HBase commented Jul 23, 2024

wchevreuil left a comment

wchevreuil Jul 23, 2024

1458451310 commented Jul 26, 2024

Apache-HBase commented Jul 26, 2024

Apache-HBase commented Jul 26, 2024

Apache-HBase commented Jul 28, 2024

Apache-HBase commented Jul 28, 2024

1458451310 commented Aug 30, 2024

HBASE-28750 Region normalizer should work in off peak if config #6113

Are you sure you want to change the base?

HBASE-28750 Region normalizer should work in off peak if config #6113

Conversation

1458451310 commented Jul 23, 2024

Apache-HBase commented Jul 23, 2024

Apache-HBase commented Jul 23, 2024

wchevreuil left a comment

Choose a reason for hiding this comment

wchevreuil Jul 23, 2024

Choose a reason for hiding this comment

1458451310 commented Jul 26, 2024

Apache-HBase commented Jul 26, 2024

Apache-HBase commented Jul 26, 2024

Apache-HBase commented Jul 28, 2024

Apache-HBase commented Jul 28, 2024

1458451310 commented Aug 30, 2024