HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking #5389

teamconfx · 2023-09-11T15:46:16Z

Description of PR

https://issues.apache.org/jira/browse/HBASE-27990
This PR adds a precheck for hbase.blockcache.minblocksize such that it must be positive.

How was this patch tested?

(1) set hbase.blockcache.minblocksize=0
(2) run org.apache.hadoop.hbase.io.hfile.TestCacheConfig#testBucketCacheConfigL1L2Setup
The test throws a IllegalArgumentException rather than a division by zero error.

Apache-HBase · 2023-09-11T16:32:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 50s	master passed
+1 💚	compile	2m 24s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 33s	the patch passed
+1 💚	compile	2m 22s	the patch passed
+1 💚	javac	2m 22s	the patch passed
+1 💚	checkstyle	0m 36s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	9m 6s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 37s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		31m 37s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux d63d4f71ca8f 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `97d512b`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-09-11T20:02:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 12s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 47s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	shadedjars	4m 38s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 36s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	4m 37s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	220m 46s	hbase-server in the patch failed.
		242m 38s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 10c2c318659b 5.4.0-156-generic #173-Ubuntu SMP Tue Jul 11 07:25:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `97d512b`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/testReport/
Max. process+thread count	4823 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-09-11T20:04:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 36s	master passed
+1 💚	compile	0m 36s	master passed
+1 💚	shadedjars	5m 3s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 20s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	5m 1s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	the patch passed
		_ Other Tests _
+1 💚	unit	222m 32s	hbase-server in the patch passed.
		244m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 40986c7b59e0 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `97d512b`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/testReport/
Max. process+thread count	4458 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9 · 2023-09-23T06:21:21Z

hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/bucket/BucketCache.java

@@ -273,6 +273,7 @@ public BucketCache(String ioEngineName, long capacity, int blockSize, int[] buck
  public BucketCache(String ioEngineName, long capacity, int blockSize, int[] bucketSizes,
    int writerThreadNum, int writerQLen, String persistencePath, int ioErrorsTolerationDuration,
    Configuration conf) throws IOException {
+    Preconditions.checkArgument(blockSize > 0, "BucketCache capacity can not be less than 0");


Better to include the incorrect blockSize value in the error message.

Apache-HBase · 2024-03-20T22:25:49Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 17s	master passed
+1 💚	compile	0m 53s	master passed
+1 💚	shadedjars	5m 37s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 55s	the patch passed
+1 💚	compile	0m 53s	the patch passed
+1 💚	javac	0m 53s	the patch passed
+1 💚	shadedjars	5m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	12m 6s	hbase-server in the patch failed.
		34m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6bda447d1d7d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0763a74`
Default Java	Eclipse Adoptium-17.0.10+7
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/testReport/
Max. process+thread count	1828 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-03-20T22:35:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	5m 3s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 38s	master passed
+1 💚	compile	3m 19s	master passed
+1 💚	checkstyle	0m 41s	master passed
+1 💚	spotless	0m 49s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 51s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 59s	the patch passed
+1 💚	compile	3m 13s	the patch passed
+1 💚	javac	3m 13s	the patch passed
-0 ⚠️	checkstyle	0m 44s	hbase-server: The patch generated 1 new + 6 unchanged - 0 fixed = 7 total (was 6)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	5m 46s	Patch does not cause any errors with Hadoop 3.3.6.
-1 ❌	spotless	1m 6s	patch has 23 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	2m 52s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 22s	The patch does not generate ASF License warnings.
		44m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux a741436ffe9f 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0763a74`
Default Java	Eclipse Adoptium-11.0.17+8
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	77 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-03-21T01:59:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 45s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 38s	master passed
+1 💚	compile	1m 6s	master passed
+1 💚	shadedjars	5m 15s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 50s	the patch passed
+1 💚	compile	0m 49s	the patch passed
+1 💚	javac	0m 49s	the patch passed
+1 💚	shadedjars	5m 13s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	221m 10s	hbase-server in the patch passed.
		247m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 515bd80d7a5e 5.4.0-172-generic #190-Ubuntu SMP Fri Feb 2 23:24:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0763a74`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/testReport/
Max. process+thread count	5453 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-03-21T02:13:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 55s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 56s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	shadedjars	5m 37s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 33s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 17s	the patch passed
+1 💚	compile	0m 48s	the patch passed
+1 💚	javac	0m 48s	the patch passed
+1 💚	shadedjars	5m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
-1 ❌	unit	232m 47s	hbase-server in the patch failed.
		261m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.45 ServerAPI=1.45 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5389
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux aab8c6c5f434 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `0763a74`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/testReport/
Max. process+thread count	4368 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5389/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…ockSize value checking (#5389) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 5a0c4de)

add a precondition check that capacity of the bucketcache is positive

c655bad

Apache9 reviewed Sep 23, 2023

View reviewed changes

include value in the error message

3409038

Apache9 approved these changes Mar 28, 2024

View reviewed changes

Apache9 merged commit 5a0c4de into apache:master Mar 28, 2024
1 check failed

Apache9 pushed a commit that referenced this pull request Mar 28, 2024

HBASE-27990 BucketCache causes ArithmeticException due to improper bl…

81e4385

…ockSize value checking (#5389) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 5a0c4de)

Apache9 pushed a commit that referenced this pull request Mar 28, 2024

HBASE-27990 BucketCache causes ArithmeticException due to improper bl…

bb094dc

…ockSize value checking (#5389) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 5a0c4de)

Apache9 pushed a commit that referenced this pull request Mar 28, 2024

HBASE-27990 BucketCache causes ArithmeticException due to improper bl…

c057037

…ockSize value checking (#5389) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 5a0c4de)

Apache9 pushed a commit that referenced this pull request Mar 28, 2024

HBASE-27990 BucketCache causes ArithmeticException due to improper bl…

9e1c1cb

…ockSize value checking (#5389) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 5a0c4de)

Apache9 pushed a commit that referenced this pull request Mar 28, 2024

HBASE-27990 BucketCache causes ArithmeticException due to improper bl…

e2d7ec9

…ockSize value checking (#5389) Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 5a0c4de)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking #5389

HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking #5389

teamconfx commented Sep 11, 2023

Apache-HBase commented Sep 11, 2023

Apache-HBase commented Sep 11, 2023

Apache-HBase commented Sep 11, 2023

Apache9 Sep 23, 2023

teamconfx Mar 20, 2024

Apache-HBase commented Mar 20, 2024

Apache-HBase commented Mar 20, 2024

Apache-HBase commented Mar 21, 2024

Apache-HBase commented Mar 21, 2024

HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking #5389

HBASE-27990. BucketCache causes ArithmeticException due to improper blockSize value checking #5389

Conversation

teamconfx commented Sep 11, 2023

Description of PR

How was this patch tested?

Apache-HBase commented Sep 11, 2023

Apache-HBase commented Sep 11, 2023

Apache-HBase commented Sep 11, 2023

Apache9 Sep 23, 2023

Choose a reason for hiding this comment

teamconfx Mar 20, 2024

Choose a reason for hiding this comment

Apache-HBase commented Mar 20, 2024

Apache-HBase commented Mar 20, 2024

Apache-HBase commented Mar 21, 2024

Apache-HBase commented Mar 21, 2024