HBASE-28011 : The logStats about LruBlockCache is not accurate #5344

guluo2016 · 2023-08-07T14:55:21Z

Apply this commit, LruBlockCache.logStats would print info,as follow:

INFO  [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=3.20 GB, usedSize=2.41 MB, freeSize=3.20 GB, max=3.20 GB, blockCount=4, accesses=13, hits=9, hitRatio=69.23%, , cachingAccesses=13, cachingHits=9, cachingHitsRatio=69.23%, evictions=29, evicted=0, evictedPerRun=0.0

Apache-HBase · 2023-08-07T15:29:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 34s	master passed
+1 💚	compile	0m 35s	master passed
+1 💚	shadedjars	4m 57s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 11s	the patch passed
+1 💚	compile	0m 36s	the patch passed
+1 💚	javac	0m 36s	the patch passed
+1 💚	shadedjars	4m 53s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
-1 ❌	unit	10m 3s	hbase-server in the patch failed.
		28m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5344
JIRA Issue	HBASE-28011
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 02ad79bdc704 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7cfa47d`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/testReport/
Max. process+thread count	1776 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-08-07T15:32:07Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 54s	master passed
+1 💚	compile	2m 24s	master passed
+1 💚	checkstyle	0m 34s	master passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 28s	the patch passed
+1 💚	compile	2m 23s	the patch passed
+1 💚	javac	2m 23s	the patch passed
+1 💚	checkstyle	0m 33s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	9m 29s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 37s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 29s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		31m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5344
JIRA Issue	HBASE-28011
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 3218f0ac80e1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7cfa47d`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-08-07T18:50:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 56s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	5m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 34s	the patch passed
+1 💚	compile	0m 43s	the patch passed
+1 💚	javac	0m 43s	the patch passed
+1 💚	shadedjars	4m 54s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 24s	the patch passed
		_ Other Tests _
+1 💚	unit	207m 41s	hbase-server in the patch passed.
		229m 59s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5344
JIRA Issue	HBASE-28011
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 159e9f18d742 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7cfa47d`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/testReport/
Max. process+thread count	4651 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani

one minor nit, else looks good

virajjasani · 2023-08-07T23:15:31Z

hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java

-    LruBlockCache.LOG.info("totalSize=" + StringUtils.byteDesc(totalSize) + ", " + "freeSize="
+    long usedSize = heapSize();
+    long freeSize = maxSize - usedSize;
+    LruBlockCache.LOG.info("usedSize=" + StringUtils.byteDesc(usedSize) + ", " + "freeSize="


how about we also add total size?

Thank for you comment, I will update later

Apache-HBase · 2023-08-08T14:46:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 31s	master passed
+1 💚	compile	2m 24s	master passed
+1 💚	checkstyle	0m 32s	master passed
+1 💚	spotless	0m 39s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 30s	the patch passed
+1 💚	compile	2m 22s	the patch passed
+1 💚	javac	2m 22s	the patch passed
+1 💚	checkstyle	0m 33s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	9m 25s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 35s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 8s	The patch does not generate ASF License warnings.
		30m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5344
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux b3960f7b3571 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7cfa47d`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani

+1

Apache-HBase · 2023-08-08T18:06:37Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 44s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	4m 55s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 36s	the patch passed
+1 💚	compile	0m 43s	the patch passed
+1 💚	javac	0m 43s	the patch passed
+1 💚	shadedjars	5m 0s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
-1 ❌	unit	209m 17s	hbase-server in the patch failed.
		231m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5344
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 80f3170f21a2 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7cfa47d`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/testReport/
Max. process+thread count	4870 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-08-08T18:52:10Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 56s	master passed
+1 💚	compile	0m 42s	master passed
+1 💚	shadedjars	5m 45s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 57s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	5m 35s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
-1 ❌	unit	251m 48s	hbase-server in the patch failed.
		276m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5344
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b80732a52174 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7cfa47d`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/testReport/
Max. process+thread count	4611 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5344/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…e#5344) (cherry picked from commit c26f2c4) Change-Id: I4e0d66c0b5015ca7e50efa3bf5c9fe9fee4ad78d

Modify block stats about LruBlockCache

215dd0f

virajjasani reviewed Aug 7, 2023

View reviewed changes

virajjasani changed the title ~~HBASE-28011:The logStats about LruBlockCache is not accurate~~ HBASE-28011 : The logStats about LruBlockCache is not accurate Aug 8, 2023

Keep total size

404c8ef

virajjasani approved these changes Aug 8, 2023

View reviewed changes

virajjasani merged commit ab4b1d8 into apache:master Aug 9, 2023

virajjasani pushed a commit that referenced this pull request Aug 9, 2023

HBASE-28011 : The logStats about LruBlockCache is not accurate (#5344)

d7be336

virajjasani pushed a commit that referenced this pull request Aug 9, 2023

HBASE-28011 : The logStats about LruBlockCache is not accurate (#5344)

438b591

virajjasani pushed a commit that referenced this pull request Aug 9, 2023

HBASE-28011 : The logStats about LruBlockCache is not accurate (#5344)

f27659b

virajjasani pushed a commit that referenced this pull request Aug 9, 2023

HBASE-28011 : The logStats about LruBlockCache is not accurate (#5344)

c26f2c4

guluo2016 deleted the hbase_lru branch August 27, 2023 14:32

vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024

HBASE-28011 : The logStats about LruBlockCache is not accurate (apach…

5eb4b83

…e#5344) (cherry picked from commit c26f2c4) Change-Id: I4e0d66c0b5015ca7e50efa3bf5c9fe9fee4ad78d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-28011 : The logStats about LruBlockCache is not accurate #5344

HBASE-28011 : The logStats about LruBlockCache is not accurate #5344

guluo2016 commented Aug 7, 2023 •

edited

Loading

Apache-HBase commented Aug 7, 2023

Apache-HBase commented Aug 7, 2023

Apache-HBase commented Aug 7, 2023

virajjasani left a comment

virajjasani Aug 7, 2023

guluo2016 Aug 8, 2023

Apache-HBase commented Aug 8, 2023

virajjasani left a comment

Apache-HBase commented Aug 8, 2023

Apache-HBase commented Aug 8, 2023

HBASE-28011 : The logStats about LruBlockCache is not accurate #5344

HBASE-28011 : The logStats about LruBlockCache is not accurate #5344

Conversation

guluo2016 commented Aug 7, 2023 • edited Loading

Apache-HBase commented Aug 7, 2023

Apache-HBase commented Aug 7, 2023

Apache-HBase commented Aug 7, 2023

virajjasani left a comment

Choose a reason for hiding this comment

virajjasani Aug 7, 2023

Choose a reason for hiding this comment

guluo2016 Aug 8, 2023

Choose a reason for hiding this comment

Apache-HBase commented Aug 8, 2023

virajjasani left a comment

Choose a reason for hiding this comment

Apache-HBase commented Aug 8, 2023

Apache-HBase commented Aug 8, 2023

guluo2016 commented Aug 7, 2023 •

edited

Loading