HBASE-25709 Close region may stuck when region is compacting and skip… #4536

sunhelly · 2022-06-15T04:06:43Z

…ped most cells read

Apache-HBase · 2022-06-15T04:42:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 25s	master passed
+1 💚	compile	2m 14s	master passed
+1 💚	checkstyle	0m 32s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 20s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 13s	the patch passed
+1 💚	compile	2m 11s	the patch passed
+1 💚	javac	2m 11s	the patch passed
+1 💚	checkstyle	0m 31s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	11m 33s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
-1 ❌	spotless	0m 37s	patch has 63 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	1m 21s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		32m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 1c0f00101764 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7fc1674`
Default Java	AdoptOpenJDK-1.8.0_282-b08
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	69 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-06-15T07:54:46Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	6m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 43s	master passed
+1 💚	compile	0m 46s	master passed
+1 💚	shadedjars	3m 42s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 35s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	3m 41s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	201m 8s	hbase-server in the patch failed.
		224m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0f2256e40c18 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `7fc1674`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/testReport/
Max. process+thread count	2498 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-06-15T15:28:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 30s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 41s	master passed
+1 💚	compile	2m 58s	master passed
+1 💚	checkstyle	0m 37s	master passed
+1 💚	spotless	0m 51s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 35s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 39s	the patch passed
+1 💚	compile	3m 5s	the patch passed
+1 💚	javac	3m 5s	the patch passed
+1 💚	checkstyle	0m 38s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	17m 14s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotless	1m 4s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 3s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		43m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux b1fd7797c845 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b365748`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	64 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-06-15T18:21:16Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 44s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 27s	master passed
+1 💚	compile	0m 33s	master passed
+1 💚	shadedjars	3m 52s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 6s	the patch passed
+1 💚	compile	0m 34s	the patch passed
+1 💚	javac	0m 34s	the patch passed
+1 💚	shadedjars	3m 49s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	199m 49s	hbase-server in the patch passed.
		217m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 55837a1a8c6a 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b365748`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/testReport/
Max. process+thread count	2464 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-06-15T18:32:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 34s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 2s	master passed
+1 💚	compile	0m 52s	master passed
+1 💚	shadedjars	4m 37s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 21s	the patch passed
+1 💚	compile	0m 57s	the patch passed
+1 💚	javac	0m 57s	the patch passed
+1 💚	shadedjars	4m 17s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	the patch passed
		_ Other Tests _
+1 💚	unit	204m 59s	hbase-server in the patch passed.
		227m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 0cc3a9198e46 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b365748`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/testReport/
Max. process+thread count	2633 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

sunhelly · 2022-08-24T11:47:11Z

Ping @virajjasani , please take a look in your convenience, thanks.

virajjasani · 2022-08-26T17:37:32Z

Sorry @sunhelly, could not get to it for long time. Let me try to get back to this.
Btw is the change in setToNewRow() something that was previously missing? Just trying to understand it better. Thank you @sunhelly

sunhelly · 2022-08-29T04:24:23Z

Thanks for your reply, @virajjasani . Yes, the condition insetToNewRow() is not complete. The problem here is that, after returned by the heartbeat cells, the matcher was unexpectedly reset, then all the deletes of the row is cleared, and the next same row cells will be wrongly matched. I think the matcher reset should only happen when scanner reaching new rows. Thanks.

sunhelly · 2023-01-29T09:22:15Z

Ping @virajjasani, please take a look in your convenience, thanks.

virajjasani · 2023-02-03T21:05:50Z

Sorry @sunhelly, could not get time. Will try my best to come back on this soon, but in the meantime if anyone else also wants to review, that is also fine.

sunhelly · 2023-02-16T11:28:12Z

Hi @bbeaudreault , could you take a review for this PR in your convenience? Thanks.

Apache-HBase · 2023-02-16T12:40:45Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	8m 22s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 42s	master passed
+1 💚	compile	3m 9s	master passed
+1 💚	checkstyle	0m 39s	master passed
+1 💚	spotless	0m 52s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 44s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 5s	the patch passed
+1 💚	compile	3m 8s	the patch passed
+1 💚	javac	3m 8s	the patch passed
+1 💚	checkstyle	0m 39s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 52s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	1m 3s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 41s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 22s	The patch does not generate ASF License warnings.
		64m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 247cabe6a887 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2bbe036`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-02-16T15:25:50Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 2s	master passed
+1 💚	compile	0m 41s	master passed
+1 💚	shadedjars	5m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 24s	the patch passed
+1 💚	compile	0m 39s	the patch passed
+1 💚	javac	0m 39s	the patch passed
+1 💚	shadedjars	4m 49s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
+1 💚	unit	205m 48s	hbase-server in the patch passed.
		229m 51s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 7f53484e2d6c 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2bbe036`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/testReport/
Max. process+thread count	2487 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-02-16T15:46:17Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	8m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	5m 40s	master passed
+1 💚	compile	1m 5s	master passed
+1 💚	shadedjars	5m 20s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 34s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	5m 13s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	5m 18s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	the patch passed
		_ Other Tests _
+1 💚	unit	212m 32s	hbase-server in the patch passed.
		250m 14s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cffc1c2cfef1 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2bbe036`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/testReport/
Max. process+thread count	2613 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

bbeaudreault · 2023-02-16T16:01:43Z

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

+        // And when reaching the heartbeat cells, try to return from the loop.
+        if (
+          scannerContext.checkSizeLimit(LimitScope.BETWEEN_CELLS)
+            || kvsScanned % cellsPerHeartbeatCheck == 0


should this only apply to !matcher.isUserScan()?

We have discussed about the feature applying to compaction scan or user scan in https://issues.apache.org/jira/browse/HBASE-25709. Actually, we want the scans not stuck/loop in skip cells, and both the user and the compaction scans should return as soon as possible.

After HBASE-27558 that you added size limit checker in the loop, it can applies to only compaction scans now.
if (scannerContext.checkSizeLimit(LimitScope.BETWEEN_CELLS)) { return scannerContext.setScannerState(NextState.MORE_VALUES).hasMoreValues(); }

I wonder actually if all of this can be simplified based on my change in HBASE-27558 -- What if we just did scannerContext.setSizeLimit(ScannerContext.LimitScope.BETWEEN_CELLS, Long.MAX_VALUE, Long.MAX_VALUE, 100*1024*1024) in Compactor? Force the StoreScanner to return every 100mb or so of blocks scanned.

I think the kvsScanned % cellsPerHeartbeatCheck was meant to avoid the cost of checking time limit (which requires hitting the clock). You're re-using it here as well to force compaction to check-in periodically based on cells/bytes scanned. But that's actually the whole point behind the changes in HBASE-27558, so that makes me think we should just apply the same change to compactions by specificying a block size limit in Compator.

Seems good idea. It's reasonable to change the solution after the headway in HBASE-27558. Thanks.

...server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java

bbeaudreault

I'm still confused by what the problem is and how these changes fix it. I understand it has something to do with large rows and TTL, but the changes here aren't super obvious how they help

I also am concerned about adding extra unnecessary comparisons in the hot code

sunhelly · 2023-02-27T08:22:02Z

I'm still confused by what the problem is and how these changes fix it. I understand it has something to do with large rows and TTL, but the changes here aren't super obvious how they help

The propose of this issue is to address the infinate loop in scan(compaction) caused by expired cells, as described in HBASE-25709, since the SKIP in InternalScanner#next(List<Cell> result, ScannerContext scannerContext) prevents checking the limit of batch, which is the only limit checker in compaction scanner context.

I also am concerned about adding extra unnecessary comparisons in the hot code

When the matcher.currentRow() is set to null by the next() steps cells, it will not brings extra comparisons. And it's also a common condition to enter setToNewRow. So I think moving the comparison from StoreScanner to ScanQueryMatcher is reasonable, WDYT? @bbeaudreault

Apache-HBase · 2023-02-27T09:34:23Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 24s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 38s	master passed
+1 💚	compile	2m 27s	master passed
+1 💚	checkstyle	0m 34s	master passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 21s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 16s	the patch passed
+1 💚	compile	2m 25s	the patch passed
+1 💚	javac	2m 25s	the patch passed
+1 💚	checkstyle	0m 30s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 53s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 38s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 28s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 10s	The patch does not generate ASF License warnings.
		38m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux bd153245fe70 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f20efaf`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	80 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-02-27T12:48:37Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 19s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 59s	master passed
+1 💚	compile	0m 39s	master passed
+1 💚	shadedjars	4m 17s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 51s	the patch passed
+1 💚	compile	0m 38s	the patch passed
+1 💚	javac	0m 38s	the patch passed
+1 💚	shadedjars	4m 14s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	the patch passed
		_ Other Tests _
-1 ❌	unit	211m 12s	hbase-server in the patch failed.
		232m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a3d0601168fa 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f20efaf`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/testReport/
Max. process+thread count	2530 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-02-27T13:42:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 13s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 2s	master passed
+1 💚	compile	0m 51s	master passed
+1 💚	shadedjars	5m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 19s	the patch passed
+1 💚	compile	1m 1s	the patch passed
+1 💚	javac	1m 1s	the patch passed
+1 💚	shadedjars	5m 3s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	the patch passed
		_ Other Tests _
-1 ❌	unit	258m 10s	hbase-server in the patch failed.
		285m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f457167836e0 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f20efaf`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/testReport/
Max. process+thread count	2444 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/4/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…ped most cells read

Apache-HBase · 2023-03-02T10:16:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	5m 14s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 29s	master passed
+1 💚	compile	2m 56s	master passed
+1 💚	checkstyle	0m 52s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 57s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 26s	the patch passed
+1 💚	compile	2m 55s	the patch passed
+1 💚	javac	2m 55s	the patch passed
+1 💚	checkstyle	0m 50s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 15s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 14s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 20s	The patch does not generate ASF License warnings.
		45m 53s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux f796d5fec5af 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `d2b0074`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-03-02T13:28:31Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 50s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 29s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 0s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	shadedjars	4m 36s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 34s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 47s	the patch passed
+1 💚	compile	0m 50s	the patch passed
+1 💚	javac	0m 50s	the patch passed
+1 💚	shadedjars	4m 36s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 33s	the patch passed
		_ Other Tests _
+1 💚	unit	2m 2s	hbase-common in the patch passed.
-1 ❌	unit	209m 45s	hbase-server in the patch failed.
		237m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 51f09149b39b 5.4.0-1092-aws #100~18.04.2-Ubuntu SMP Tue Nov 29 08:39:52 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `d2b0074`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/testReport/
Max. process+thread count	2335 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-03-02T13:42:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	5m 49s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 8s	Maven dependency ordering for branch
+1 💚	mvninstall	3m 24s	master passed
+1 💚	compile	1m 4s	master passed
+1 💚	shadedjars	4m 24s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 42s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	3m 21s	the patch passed
+1 💚	compile	1m 6s	the patch passed
+1 💚	javac	1m 6s	the patch passed
+1 💚	shadedjars	4m 26s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	the patch passed
		_ Other Tests _
+1 💚	unit	2m 28s	hbase-common in the patch passed.
+1 💚	unit	218m 35s	hbase-server in the patch passed.
		251m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4536
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 75f8ce70f00e 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `d2b0074`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/testReport/
Max. process+thread count	2581 (vs. ulimit of 30000)
modules	C: hbase-common hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4536/5/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java

…ped most cells read (apache#4536) Signed-off-by: Bryan Beaudreault <[email protected]>

…ped most cells read (#4536) (#5086) Signed-off-by: Bryan Beaudreault <[email protected]>

…ped most cells read (#4536) (#5085) Signed-off-by: Bryan Beaudreault <[email protected]>

…and skipped most cells read (#4536)" This reverts commit 37858bb.

…and skipped most cells read (#4536) (#5086)" This reverts commit 1ac7bbc.

sunhelly requested a review from virajjasani June 20, 2022 11:06

sunhelly force-pushed the HBASE-25709 branch from e622690 to 62b1840 Compare February 16, 2023 11:26

bbeaudreault reviewed Feb 16, 2023

View reviewed changes

...server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/ScanQueryMatcher.java Outdated Show resolved Hide resolved

bbeaudreault reviewed Feb 16, 2023

View reviewed changes

sunhelly force-pushed the HBASE-25709 branch 2 times, most recently from c5f6d1a to ec76106 Compare February 27, 2023 08:49

HBASE-25709 Close region may stuck when region is compacting and skip…

4101d93

…ped most cells read

sunhelly force-pushed the HBASE-25709 branch from ec76106 to 4101d93 Compare March 2, 2023 09:28

bbeaudreault reviewed Mar 4, 2023

View reviewed changes

hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java Show resolved Hide resolved

bbeaudreault approved these changes Mar 6, 2023

View reviewed changes

sunhelly merged commit 37858bb into apache:master Mar 6, 2023

sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 6, 2023

HBASE-25709 Close region may stuck when region is compacting and skip…

f4c9f08

…ped most cells read (apache#4536) Signed-off-by: Bryan Beaudreault <[email protected]>

sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 6, 2023

HBASE-25709 Close region may stuck when region is compacting and skip…

c824b8d

…ped most cells read (apache#4536) Signed-off-by: Bryan Beaudreault <[email protected]>

This was referenced Mar 6, 2023

Backport 'HBASE-25709 Close region may stuck when region is compacting and skip… #5085

Merged

Backport 'HBASE-25709 Close region may stuck when region is compacting and skip… #5086

Merged

sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 6, 2023

HBASE-25709 Close region may stuck when region is compacting and skip…

ad48ef7

…ped most cells read (apache#4536) Signed-off-by: Bryan Beaudreault <[email protected]>

sunhelly added a commit to sunhelly/hbase that referenced this pull request Mar 6, 2023

HBASE-25709 Close region may stuck when region is compacting and skip…

1582e69

…ped most cells read (apache#4536) Signed-off-by: Bryan Beaudreault <[email protected]>

sunhelly added a commit that referenced this pull request Mar 6, 2023

HBASE-25709 Close region may stuck when region is compacting and skip…

1ac7bbc

…ped most cells read (#4536) (#5086) Signed-off-by: Bryan Beaudreault <[email protected]>

sunhelly added a commit that referenced this pull request Mar 6, 2023

HBASE-25709 Close region may stuck when region is compacting and skip…

1ac6f16

…ped most cells read (#4536) (#5085) Signed-off-by: Bryan Beaudreault <[email protected]>

sunhelly added a commit that referenced this pull request Mar 6, 2023

Revert "HBASE-25709 Close region may stuck when region is compacting …

7abea1c

…and skipped most cells read (#4536)" This reverts commit 37858bb.

sunhelly added a commit that referenced this pull request Mar 6, 2023

Revert "HBASE-25709 Close region may stuck when region is compacting …

74c893c

…and skipped most cells read (#4536) (#5086)" This reverts commit 1ac7bbc.

asfgit pushed a commit that referenced this pull request Mar 6, 2023

Revert "HBASE-25709 Close region may stuck when region is compacting …

2cdcab2

…and skipped most cells read (#4536) (#5086)" This reverts commit 1ac7bbc.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-25709 Close region may stuck when region is compacting and skip… #4536

HBASE-25709 Close region may stuck when region is compacting and skip… #4536

sunhelly commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

sunhelly commented Aug 24, 2022

virajjasani commented Aug 26, 2022 •

edited

Loading

sunhelly commented Aug 29, 2022

sunhelly commented Jan 29, 2023

virajjasani commented Feb 3, 2023

sunhelly commented Feb 16, 2023

Apache-HBase commented Feb 16, 2023

Apache-HBase commented Feb 16, 2023

Apache-HBase commented Feb 16, 2023

bbeaudreault Feb 16, 2023

sunhelly Feb 27, 2023

sunhelly Feb 27, 2023

bbeaudreault Feb 27, 2023

bbeaudreault Feb 27, 2023

sunhelly Feb 28, 2023

bbeaudreault left a comment

sunhelly commented Feb 27, 2023

Apache-HBase commented Feb 27, 2023

Apache-HBase commented Feb 27, 2023

Apache-HBase commented Feb 27, 2023

Apache-HBase commented Mar 2, 2023

Apache-HBase commented Mar 2, 2023

Apache-HBase commented Mar 2, 2023

HBASE-25709 Close region may stuck when region is compacting and skip… #4536

HBASE-25709 Close region may stuck when region is compacting and skip… #4536

Conversation

sunhelly commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

Apache-HBase commented Jun 15, 2022

sunhelly commented Aug 24, 2022

virajjasani commented Aug 26, 2022 • edited Loading

sunhelly commented Aug 29, 2022

sunhelly commented Jan 29, 2023

virajjasani commented Feb 3, 2023

sunhelly commented Feb 16, 2023

Apache-HBase commented Feb 16, 2023

Apache-HBase commented Feb 16, 2023

Apache-HBase commented Feb 16, 2023

bbeaudreault Feb 16, 2023

Choose a reason for hiding this comment

sunhelly Feb 27, 2023

Choose a reason for hiding this comment

sunhelly Feb 27, 2023

Choose a reason for hiding this comment

bbeaudreault Feb 27, 2023

Choose a reason for hiding this comment

bbeaudreault Feb 27, 2023

Choose a reason for hiding this comment

sunhelly Feb 28, 2023

Choose a reason for hiding this comment

bbeaudreault left a comment

Choose a reason for hiding this comment

sunhelly commented Feb 27, 2023

Apache-HBase commented Feb 27, 2023

Apache-HBase commented Feb 27, 2023

Apache-HBase commented Feb 27, 2023

Apache-HBase commented Mar 2, 2023

Apache-HBase commented Mar 2, 2023

Apache-HBase commented Mar 2, 2023

virajjasani commented Aug 26, 2022 •

edited

Loading