HBASE-27543 May be miss data when use mob #4935

WenFeiYi · 2022-12-21T12:47:47Z

No description provided.

Apache-HBase · 2022-12-21T13:22:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 8s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	2m 51s	master passed
+1 💚	compile	2m 25s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 44s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 30s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 29s	the patch passed
+1 💚	compile	2m 27s	the patch passed
+1 💚	javac	2m 27s	the patch passed
+1 💚	checkstyle	0m 35s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	8m 52s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 34s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 11s	The patch does not generate ASF License warnings.
		31m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 34afa6986f09 5.4.0-131-generic #147-Ubuntu SMP Fri Oct 14 17:07:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `222ec68`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-12-21T17:00:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 59s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 40s	master passed
+1 💚	compile	1m 4s	master passed
+1 💚	shadedjars	5m 30s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 34s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 8s	the patch passed
+1 💚	compile	0m 49s	the patch passed
+1 💚	javac	0m 49s	the patch passed
+1 💚	shadedjars	5m 32s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	the patch passed
		_ Other Tests _
-1 ❌	unit	222m 50s	hbase-server in the patch failed.
		250m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux dfa8ab5ace91 5.4.0-1085-aws #92~18.04.1-Ubuntu SMP Wed Aug 31 17:21:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `222ec68`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/testReport/
Max. process+thread count	2433 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell

Change seems fine.
Is it possible to add a unit test?

Apache9 · 2023-01-18T09:59:14Z

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/Compactor.java

        tmp == null ? HConstants.LATEST_TIMESTAMP : TimeRangeTracker.parseFrom(tmp).getMax();
+      fd.latestPutTs = Math.max(fd.latestPutTs, latestPutTs);


Check the code, we do not assign any value to fd.latestPutTs, so what is the actual problem here? The latestPutTs info for bulk loaded files can not negative and cause problem?

Apache-HBase · 2023-01-30T04:38:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 57s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 25s	master passed
+1 💚	compile	2m 51s	master passed
+1 💚	checkstyle	0m 38s	master passed
+1 💚	spotless	0m 48s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 46s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 44s	the patch passed
+1 💚	compile	2m 54s	the patch passed
+1 💚	javac	2m 54s	the patch passed
-0 ⚠️	checkstyle	0m 37s	hbase-server: The patch generated 1 new + 7 unchanged - 1 fixed = 8 total (was 8)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 20s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚	spotless	0m 52s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 3s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		53m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux fc05fb47f83c 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `382681e`
Default Java	Eclipse Adoptium-11.0.17+8
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	82 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-30T07:40:11Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 54s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 39s	master passed
+1 💚	compile	0m 48s	master passed
+1 💚	shadedjars	4m 26s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 17s	the patch passed
+1 💚	compile	0m 48s	the patch passed
+1 💚	javac	0m 48s	the patch passed
+1 💚	shadedjars	4m 27s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	211m 21s	hbase-server in the patch passed.
		234m 52s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 153f71c1335e 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `382681e`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/testReport/
Max. process+thread count	2488 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-30T07:40:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 1s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 38s	master passed
+1 💚	compile	0m 49s	master passed
+1 💚	shadedjars	5m 20s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 33s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 12s	the patch passed
+1 💚	compile	0m 55s	the patch passed
+1 💚	javac	0m 55s	the patch passed
+1 💚	shadedjars	5m 25s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 28s	the patch passed
		_ Other Tests _
+1 💚	unit	207m 54s	hbase-server in the patch passed.
		235m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a1c829675bbe 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `382681e`
Default Java	Temurin-1.8.0_352-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/testReport/
Max. process+thread count	2641 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-30T09:38:23Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 53s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 7s	master passed
+1 💚	compile	1m 0s	master passed
+1 💚	shadedjars	4m 52s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 27s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 52s	the patch passed
+1 💚	compile	0m 54s	the patch passed
+1 💚	javac	0m 54s	the patch passed
+1 💚	shadedjars	5m 8s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 30s	the patch passed
		_ Other Tests _
-1 ❌	unit	14m 16s	hbase-server in the patch failed.
		37m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a30499c26dff 5.4.0-137-generic #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `382681e`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/testReport/
Max. process+thread count	590 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-30T09:39:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 50s	master passed
+1 💚	compile	2m 27s	master passed
+1 💚	checkstyle	0m 34s	master passed
+1 💚	spotless	0m 41s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 12s	the patch passed
+1 💚	compile	2m 24s	the patch passed
+1 💚	javac	2m 24s	the patch passed
-0 ⚠️	checkstyle	0m 32s	hbase-server: The patch generated 1 new + 7 unchanged - 1 fixed = 8 total (was 8)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 52s	Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
-1 ❌	spotless	0m 34s	patch has 63 errors when running spotless:check, run spotless:apply to fix.
+1 💚	spotbugs	1m 29s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 9s	The patch does not generate ASF License warnings.
		38m 57s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 29109ac13629 5.4.0-1093-aws #102~18.04.2-Ubuntu SMP Wed Dec 7 00:31:59 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `382681e`
Default Java	Eclipse Adoptium-11.0.17+8
checkstyle	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	81 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-01-30T09:39:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 36s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 39s	master passed
+1 💚	compile	0m 52s	master passed
+1 💚	shadedjars	5m 4s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 0s	the patch passed
+1 💚	compile	0m 52s	the patch passed
+1 💚	javac	0m 52s	the patch passed
+1 💚	shadedjars	5m 30s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 25s	the patch passed
		_ Other Tests _
-1 ❌	unit	13m 55s	hbase-server in the patch failed.
		39m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4935
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 616343ce17ae 5.4.0-135-generic #152-Ubuntu SMP Wed Nov 23 20:19:22 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `382681e`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/testReport/
Max. process+thread count	935 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4935/3/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

apurtell approved these changes Jan 17, 2023

View reviewed changes

Apache9 requested changes Jan 18, 2023

View reviewed changes

WenFeiYi force-pushed the master branch from eae0c46 to 1666d58 Compare January 30, 2023 03:34

WenFeiYi force-pushed the master branch from 1666d58 to b369668 Compare January 30, 2023 08:46

HBASE-27543 May be miss data when use mob

5038990

WenFeiYi force-pushed the master branch from b369668 to 5038990 Compare January 30, 2023 08:51

apurtell self-requested a review March 28, 2023 04:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27543 May be miss data when use mob #4935

HBASE-27543 May be miss data when use mob #4935

WenFeiYi commented Dec 21, 2022

Apache-HBase commented Dec 21, 2022

Apache-HBase commented Dec 21, 2022

apurtell left a comment

Apache9 Jan 18, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

		tmp == null ? HConstants.LATEST_TIMESTAMP : TimeRangeTracker.parseFrom(tmp).getMax();
		fd.latestPutTs = Math.max(fd.latestPutTs, latestPutTs);

HBASE-27543 May be miss data when use mob #4935

Are you sure you want to change the base?

HBASE-27543 May be miss data when use mob #4935

Conversation

WenFeiYi commented Dec 21, 2022

Apache-HBase commented Dec 21, 2022

Apache-HBase commented Dec 21, 2022

apurtell left a comment

Choose a reason for hiding this comment

Apache9 Jan 18, 2023

Choose a reason for hiding this comment

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023

Apache-HBase commented Jan 30, 2023