HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3… #3346

bharathv · 2021-06-01T06:04:15Z

…332)

This bug was exposed by the test from HBASE-25924. Since this wal
implementations close the wal asynchronously, replication can potentially
miss the trailer bytes. (see jira comment for detailed analysis).

While this is not a correctness problem (since trailer does not have any entry data),
it erroneously bumps a metric that is used to track skipped bytes in WAL resulting
in false alarms which is something we should avoid.

Reviewed-by: Rushabh Shah [email protected]
Signed-off-by: Viraj Jasani [email protected]
Signed-off-by Anoop Sam John [email protected]
(cherry picked from commit b04c3c7)

…ache#3332) This bug was exposed by the test from HBASE-25924. Since this wal implementations close the wal asynchronously, replication can potentially miss the trailer bytes. (see jira comment for detailed analysis). While this is not a correctness problem (since trailer does not have any entry data), it erroneously bumps a metric that is used to track skipped bytes in WAL resulting in false alarms which is something we should avoid. Reviewed-by: Rushabh Shah <[email protected]> Signed-off-by: Viraj Jasani <[email protected]> Signed-off-by Anoop Sam John <[email protected]> (cherry picked from commit b04c3c7)

Apache-HBase · 2021-06-01T06:55:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 26s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	4m 9s	branch-2.4 passed
+1 💚	compile	3m 31s	branch-2.4 passed
+1 💚	checkstyle	1m 15s	branch-2.4 passed
+1 💚	spotbugs	2m 18s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 44s	the patch passed
+1 💚	compile	3m 22s	the patch passed
+1 💚	javac	3m 22s	the patch passed
-0 ⚠️	checkstyle	1m 12s	hbase-server: The patch generated 2 new + 15 unchanged - 0 fixed = 17 total (was 15)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 8s	Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.
+1 💚	spotbugs	2m 21s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		50m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3346
JIRA Issue	HBASE-25932
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 61de2104f512 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `1972261`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	86 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-06-01T08:51:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 8s	Docker mode activated.
-0 ⚠️	yetus	0m 6s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	4m 23s	branch-2.4 passed
+1 💚	compile	1m 6s	branch-2.4 passed
+1 💚	shadedjars	6m 51s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 58s	the patch passed
+1 💚	compile	1m 6s	the patch passed
+1 💚	javac	1m 6s	the patch passed
+1 💚	shadedjars	6m 48s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	the patch passed
		_ Other Tests _
+1 💚	unit	137m 32s	hbase-server in the patch passed.
		166m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3346
JIRA Issue	HBASE-25932
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1be39224e3c8 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `1972261`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/testReport/
Max. process+thread count	4132 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-06-01T09:58:48Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 14s	Docker mode activated.
-0 ⚠️	yetus	0m 7s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ branch-2.4 Compile Tests _
+1 💚	mvninstall	4m 2s	branch-2.4 passed
+1 💚	compile	0m 59s	branch-2.4 passed
+1 💚	shadedjars	6m 32s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	branch-2.4 passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 41s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	6m 33s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	the patch passed
		_ Other Tests _
+1 💚	unit	206m 21s	hbase-server in the patch passed.
		233m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#3346
JIRA Issue	HBASE-25932
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux cc2c7b2bb199 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.4 / `1972261`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/testReport/
Max. process+thread count	2530 (vs. ulimit of 12500)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3346/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

bharathv merged commit 8a41cb7 into apache:branch-2.4 Jun 1, 2021

bharathv deleted the HBASE-25932-branch-2.4 branch June 1, 2021 15:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3… #3346

HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3… #3346

bharathv commented Jun 1, 2021

Apache-HBase commented Jun 1, 2021

Apache-HBase commented Jun 1, 2021

Apache-HBase commented Jun 1, 2021

HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3… #3346

HBASE-25932: Ensure replication reads the trailer bytes from WAL. (#3… #3346

Conversation

bharathv commented Jun 1, 2021

Apache-HBase commented Jun 1, 2021

Apache-HBase commented Jun 1, 2021

Apache-HBase commented Jun 1, 2021