Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-23205 Correctly update the position of WALs currently being replicated (2) #944

Merged
merged 1 commit into from
Jan 2, 2020

Conversation

JeongDaeKim
Copy link

https://issues.apache.org/jira/browse/HBASE-23205

#749 (comment)

No luck with the pre-commit. I tried a rebase, but still the job fails while starting. Ain't sure if it's something specific to the PR commits. @JeongDaeKim , would u mind squash these commits on one of you local branches, then open a new PR for branch-1 with this? Please ping me once you open the new PR.

@JeongDaeKim
Copy link
Author

@wchevreuil I squashed my commits and made a new PR

@JeongDaeKim JeongDaeKim force-pushed the HBASE-23205-2 branch 2 times, most recently from 032b63f to 35f0a8d Compare December 17, 2019 02:56
@JeongDaeKim
Copy link
Author

@wchevreuil It seems pre-commit hook was not triggered again. I have no idea what conditions should be required to trigger it.

@wchevreuil
Copy link
Contributor

Thanks @JeongDaeKim , it's actually crashing at the beginning due to same error from previous PR: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/3/console. Let me try figure why this happens.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+1 💚 mvninstall 8m 20s branch-1 passed
+1 💚 compile 0m 41s branch-1 passed with JDK v1.8.0_232
+1 💚 compile 0m 46s branch-1 passed with JDK v1.7.0_242
+1 💚 checkstyle 1m 48s branch-1 passed
+1 💚 shadedjars 3m 9s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 36s branch-1 passed with JDK v1.8.0_232
+1 💚 javadoc 0m 43s branch-1 passed with JDK v1.7.0_242
+0 🆗 spotbugs 3m 1s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 58s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 4s the patch passed
+1 💚 compile 0m 41s the patch passed with JDK v1.8.0_232
+1 💚 javac 0m 41s the patch passed
+1 💚 compile 0m 46s the patch passed with JDK v1.7.0_242
+1 💚 javac 0m 46s the patch passed
+1 💚 checkstyle 1m 44s hbase-server: The patch generated 0 new + 24 unchanged - 12 fixed = 24 total (was 36)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 2s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 5m 9s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 32s the patch passed with JDK v1.8.0_232
+1 💚 javadoc 0m 42s the patch passed with JDK v1.7.0_242
+1 💚 findbugs 3m 4s the patch passed
_ Other Tests _
-1 ❌ unit 45m 42s hbase-server in the patch failed.
+1 💚 asflicense 0m 20s The patch does not generate ASF License warnings.
87m 1s
Reason Tests
Failed junit tests hadoop.hbase.client.TestClientClusterStatus
hadoop.hbase.procedure.TestProcedureManager
hadoop.hbase.client.TestRollbackFromClient
hadoop.hbase.regionserver.TestHRegionFileSystem
hadoop.hbase.master.balancer.TestRegionLocationFinder
Subsystem Report/Notes
Docker Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/4/artifact/out/Dockerfile
GITHUB PR #944
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux 03329d85f4b5 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-944/out/precommit/personality/provided.sh
git revision branch-1 / 1451063
Default Java 1.7.0_242
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/4/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/4/testReport/
Max. process+thread count 605 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/4/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

@JeongDaeKim
Copy link
Author

Rebased and force-pushed twice to trigger pre-commit hook, and somehow it worked finally.

I checked failed tests
all failures are same as below and it seems it come from HBASE-23590 #950 (all tests passed after i removed the commit) @virajjasani ?

2020-01-02 19:29:11,915 ERROR [RpcServer.default.FPBQ.Fifo.handler=4,queue=0,port=52812] master.MasterRpcServices(403): Region server 10.70.8.163,52826,1577960947462 reported a fatal error:
ABORTING region server 10.70.8.163,52826,1577960947462: Unhandled: null
Cause:
java.lang.NullPointerException
    at org.apache.hadoop.hbase.regionserver.HStore.getMaxCompactedStoreFileRefCount(HStore.java:2881)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.createRegionLoad(HRegionServer.java:1601)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:1312)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1249)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1052)
    at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:159)
    at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:112)
    at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:143)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:360)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1824)
    at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:334)
    at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:141)
    at java.lang.Thread.run(Thread.java:748)

another build is running now because of my two pushes (https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/5/),and it will produce the same result as the previous build 4.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 33s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-1 Compile Tests _
+1 💚 mvninstall 8m 31s branch-1 passed
+1 💚 compile 0m 40s branch-1 passed with JDK v1.8.0_232
+1 💚 compile 0m 46s branch-1 passed with JDK v1.7.0_242
+1 💚 checkstyle 1m 50s branch-1 passed
+1 💚 shadedjars 3m 11s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 36s branch-1 passed with JDK v1.8.0_232
+1 💚 javadoc 0m 41s branch-1 passed with JDK v1.7.0_242
+0 🆗 spotbugs 3m 0s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 58s branch-1 passed
_ Patch Compile Tests _
+1 💚 mvninstall 2m 4s the patch passed
+1 💚 compile 0m 41s the patch passed with JDK v1.8.0_232
+1 💚 javac 0m 41s the patch passed
+1 💚 compile 0m 46s the patch passed with JDK v1.7.0_242
+1 💚 javac 0m 46s the patch passed
+1 💚 checkstyle 1m 44s hbase-server: The patch generated 0 new + 24 unchanged - 12 fixed = 24 total (was 36)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedjars 3m 3s patch has no errors when building our shaded downstream artifacts.
+1 💚 hadoopcheck 5m 10s Patch does not cause any errors with Hadoop 2.8.5 2.9.2.
+1 💚 javadoc 0m 31s the patch passed with JDK v1.8.0_232
+1 💚 javadoc 0m 42s the patch passed with JDK v1.7.0_242
+1 💚 findbugs 3m 5s the patch passed
_ Other Tests _
-1 ❌ unit 45m 38s hbase-server in the patch failed.
+1 💚 asflicense 0m 20s The patch does not generate ASF License warnings.
87m 11s
Reason Tests
Failed junit tests hadoop.hbase.client.TestClientClusterStatus
hadoop.hbase.procedure.TestProcedureManager
hadoop.hbase.client.TestRollbackFromClient
hadoop.hbase.regionserver.TestHRegionFileSystem
hadoop.hbase.master.balancer.TestRegionLocationFinder
Subsystem Report/Notes
Docker Client=19.03.4 Server=19.03.4 base: https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/5/artifact/out/Dockerfile
GITHUB PR #944
Optional Tests dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile
uname Linux df45beee7258 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality /home/jenkins/jenkins-slave/workspace/HBase-PreCommit-GitHub-PR_PR-944/out/precommit/personality/provided.sh
git revision branch-1 / 1451063
Default Java 1.7.0_242
Multi-JDK versions /usr/lib/jvm/zulu-8-amd64:1.8.0_232 /usr/lib/jvm/zulu-7-amd64:1.7.0_242
unit https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/5/artifact/out/patch-unit-hbase-server.txt
Test Results https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/5/testReport/
Max. process+thread count 605 (vs. ulimit of 10000)
modules C: hbase-server U: hbase-server
Console output https://builds.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-944/5/console
versions git=1.9.1 maven=3.0.5 findbugs=3.0.1
Powered by Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

Copy link
Contributor

@wchevreuil wchevreuil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests failures seems unrelated. +1.

@wchevreuil wchevreuil merged commit e0c419d into apache:branch-1 Jan 2, 2020
@virajjasani
Copy link
Contributor

virajjasani commented Jan 2, 2020

Thanks @JeongDaeKim for pointing out test failures, these happen on branch-1 since we have null initialization for list of store files as opposed to empty lists in master and branch-2.
Will take care of this in a while, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants