Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HBASE-27762 Include EventType and ProcedureV2 pid in logging via MDC #5145

Merged
merged 4 commits into from
Apr 17, 2023

Conversation

ndimiduk
Copy link
Member

No description provided.

@ndimiduk
Copy link
Member Author

ndimiduk commented Mar 27, 2023

With this change, on the region server, we get log messages like,

2023-03-27T15:46:01,930 INFO  [RS_CLAIM_REPLICATION_QUEUE-regionserver/scipio:0-0 ({event_type=RS_CLAIM_REPLICATION_QUEUE, pid=33})] replication.ZKReplicationQueueStorage(391): Atomically moving scipio,51464,1679924724756/region_replica_replication's WALs to scipio,51466,1679924724781

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@ndimiduk ndimiduk force-pushed the 27762-eventtype-pid-mdc branch from 8534d42 to e065f93 Compare March 29, 2023 12:01
@Apache-HBase

This comment was marked as outdated.

@@ -96,6 +97,10 @@ public EventHandler prepare() throws Exception {
public void run() {
Span span = TraceUtil.getGlobalTracer().spanBuilder(getClass().getSimpleName())
.setParent(Context.current().with(parent)).startSpan();
// assume that this is the top of an execution on a new or reused thread, that we're safe to
// blast any existing MDC state.
MDC.clear();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only call clear at the beginning? Is it possible to clear MDC at the end of a execution?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, seems no harm in that. You'd like to see it clear on both beginning and end?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer we do something like

MDC.put("blabla", "blabla");
try {
  ...
} finally {
  MDC.clear();
}

@Apache-HBase

This comment was marked as outdated.

@Apache-HBase

This comment was marked as outdated.

@ndimiduk ndimiduk force-pushed the 27762-eventtype-pid-mdc branch from e065f93 to ce60ae8 Compare March 30, 2023 11:24
@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 3s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ master Compile Tests _
+0 🆗 mvndep 0m 16s Maven dependency ordering for branch
+1 💚 mvninstall 3m 25s master passed
+1 💚 compile 4m 39s master passed
+1 💚 checkstyle 1m 8s master passed
+1 💚 spotless 0m 42s branch has no errors when running spotless:check.
-1 ❌ spotbugs 7m 6s root in master has 4 extant spotbugs warnings.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 11s Maven dependency ordering for patch
+1 💚 mvninstall 3m 17s the patch passed
+1 💚 compile 4m 45s the patch passed
+1 💚 javac 4m 45s the patch passed
+1 💚 checkstyle 1m 6s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 12m 34s Patch does not cause any errors with Hadoop 3.2.4 3.3.4.
+1 💚 spotless 0m 41s patch has no errors when running spotless:check.
+1 💚 spotbugs 9m 8s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 31s The patch does not generate ASF License warnings.
60m 5s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #5145
Optional Tests dupname asflicense spotless javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux f9ea9b4f86ba 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e5620e2
Default Java Eclipse Adoptium-11.0.17+8
spotbugs https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/artifact/yetus-general-check/output/branch-spotbugs-root-warnings.html
Max. process+thread count 177 (vs. ulimit of 30000)
modules C: hbase-logging hbase-server . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/console
versions git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 19s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 12s Maven dependency ordering for branch
+1 💚 mvninstall 3m 15s master passed
+1 💚 compile 1m 53s master passed
+1 💚 shadedjars 4m 22s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 2m 26s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 3m 21s the patch passed
+1 💚 compile 1m 51s the patch passed
+1 💚 javac 1m 51s the patch passed
+1 💚 shadedjars 4m 21s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 2m 25s the patch passed
_ Other Tests _
+1 💚 unit 270m 38s root in the patch passed.
301m 21s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #5145
Optional Tests unit javac javadoc shadedjars compile
uname Linux a90d70587342 5.4.0-144-generic #161-Ubuntu SMP Fri Feb 3 14:49:04 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e5620e2
Default Java Eclipse Adoptium-11.0.17+8
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/testReport/
Max. process+thread count 4618 (vs. ulimit of 30000)
modules C: hbase-logging hbase-server . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 24s Docker mode activated.
-0 ⚠️ yetus 0m 3s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ master Compile Tests _
+0 🆗 mvndep 0m 11s Maven dependency ordering for branch
+1 💚 mvninstall 3m 1s master passed
+1 💚 compile 1m 31s master passed
+1 💚 shadedjars 4m 34s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 45s master passed
_ Patch Compile Tests _
+0 🆗 mvndep 0m 10s Maven dependency ordering for patch
+1 💚 mvninstall 2m 41s the patch passed
+1 💚 compile 1m 27s the patch passed
+1 💚 javac 1m 27s the patch passed
+1 💚 shadedjars 4m 29s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 1m 43s the patch passed
_ Other Tests _
-1 ❌ unit 323m 0s root in the patch failed.
349m 57s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR #5145
Optional Tests unit javac javadoc shadedjars compile
uname Linux ef16b9b96e30 5.4.0-1094-aws #102~18.04.1-Ubuntu SMP Tue Jan 10 21:07:03 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision master / e5620e2
Default Java Temurin-1.8.0_352-b08
unit https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-root.txt
Test Results https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/testReport/
Max. process+thread count 2244 (vs. ulimit of 30000)
modules C: hbase-logging hbase-server . U: .
Console output https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5145/3/console
versions git=2.34.1 maven=3.8.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@ndimiduk
Copy link
Member Author

Test failure is https://issues.apache.org/jira/browse/HBASE-27720 .

@ndimiduk ndimiduk merged commit 63471ef into apache:master Apr 17, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 17, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 28, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 28, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 28, 2023
ndimiduk added a commit to ndimiduk/hbase that referenced this pull request Apr 28, 2023
ndimiduk added a commit that referenced this pull request May 1, 2023
ndimiduk added a commit that referenced this pull request May 1, 2023
ndimiduk added a commit that referenced this pull request May 1, 2023
vinayakphegde pushed a commit to vinayakphegde/hbase that referenced this pull request Apr 4, 2024
…pache#5145)

Signed-off-by: Duo Zhang <[email protected]>
(cherry picked from commit 1d3d900)
Change-Id: Id2271f761dc7e858068d42f46e55a40eda4c5e36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants