rpc: log RPC heartbeat errors at error level #93388
Conversation
|
This change is |
Wait what? the HEALTH events are saved at severity level INFO by default. This should not be needed. |
|
RPC heartbeat failures due to e.g. network latencies can cause clusters to completely fall apart. Currently, there is no clear indication of this happening at all in the default |
|
I am not objecting to your change, but the PR description as-is is factually incorrect. Try this: "Currently heartbeat failures are logged at level INFO. This makes it hard to identify them as anomalies when scanning the log files. This commit changes the level to ERROR to make them stand out during troubleshooting." |
|
I see, I'll reword the message. Isn't it factually correct though, in that health INFO events aren't logged to |
Currently heartbeat failures are logged at level INFO. This makes it hard to identify them as anomalies when scanning the log files. This commit changes the level to ERROR to make them stand out during troubleshooting. Epic: none Release note: None
erikgrinaker
force-pushed
the
rpc-log-error
branch
from
d9735aa to
e95ed64
Compare
|
You can move your commit message under "Release note (cli change):" since we document changes to non-DEV logging output. |
|
The original change to INFO was done recently in #88625 (which didn't have a release note for this), so this PR essentially reverts to the 22.2 behavior. |
|
bors r+ |
|
Build failed: |
|
bors r+ |
|
Build failed (retrying...): |
|
Build failed (retrying...): |
|
Build failed (retrying...): |
|
Build failed: |
|
bors retry |
|
Build failed (retrying...): |
|
Build succeeded: |
Currently heartbeat failures are logged at level INFO. This makes it hard to identify them as anomalies when scanning the log files. This commit changes the level to ERROR to make them stand out during troubleshooting.
Epic: none
Release note: None