Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

*: too many log messages - reduce frequency of logging #89075

Closed
23 of 24 tasks
knz opened this issue Sep 30, 2022 · 2 comments
Closed
23 of 24 tasks

*: too many log messages - reduce frequency of logging #89075

knz opened this issue Sep 30, 2022 · 2 comments
Labels
A-logging In and around the logging infrastructure. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.

Comments

@knz
Copy link
Contributor

knz commented Sep 30, 2022 •
edited
Loading

Describe the problem

We are producing too many log events, which makes log files unnecessary large, network log collection costs unnecessary expensive, and generally hinders our ability to operate.

The attached file is a breakdown of log events per source and per version.
logs.txt

Some of the top contenders:

Jira issue: CRDB-20104
@knz knz added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. A-logging In and around the logging infrastructure. labels Sep 30, 2022
This was referenced Sep 30, 2022
Closed
Merged
Merged
Merged
craig bot pushed a commit that referenced this issue Sep 30, 2022
@knz
Merge #89077
3a9f5b8 
89077: kvserver: don't log "local QPS is below max threshold" by default r=aliher1911 a=knz

Context: #89075

This log message is the 9th most voluminous across the CC fleet. It's not surprising since most stores are idle.
There's no good reason to have it logged by default - it doesn't say anything interesting.
Let's just keep it in traces.

Release note: None

Co-authored-by: Raphael 'kena' Poss <[email protected]>
@knz
Copy link
Contributor Author

knz commented Oct 5, 2022

I'm waiting for access to Splunk to compute how much % volume we're reducing by addressing the above.

@knz
Copy link
Contributor Author

knz commented Oct 5, 2022

Back of the envelope says it's more than 50%, which is good enough to close this issue.

@knz knz closed this as completed Oct 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-logging In and around the logging infrastructure. C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@knz