-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-21.2: sql: new SQL Stats cluster settings to improve write traffic #78966
Conversation
Thanks for opening a backport. Please check the backport criteria before merging:
If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.
Add a brief release justification to the body of your PR to justify this backport. Some other things to consider:
|
7af5808
to
50d4ba8
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @Azhng and @maryliag)
Resolves #78339 Previously, SQL Stats flushed to system table as soon as the in-memory buffer is full. This means the size of the system tables that back SQL Stats could grow faster than the cleanup job. Additionally, when the SQL Stats flush is disabled, the SQL Stats is unable to collect any more new statement / transaction statistics when the in-memory store is full. This commit introduces two non-public cluster settings: * `sql.stats.flush.minimum_interval`: this setting limits minimum interval between each flush operation. If a flush operation is triggered sooner than what is allowed by the minimum interval, (e.g. when the in-memory SQL Stats store is full), the flush operation is aborted. By default this cluster setting is set to 0. * `sql.stats.flush.force_cleanup.enabled`: which allows the in-memory SQL Stats to be cleared at the interval specified by `sql.stats.flush.interval`, even if the SQL Stats flush is disabled. By default, this cluster setting is set to false. This commit also updated the stmt_grouping_in_explicit_txn data driven test to ensure the output order is deterministic. Release note: None
50d4ba8
to
e7b0f3a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 1 of 0 LGTMs obtained (and 1 stale) (waiting on @Azhng)
Backport 1/1 commits from #78446 on behalf of @Azhng.
/cc @cockroachdb/release
Resolves #78339
Previously, SQL Stats flushed to system table as soon as the in-memory
buffer is full. This means the size of the system tables that back SQL
Stats could grow faster than the cleanup job. Additionally, when the
SQL Stats flush is disabled, the SQL Stats is unable to collect any more
new statement / transaction statistics when the in-memory store is full.
This commit introduces two non-public cluster settings:
sql.stats.flush.minimum_interval
: this setting limits minimum intervalbetween each flush operation. If a flush operation is triggered sooner
than what is allowed by the minimum interval, (e.g. when the in-memory
SQL Stats store is full), the flush operation is aborted.
sql.stats.flush.force_cleanup.enabled
: which allows thein-memory SQL Stats to be cleared at the interval specified by
sql.stats.flush.interval
, even if the SQL Stats flush is disabled.This commit also updated the stmt_grouping_in_explicit_txn data driven
test to ensure the output order is deterministic.
Release note: None
Release justification: low risk high benefit change