Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

jobs: backportable forms of protected_{record_count,age_sec} [22.2, 22.1] #98532

Closed
irfansharif opened this issue Mar 13, 2023 · 1 comment · Fixed by #98540
Closed

jobs: backportable forms of protected_{record_count,age_sec} [22.2, 22.1] #98532

irfansharif opened this issue Mar 13, 2023 · 1 comment · Fixed by #98540
Assignees
@irfansharif
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@irfansharif
Copy link
Contributor

irfansharif commented Mar 13, 2023 •
edited by cockroach-jira-scripts
Loading

Is your feature request related to a problem? Please describe.

a0d6c19 introduced per-job metrics for managed protected timestamps, surfacing the total number of managed timestamps and oldest age across all of them. For 23.1+ clusters we can use it to sanity check whether PTS records are why we're not GC-ing MVCC garbage -- it's quite opaque otherwise. This issue tracks exposing similar metrics for 22.2 and 22.1 to help in internal cases like https://github.com/cockroachlabs/support/issues/2161.

Jira issue: CRDB-25311
@irfansharif irfansharif added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Mar 13, 2023
irfansharif added a commit to irfansharif/cockroach that referenced this issue Mar 13, 2023
@irfansharif
spanconfig: export metrics for protected timestamp records
492e166 
This commit introduces two new metrics, to help understand the effects
of protected timestamps:
- spanconfig.kvsubscriber.protected_record_count, which exports the
  number of protected timestamp records as seen by KV.
- spanconfig.kvsubscriber.oldest_protected_record_nanos, which exports
  difference between the current time and the oldest protected
  timestamp. Sudden drops indicate a record being released; an
  ever-increasing duration would indicate the oldest record sticking
  around and preventing GC if > the configured GC TTL.

Fixes cockroachdb#98532 (as a backportable alternative to
a0d6c19 for 22.1, 22.2).

Release note: None
@irfansharif irfansharif mentioned this issue Mar 13, 2023
Merged
craig bot pushed a commit that referenced this issue Mar 14, 2023
@Xiang-Gu @irfansharif
Merge #97820 #98540
a79338a 
97820: schemachanger: Enable ALTER PRIMARY KEY USING HASH r=Xiang-Gu a=Xiang-Gu

We enable ALTER PRIMARY KEY USING HASH in the declarative schema changer. This commit also cleaned up some common logic to be re-used with CREATE INDEX ... USING HASH.

Note that we still fallback to legacy schema changer if the old primary key is on the implicit `rowid` column, because we don't support `ADD COLUMN, DROP COLUMN` in one statement yet (adding a shard column and dropping the `rowid` column).

Fixes: #96730
Epic: None

Release note: None

98540: spanconfig: export metrics for protected timestamp records r=irfansharif a=irfansharif

This commit introduces two new metrics, to help understand the effects of protected timestamps:
- `spanconfig.kvsubscriber.protected_record_count`, which exports the number of protected timestamp records as seen by KV.
- `spanconfig.kvsubscriber.oldest_protected_record_nanos`, which exports difference between the current time and the oldest protected timestamp. Sudden drops indicate a record being released; an ever-increasing duration would indicate the oldest record sticking around and preventing GC if > the configured GC TTL.

Fixes #98532 (as a backportable alternative to a0d6c19 for 22.1, 22.2).

Release note: None

Co-authored-by: Xiang Gu <[email protected]>
Co-authored-by: irfan sharif <[email protected]>
@craig craig bot closed this as completed in 8be8d9d Mar 14, 2023
@irfansharif irfansharif self-assigned this Mar 14, 2023
@irfansharif irfansharif reopened this Mar 14, 2023
@irfansharif
Copy link
Contributor Author

irfansharif commented Mar 14, 2023 •
edited
Loading

(Keeping it open until I open backports.)

@irfansharif irfansharif mentioned this issue Mar 16, 2023
Merged
@irfansharif irfansharif mentioned this issue Mar 16, 2023
Merged
irfansharif added a commit to irfansharif/cockroach that referenced this issue Mar 16, 2023
@irfansharif
spanconfig: export metrics for protected timestamp records
480c6c1 
This commit introduces two new metrics, to help understand the effects
of protected timestamps:
- spanconfig.kvsubscriber.protected_record_count, which exports the
  number of protected timestamp records as seen by KV.
- spanconfig.kvsubscriber.oldest_protected_record_nanos, which exports
  difference between the current time and the oldest protected
  timestamp. Sudden drops indicate a record being released; an
  ever-increasing duration would indicate the oldest record sticking
  around and preventing GC if > the configured GC TTL.

Fixes cockroachdb#98532 (as a backportable alternative to
a0d6c19 for 22.1, 22.2).

Release note: None
irfansharif added a commit to irfansharif/cockroach that referenced this issue Mar 18, 2023
@irfansharif
spanconfig: export metrics for protected timestamp records
9905620 
This commit introduces two new metrics, to help understand the effects
of protected timestamps:
- spanconfig.kvsubscriber.protected_record_count, which exports the
  number of protected timestamp records as seen by KV.
- spanconfig.kvsubscriber.oldest_protected_record_nanos, which exports
  difference between the current time and the oldest protected
  timestamp. Sudden drops indicate a record being released; an
  ever-increasing duration would indicate the oldest record sticking
  around and preventing GC if > the configured GC TTL.

Fixes cockroachdb#98532 (as a backportable alternative to
a0d6c19 for 22.1, 22.2).

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@irfansharif irfansharif

Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

spanconfig: export metrics for protected timestamp records irfansharif/cockroach
1 participant
@irfansharif