Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cdc: show all changefeed jobs in SHOW CHANGEFEED JOBS #98175

Merged
merged 2 commits into from
Mar 14, 2023

Conversation

jayshrivastava
Copy link
Contributor

@jayshrivastava jayshrivastava commented Mar 7, 2023

cdc: show all changefeed jobs in SHOW CHANGEFEED JOBS

Release note (general change): Previously, the output of SHOW CHANGEFEED JOBS was limited to show unfinished jobs and finished jobs from the last 14 days. This change makes the command show all changefeed jobs, regardless of if they finished and when they finished. Note that jobs still obey the cluster setting jobs.retention_time. Completed jobs older than that time are deleted.

Fixes: #97883

jobs: add virtual index for job_type in crdb_internal.jobs

This change adds a virtual index on the job_type column
of crdb_internal.jobs. This change should make queries
on that table which filter on job type (such as SHOW CHANGEFEED JOBS) more efficient.

Release note: None

Epic: None

@blathers-crl
Copy link

blathers-crl bot commented Mar 7, 2023

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@jayshrivastava
Copy link
Contributor Author

There doesn't seem to be a reason for why we limit it to 12h. There is a mention of it in the original PR #64956 (review). Perhaps this limit was added due to instability in the jobs system / virtual tables. There's no discussion on Slack which I could find either.

@jayshrivastava jayshrivastava marked this pull request as ready for review March 9, 2023 15:41
@jayshrivastava jayshrivastava requested a review from a team as a code owner March 9, 2023 15:41
@jayshrivastava jayshrivastava requested a review from a team March 13, 2023 20:30
@jayshrivastava jayshrivastava requested a review from a team as a code owner March 13, 2023 20:30
Copy link
Contributor

@HonoreDB HonoreDB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @DrewKimball)

@jayshrivastava
Copy link
Contributor Author

The virtual index causes a lot of tests to fail in many different ways, including but not limited to mixed version tests. I'm going to hold off on merging for now.

@jayshrivastava jayshrivastava requested a review from a team March 14, 2023 00:20
@jayshrivastava jayshrivastava requested a review from a team as a code owner March 14, 2023 00:20
@jayshrivastava jayshrivastava force-pushed the show-changefeed-jobs branch 2 times, most recently from 67a69a3 to 5065385 Compare March 14, 2023 14:19
Release note (general change): Previously, the output of `SHOW CHANGEFEED JOBS`
was limited to show unfinished jobs and finished jobs from the last 14 days.
This change makes the command show all changefeed jobs, regardless of if they finished
and when they finished. Note that jobs still obey the cluster setting `jobs.retention_time`.
Completed jobs older than that time are deleted.

Fixes: cockroachdb#97883
Epic: None
This change adds a virtual index on the `job_type` column
of `crdb_internal.jobs`. This change should make queries
on that table which filter on job type (such as `SHOW
CHANGEFEED JOBS`) more efficient.

Release note: None
@jayshrivastava
Copy link
Contributor Author

bors r=HonoreDB

@craig craig bot merged commit a36d88b into cockroachdb:master Mar 14, 2023
@craig
Copy link
Contributor

craig bot commented Mar 14, 2023

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cdc: SHOW CHANGEFEED JOBS only shows jobs from the last 12h
3 participants