Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

release-23.2: cli/doctor, doctor: use right jobs table, skip dropped descs #124365

Merged
merged 2 commits into from
Aug 7, 2024

Conversation

annrpom
Copy link
Contributor

@annrpom annrpom commented May 17, 2024

Backport 2/2 commits from #123298.

/cc @cockroachdb/release

Release justification: non-production change to debugging tool


cli/doctor: doctor should read from the right jobs table

In #97762, we started writing a job's payload (and progress)
information to the system.jobs_info table. As a result, we
had to change the parts of our code that relied on the system.jobs
table to use crdb_internal.system_jobs instead (since that table
would inaccurately report that some payloads were NULL).
This change did not occur for the in-memory representation of the jobs
table created by debug doctor -- which can result in missing job
false-positives. This patch updates debug doctor's representation of
the jobs table by referring to crdb_internal.system_jobs instead.

Epic: none
Fixes: #122675

Release note: None


doctor: skip validation for dropped descriptors

In some cases, dropped descriptors appear in our system.descriptors
table with dangling job mutations without an associated job.
This patch teaches debug doctor examine to skip validation
on such dropped descriptors.

Epic: none

Fixes: #123477
Fixes: #122956

Release note: none

@annrpom annrpom requested review from a team as code owners May 17, 2024 20:28
@annrpom annrpom requested review from nameisbhaskar and vidit-bhat and removed request for a team May 17, 2024 20:28
Copy link

blathers-crl bot commented May 17, 2024

Thanks for opening a backport.

Please check the backport criteria before merging:

  • Backports should only be created for serious
    issues
    or test-only changes.
  • Backports should not break backwards-compatibility.
  • Backports should change as little code as possible.
  • Backports should not change on-disk formats or node communication protocols.
  • Backports should not add new functionality (except as defined
    here).
  • Backports must not add, edit, or otherwise modify cluster versions; or add version gates.
  • All backports must be reviewed by the owning areas TL and one additional
    TL. For more information as to how that review should be conducted, please consult the backport
    policy
    .
If your backport adds new functionality, please ensure that the following additional criteria are satisfied:
  • There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
  • The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
  • New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters. State changes must be further protected such that nodes running old binaries will not be negatively impacted by the new state (with a mixed version test added).
  • The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.
  • Your backport must be accompanied by a post to the appropriate Slack
    channel (#db-backports-point-releases or #db-backports-XX-X-release) for awareness and discussion.

Also, please add a brief release justification to the body of your PR to justify this
backport.

@blathers-crl blathers-crl bot added the backport Label PR's that are backports to older release branches label May 17, 2024
@cockroach-teamcity
Copy link
Member

This change is Reviewable

Copy link
Collaborator

@rafiss rafiss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @annrpom, @nameisbhaskar, and @vidit-bhat)


pkg/sql/doctor/doctor_test.go line 200 at r2 (raw file):

			},
			valid:    true,
			expected: "Examining 1 descriptors and 0 namespace entries...\n",

nit: just noticed this. why are we removing the expected error check?


pkg/sql/doctor/doctor_test.go line 419 at r2 (raw file):

			},
			valid:    true,
			expected: "Examining 2 descriptors and 1 namespace entries...\n",

nit: just noticed this. why are we removing the expected error check?

Copy link
Contributor Author

@annrpom annrpom left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolving nits since they were addressed #124302 (comment)

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nameisbhaskar and @vidit-bhat)

@annrpom annrpom requested a review from rafiss June 24, 2024 21:31
@annrpom annrpom force-pushed the backport23.2-123298 branch from 081b471 to 91664e0 Compare June 25, 2024 02:22
@annrpom annrpom requested a review from a team as a code owner June 25, 2024 02:22
@annrpom annrpom removed the request for review from a team June 25, 2024 02:22
annrpom added 2 commits June 25, 2024 09:30
In cockroachdb#97762, we started writing a job's payload (and progress)
information to the `system.jobs_info` table. As a result, we
had to change the parts of our code that relied on the `system.jobs`
table to use `crdb_internal.system_jobs` instead (since that table
would inaccurately report that some `payload`s were `NULL`).
This change did not occur for the in-memory representation of the jobs
table created by debug doctor -- which can result in missing job
false-positives. This patch updates debug doctor's representation of
the jobs table by referring to `crdb_internal.system_jobs` instead.

Epic: none
Fixes: cockroachdb#122675

Release note: None
In some cases, dropped descriptors appear in our `system.descriptors`
table with dangling job mutations without an associated job.
This patch teaches debug doctor examine to skip validation
on such dropped descriptors.

Epic: none

Fixes: cockroachdb#123477
Fixes: cockroachdb#122956

Release note: none
@annrpom annrpom force-pushed the backport23.2-123298 branch from 91664e0 to 4c50a21 Compare June 25, 2024 13:30
@rafiss rafiss merged commit 9cc06ba into cockroachdb:release-23.2 Aug 7, 2024
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Label PR's that are backports to older release branches
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants