Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql: lease acquisition of OFFLINE descs may starve bulk operations #62959

Merged
merged 1 commit into from
Apr 1, 2021

Conversation

fqazi
Copy link
Collaborator

@fqazi fqazi commented Apr 1, 2021

Fixes: #61798

Previously, offline descriptors would never have their leases
cached and they would be released once the reference count hit zero.
This was inadequate because when attempting to online these tables
again the lease acquisition could be pushed back by other operations,
leading to starvation / live locks. To address this, this patch will
allow the leases of offline descriptors to be cached.

Release note (bug fix): Lease acquisitions of descriptor in a
offline state may starve out bulk operations (backup / restore)

@fqazi fqazi requested review from a team and pbardea and removed request for a team April 1, 2021 14:17
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@fqazi fqazi requested review from ajwerner and a team April 1, 2021 14:17
@fqazi fqazi force-pushed the leaseStarvation branch from 2d8fc1e to 12fd474 Compare April 1, 2021 14:47
Copy link
Contributor

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 4 of 5 files at r1.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @fqazi and @pbardea)


pkg/sql/catalog/lease/lease.go, line 1418 at r1 (raw file):

				err2 := m.Release(desc)
				if err2 != nil {

nit: I'd prefere releaseErr to err2

@fqazi fqazi force-pushed the leaseStarvation branch from 12fd474 to 73d9ce8 Compare April 1, 2021 15:09
@fqazi
Copy link
Collaborator Author

fqazi commented Apr 1, 2021

bors r=ajwerner

@craig
Copy link
Contributor

craig bot commented Apr 1, 2021

Build failed:

@fqazi
Copy link
Collaborator Author

fqazi commented Apr 1, 2021

bors r-

Fixes: cockroachdb#61798

Previously, offline descriptors would never have their leases
cached and they would be released once the reference count hit zero.
This was inadequate because when attempting to online these tables
again the lease acquisition could be pushed back by other operations,
leading to starvation / live locks. To address this, this patch will
allow the leases of offline descriptors to be cached.

Release note (bug fix): Lease acquisitions of descriptor in a
offline state may starve out bulk operations (backup / restore)
@fqazi fqazi force-pushed the leaseStarvation branch from 73d9ce8 to 4b6518c Compare April 1, 2021 17:28
@fqazi
Copy link
Collaborator Author

fqazi commented Apr 1, 2021

bors r=ajwerner

@craig
Copy link
Contributor

craig bot commented Apr 1, 2021

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Apr 1, 2021

Build succeeded:

@knz
Copy link
Contributor

knz commented Apr 13, 2021

this one may need a backport on the 21.1 branch as well

@mikeCRL
Copy link
Contributor

mikeCRL commented Apr 28, 2021

I further summarized the release note text. Can anyone please +1 or edit this? @ajwerner @fqazi

Allow the leases of offline descriptors to be cached, preventing issues with lease acquisitions during bulk operations (backup and restore operations). [#63558][#63558]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

sql/catalog/lease: high priority reads in lease acquisition of OFFLINE descs may starve bulk operations
5 participants