Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kv: initialize consistencyLimiter during Store construction, before Start #100554

Merged
merged 1 commit into from
Apr 4, 2023

Conversation

nvanbenschoten
Copy link
Member

Fixes #96231.

This commit attempts to fix #96231. It moves the initialization of Store.consistencyLimiter up from the bottom of Store.Start into NewStore. It is unsafe to initialize this variable after the call to Store.processRaft, which starts Raft processing. Beyond that point, incoming Raft traffic is permitted and calls to computeChecksumPostApply are possible.

The two stacktraces we have in that issue conclusively point to the Store.consistencyLimiter being nil during a call to (*Replica).computeChecksumPostApply. This startup-time race is the only possible explanation I could come up with.

Release note (bug fix): Fixed a rare startup race that could cause an invalid memory address or nil pointer dereference error.

…tart

Fixes cockroachdb#96231.

This commit attempts to fix cockroachdb#96231. It moves the initialization of
`Store.consistencyLimiter` up from the bottom of `Store.Start` into
`NewStore`. It is unsafe to initialize this variable after the call to
`Store.processRaft`, which starts Raft processing. Beyond that point,
incoming Raft traffic is permitted and calls to `computeChecksumPostApply`
are possible.

The two stacktraces we have in that issue conclusively point to the
`Store.consistencyLimiter` being nil during a call to
`(*Replica).computeChecksumPostApply`. This startup-time race is the
only possible explanation I could come up with.

Release note (bug fix): Fixed a rare startup race that could cause a
`invalid memory address or nil pointer dereference` error.
@nvanbenschoten nvanbenschoten added backport-22.1.x backport-23.1.x Flags PRs that need to be backported to 23.1 labels Apr 4, 2023
@nvanbenschoten nvanbenschoten requested a review from a team as a code owner April 4, 2023 04:38
@blathers-crl
Copy link

blathers-crl bot commented Apr 4, 2023

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@nvanbenschoten
Copy link
Member Author

bors r=aliher1911

@craig
Copy link
Contributor

craig bot commented Apr 4, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Apr 4, 2023

Build succeeded:

@blathers-crl
Copy link

blathers-crl bot commented Apr 4, 2023

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error creating merge commit from 35c9283 to blathers/backport-release-22.1-100554: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.1.x failed. See errors above.


error creating merge commit from 35c9283 to blathers/backport-release-22.2-100554: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 22.2.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@nvanbenschoten nvanbenschoten deleted the nvanbenschoten/fix96231 branch April 7, 2023 16:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.1.x Flags PRs that need to be backported to 23.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kvserver: v22.2.3: nil pointer in (quotapool.RateLimiter).WaitN
3 participants