-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cfetcher: correctly update the limit hint #88391
Conversation
In 41fa8b6 (which was supposed to be a "noop" refactor) we introduced a bug which made it so that we no longer update the remaining limit hint correctly. As a result, the cFetcher might no longer respect the limit hint. What makes things worse is the fact that the KV layer still does everything correctly, so when the cFetcher asks for more rows that exceed the limit, the KV layer does a BatchRequest with 10x of the original limit. This is now fixed by correctly updating the limit hint right before emitting the batch. Release note (bug fix): CockroachDB no longer fetches unnecessary rows for queries with LIMITs. The bug was introduced in 22.1.7.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @DrewKimball and @mgartner)
pkg/sql/colfetcher/cfetcher.go
line 852 at r1 (raw file):
cf.shiftState() if cf.machine.limitHint > 0 && cf.machine.rowIdx >= cf.machine.limitHint {
The bug was that this condition would never be true if LIMIT
exceeds 1024 because rowIdx
is at most 1024.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @mgartner)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: complete! 2 of 0 LGTMs obtained (waiting on @DrewKimball and @mgartner)
TFTRs! bors r+ |
Build succeeded: |
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from 1d6621c to blathers/backport-release-22.1-88391: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 22.1.x failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan. |
In 41fa8b6 (which was supposed to be a "noop" refactor) we introduced a bug which made it so that we no longer update the remaining limit hint correctly. As a result, the cFetcher might no longer respect the limit hint. What makes things worse is the fact that the KV layer still does everything correctly, so when the cFetcher asks for more rows that exceed the limit, the KV layer does a BatchRequest with 10x of the original limit. This is now fixed by correctly updating the limit hint right before emitting the batch.
Addresses: #88382.
Epic: CRDB-20535
Release note (bug fix): CockroachDB no longer fetches unnecessary rows for queries with LIMITs. The bug was introduced in 22.1.7.