Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

util/paging: choose min paging size default value as 128, and max value as 8192 #36331

Merged
merged 6 commits into from
Jul 19, 2022
Merged

util/paging: choose min paging size default value as 128, and max value as 8192 #36331

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
85bed73
util/paging: choose min paging size default value as 128, and max val…
tiancaiamao Jul 19, 2022
b06a101
fix CI
tiancaiamao Jul 19, 2022
9f45edb
Merge branch 'master' into default-paging-size
ti-chi-bot Jul 19, 2022
9d580d6
Merge branch 'master' into default-paging-size
ti-chi-bot Jul 19, 2022
1667dc3
Merge branch 'master' into default-paging-size
ti-chi-bot Jul 19, 2022
8c106a4
Merge branch 'master' into default-paging-size
ti-chi-bot Jul 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions util/paging/paging.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,15 @@ package paging
import "math"

// A paging request may be separated into multi requests if there are more data than a page.
// The paging size grows from min to max, it's not well tuned yet.
// e.g. a paging request scans over range (r1, r200), it requires 64 rows in the first batch,
// The paging size grows from min to max. See https://github.com/pingcap/tidb/issues/36328
// e.g. a paging request scans over range (r1, r200), it requires 128 rows in the first batch,
// if it's not drained, then the paging size grows, the new range is calculated like (r100, r200), then send a request again.
// Compare with the common unary request, paging request allows early access of data, it offers a streaming-like way processing data.
// TODO: may make the paging parameters configurable.
const (
MinPagingSize uint64 = 64
MinPagingSize uint64 = 128
maxPagingSizeShift = 7
pagingSizeGrow = 2
MaxPagingSize = MinPagingSize << maxPagingSizeShift
MaxPagingSize = 8192
pagingGrowingSum = ((2 << maxPagingSizeShift) - 1) * MinPagingSize
Threshold uint64 = 960
)
Expand Down