Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

storage: increase maxSize of key-value to 512MB, in pebbleMVCCScanner #61818

Merged
merged 1 commit into from
Mar 11, 2021
Merged

storage: increase maxSize of key-value to 512MB, in pebbleMVCCScanner #61818

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion pkg/storage/pebble_mvcc_scanner.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,12 @@ func (p *pebbleResults) put(key MVCCKey, value []byte) {
// Key value lengths take up 8 bytes (2 x Uint32).
const kvLenSize = 8
const minSize = 16
const maxSize = 128 << 20 // 128 MB
// This used to be 128MB, but the enforcement of key+value size limit is
// currently done based on the cluster setting kv.raft.command.max_size,
// which defaults to 64MB but can be increased. Until we add proper
// enforcement, we are increasing this to 512MB, under the assumption that
// no one will increase kv.raft.command.max_size to > 512MB.
const maxSize = 512 << 20 // 512 MB

// We maintain a list of buffers, always encoding into the last one (a.k.a.
// pebbleResults.repr). The size of the buffers is exponentially increasing,
Expand Down