Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

raftstore: increase region size from 64M to 96M #1449

Merged
merged 33 commits into from
May 24, 2017
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
f915700
raftstore: increase region size from 64M to 96M
ngaut Dec 28, 2016
b1b99dc
change config file
ngaut Dec 28, 2016
66105ac
address comments
ngaut Dec 28, 2016
bba132d
address comments
ngaut Dec 29, 2016
3879ee8
address comments
ngaut Dec 29, 2016
e06396f
increase region max size
ngaut Dec 29, 2016
7687fcd
Merge branch 'master' into ngaut/increase-region-size
ngaut Jan 2, 2017
2939d31
Merge branch 'master' into ngaut/increase-region-size
ngaut Jan 6, 2017
0c2b83f
Merge branch 'master' into ngaut/increase-region-size
ngaut Jan 19, 2017
5b406a4
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 5, 2017
ea66564
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 7, 2017
a452f7b
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 9, 2017
7ea42b7
tikv-server: remove unused import
ngaut Feb 9, 2017
d6409d0
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 12, 2017
17c73ce
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 18, 2017
14b18a6
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 18, 2017
6f35f5a
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 21, 2017
4375e0d
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 21, 2017
a1919fd
Merge branch 'master' into ngaut/increase-region-size
ngaut Feb 26, 2017
e905241
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 7, 2017
223be09
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 12, 2017
7b5a809
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 14, 2017
e9c3299
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 20, 2017
8d57f4a
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 22, 2017
6b567d6
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 24, 2017
0e0893a
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 25, 2017
9203d25
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 27, 2017
0487fd8
Merge branch 'master' into ngaut/increase-region-size
ngaut Mar 30, 2017
461d55c
Merge branch 'master' into ngaut/increase-region-size
ngaut May 8, 2017
c84ba3c
Merge branch 'master' into ngaut/increase-region-size
AndreMouche May 10, 2017
34aa04e
Merge branch 'master' into ngaut/increase-region-size
ngaut May 15, 2017
2c7aadd
Merge branch 'master' into ngaut/increase-region-size
ngaut May 22, 2017
c0c9cca
Merge branch 'master' into ngaut/increase-region-size
ngaut May 24, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions etc/config-template.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,11 @@ pd-store-heartbeat-tick-interval = "10s"
# When the region's size exceeds region-max-size, we will split the region
# into two which the left region's size will be region-split-size or a little
# bit smaller.
region-max-size = "80MB"
region-split-size = "64MB"
region-max-size = "128MB"
region-split-size = "96MB"
# When region size changes exceeds region-split-check-diff, we should check
# whether the region should be split or not.
region-split-check-diff = "8MB"
region-split-check-diff = "24MB"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why 24M

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To reduce scanning times.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will delay split scan.

# Interval to check region whether need to be split or not.
split-region-check-tick-interval = "5s"

Expand Down
6 changes: 3 additions & 3 deletions src/bin/tikv-server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -412,14 +412,14 @@ fn build_cfg(matches: &Matches, config: &toml::Value, cluster_id: u64, addr: Str
cfg.raft_store.region_split_size =
get_toml_int(config,
"raftstore.region-split-size",
Some(64 * 1024 * 1024)) as u64;
Some(96 * 1024 * 1024)) as u64;
cfg.raft_store.region_max_size =
get_toml_int(config, "raftstore.region-max-size", Some(80 * 1024 * 1024)) as u64;
get_toml_int(config, "raftstore.region-max-size", Some(128 * 1024 * 1024)) as u64;

cfg.raft_store.region_check_size_diff =
get_toml_int(config,
"raftstore.region-split-check-diff",
Some(8 * 1024 * 1024)) as u64;
Some(24 * 1024 * 1024)) as u64;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be 16.


cfg.raft_store.raft_log_gc_tick_interval =
get_toml_int(config, "raftstore.raft-log-gc-tick-interval", Some(10_000)) as u64;
Expand Down
6 changes: 3 additions & 3 deletions src/raftstore/store/config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ const RAFT_LOG_GC_THRESHOLD: u64 = 50;
const RAFT_LOG_GC_COUNT_LIMIT: u64 = REGION_SPLIT_SIZE * 3 / 4 / 1024;
const RAFT_LOG_GC_SIZE_LIMIT: u64 = REGION_SPLIT_SIZE * 3 / 4;
const SPLIT_REGION_CHECK_TICK_INTERVAL: u64 = 10000;
const REGION_SPLIT_SIZE: u64 = 64 * 1024 * 1024;
const REGION_MAX_SIZE: u64 = 80 * 1024 * 1024;
const REGION_CHECK_DIFF: u64 = 8 * 1024 * 1024;
const REGION_SPLIT_SIZE: u64 = 96 * 1024 * 1024;
const REGION_MAX_SIZE: u64 = REGION_SPLIT_SIZE + REGION_SPLIT_SIZE / 3;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think REGION_SPLIT_SIZE / 4 * 5 is more appropriate. REGION_SPLIT_SIZE can be divided by 4 or 8 normally.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't need to be accurate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it will look ugly when change the split size to something else, like 160m, the max size will become 213m in which case. But if using 5 / 4, default max size will become 200m.

const REGION_CHECK_DIFF: u64 = REGION_SPLIT_SIZE / 4;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be 8.

const REGION_COMPACT_CHECK_TICK_INTERVAL: u64 = 5 * 60 * 1000; // 5 min
const REGION_COMPACT_DELETE_KEYS_COUNT: u64 = 1_000_000;
const PD_HEARTBEAT_TICK_INTERVAL: u64 = 5000;
Expand Down