Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dbnode] Set default values for BootstrapPeersConfiguration #3420

Merged
merged 2 commits into from
Apr 16, 2021

Conversation

vpranckaitis
Copy link
Collaborator

What this PR does / why we need it:

Fixes #3286: when only a subset of BootstrapPeersConfiguration parameters were supplied in config, others defaulted to 0 instead of default values mentioned in parameter description.

type BootstrapPeersConfiguration struct {
// StreamShardConcurrency controls how many shards in parallel to stream
// for in memory data being streamed between peers (most recent block).
// Defaults to: numCPU.
StreamShardConcurrency int `yaml:"streamShardConcurrency"`
// StreamPersistShardConcurrency controls how many shards in parallel to stream
// for historical data being streamed between peers (historical blocks).
// Defaults to: numCPU / 2.
StreamPersistShardConcurrency int `yaml:"streamPersistShardConcurrency"`
// StreamPersistShardFlushConcurrency controls how many shards in parallel to flush
// for historical data being streamed between peers (historical blocks).
// Defaults to: 1.
StreamPersistShardFlushConcurrency int `yaml:"streamPersistShardFlushConcurrency"`
}

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

Does this PR require updating code package or user-facing documentation?:

NONE

Comment on lines +358 to +366
if pCfg.StreamShardConcurrency != nil {
pOpts = pOpts.SetDefaultShardConcurrency(*pCfg.StreamShardConcurrency)
}
if pCfg.StreamPersistShardConcurrency != nil {
pOpts = pOpts.SetShardPersistenceConcurrency(*pCfg.StreamPersistShardConcurrency)
}
if pCfg.StreamPersistShardFlushConcurrency != nil {
pOpts = pOpts.SetShardPersistenceFlushConcurrency(*pCfg.StreamPersistShardFlushConcurrency)
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default values are already being set in peers.NewOptions():

func NewOptions() Options {
return &options{
resultOpts: result.NewOptions(),
defaultShardConcurrency: DefaultShardConcurrency,
shardPersistenceConcurrency: DefaultShardPersistenceConcurrency,
shardPersistenceFlushConcurrency: DefaultShardPersistenceFlushConcurrency,
indexSegmentConcurrency: fsbootstrapper.DefaultIndexSegmentConcurrency,
persistenceMaxQueueSize: defaultPersistenceMaxQueueSize,
// Use a zero pool, this should be overridden at config time.
contextPool: context.NewPool(context.NewOptions().
SetContextPoolOptions(pool.NewObjectPoolOptions().SetSize(0)).
SetFinalizerPoolOptions(pool.NewObjectPoolOptions().SetSize(0))),
}
}

@codecov
Copy link

codecov bot commented Apr 15, 2021

Codecov Report

Merging #3420 (92cc8c1) into master (92cc8c1) will not change coverage.
The diff coverage is n/a.

❗ Current head 92cc8c1 differs from pull request most recent head 05bd563. Consider uploading reports for the commit 05bd563 to get more accurate results

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #3420   +/-   ##
=======================================
  Coverage    72.4%    72.4%           
=======================================
  Files        1100     1100           
  Lines      102602   102602           
=======================================
  Hits        74347    74347           
  Misses      23151    23151           
  Partials     5104     5104           
Flag Coverage Δ
aggregator 76.9% <0.0%> (ø)
cluster 84.9% <0.0%> (ø)
collector 84.3% <0.0%> (ø)
dbnode 79.0% <0.0%> (ø)
m3em 74.4% <0.0%> (ø)
m3ninx 73.5% <0.0%> (ø)
metrics 19.8% <0.0%> (ø)
msg 74.4% <0.0%> (ø)
query 67.0% <0.0%> (ø)
x 80.5% <0.0%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92cc8c1...05bd563. Read the comment docs.

Copy link
Collaborator

@linasm linasm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vpranckaitis vpranckaitis enabled auto-merge (squash) April 16, 2021 06:22
@vpranckaitis vpranckaitis merged commit 24d78a4 into master Apr 16, 2021
@vpranckaitis vpranckaitis deleted the vilius/peer_bootstrap_config_default_values branch April 16, 2021 06:38
soundvibe added a commit that referenced this pull request Apr 16, 2021
* master:
  [dbnode] Set default values for BootstrapPeersConfiguration (#3420)
  [integration-tests] Use explicit version for quay.io/m3db/prometheus_remote_client_golang (#3422)
  [dtest] Fix dtest docker compose config: env => environment (#3421)
  Fix broken links to edit pages (#3419)
  [dbnode] Fix races in source_data_test.go (#3418)
  [coordinator] add more information to processed count metric (#3415)
  [dbnode] Avoid use of grow on demand worker pool for fetchTagged and aggregate (#3416)
  [docs] Fix m3aggregagtor typo (#3417)
  [x/log] Bump zap version and add logging encoder configuration (#3377)
  Do not use buffer channels if growOnDemand is true (#3414)
  [dbnode] Fix TestSeriesWriteReadParallel datapoints too far in past with -race flag (#3413)
  [docs] Update m3db operator docs with v0.13.0 features (#3397)
  [aggregator] Fix followerFlushManager metrics (#3411)
  [query] Restore optimization to skip initial fetch for timeShift and unary fns (#3408)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

m3db defaults streamXConcurrency options to 0 if a subset is specified
3 participants