Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Change memcached client default timeout setting #1000

Merged
merged 4 commits into from
Feb 2, 2022
Merged

Change memcached client default timeout setting #1000

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
2463ca3
Change memcached client default timeout setting
ortuman Feb 2, 2022
b23cedb
updated CHANGELOG.md
ortuman Feb 2, 2022
832358f
make linter happy
ortuman Feb 2, 2022
45914b3
updated CHANGELOG.md
ortuman Feb 2, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@
* [CHANGE] Ruler: removed `-ruler.enable-sharding` option, and change default value of `-ruler.ring.store` to `memberlist`. #943
* [CHANGE] Ruler: `-ruler.alertmanager-use-v2` now defaults to `true`. #954
* [CHANGE] Compactor: Removed `-compactor.sharding-enabled` option. Sharding in compactor is now always enabled. Default value of `-compactor.ring.store` has changed from `consul` to `memberlist`. Default value of `-compactor.ring.wait-stability-min-duration` is now 0, which disables the feature. #956
* [CHANGE] Changed default settings for memcached clients: #959
* [CHANGE] Changed default settings for memcached clients: #959 #1000
* The default value for the following config options has changed from `10000` to `25000`:
* `-blocks-storage.bucket-store.chunks-cache.memcached.max-async-buffer-size`
* `-blocks-storage.bucket-store.index-cache.memcached.max-async-buffer-size`
Expand All @@ -242,6 +242,11 @@
* `-blocks-storage.bucket-store.index-cache.memcached.max-idle-connections`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed the indentation of the sublists above is wrong. Can you fix them too, please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That must be very subtle.

Maybe my eye is betraying me, but after reviewing, that list seems to have the same indentation level as the rest. 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this might be some rendering issue on GitHub, when checking out the code it looks the same as the others to me

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be. I'm going to merge this PR and I will double check it, just in case.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I can't see anything wrong locally and it also renders fine here:
https://github.com/grafana/mimir/blob/main/CHANGELOG.md

* `-blocks-storage.bucket-store.metadata-cache.memcached.max-idle-connections`
* `-frontend.results-cache.memcached.max-idle-connections`
* The default value for the following config options has changed from `100ms` to `200ms`:
* `blocks-storage.bucket-store.metadata-cache.memcached.timeout`
* `blocks-storage.bucket-store.index-cache.memcached.timeout`
* `blocks-storage.bucket-store.chunks-cache.memcached.timeout`
* `frontend.results-cache.memcached.timeout`
ortuman marked this conversation as resolved.
Show resolved Hide resolved
* [CHANGE] Querier: removed `-querier.worker-match-max-concurrent` and `-querier.worker-parallelism` CLI flags (and their respective YAML config options). Mimir now behaves like if `-querier.worker-match-max-concurrent` is always enabled and you should configure the max concurrency per querier process using `-querier.max-concurrent` instead. #958
* [CHANGE] Distributor: change default value of `-distributor.instance-limits.max-inflight-push-requests` to `2000`. #964
* [CHANGE] Distributor: change default value of `-distributor.remote-timeout` from `2s` to `20s`. #970
Expand Down
8 changes: 4 additions & 4 deletions cmd/mimir/help-all.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ Usage of ./cmd/mimir/mimir:
-blocks-storage.bucket-store.chunks-cache.memcached.max-item-size int
The maximum size of an item stored in memcached. Bigger items are not stored. If set to 0, no maximum size is enforced. (default 1048576)
-blocks-storage.bucket-store.chunks-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-blocks-storage.bucket-store.chunks-cache.subrange-size int
Size of each subrange that bucket object is split into for better caching. (default 16000)
-blocks-storage.bucket-store.chunks-cache.subrange-ttl duration
Expand Down Expand Up @@ -314,7 +314,7 @@ Usage of ./cmd/mimir/mimir:
-blocks-storage.bucket-store.index-cache.memcached.max-item-size int
The maximum size of an item stored in memcached. Bigger items are not stored. If set to 0, no maximum size is enforced. (default 1048576)
-blocks-storage.bucket-store.index-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-blocks-storage.bucket-store.index-header-lazy-loading-enabled
If enabled, store-gateway will lazy load an index-header only once required by a query. (default true)
-blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout duration
Expand Down Expand Up @@ -350,7 +350,7 @@ Usage of ./cmd/mimir/mimir:
-blocks-storage.bucket-store.metadata-cache.memcached.max-item-size int
The maximum size of an item stored in memcached. Bigger items are not stored. If set to 0, no maximum size is enforced. (default 1048576)
-blocks-storage.bucket-store.metadata-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-blocks-storage.bucket-store.metadata-cache.metafile-attributes-ttl duration
How long to cache attributes of the block metafile. (default 168h0m0s)
-blocks-storage.bucket-store.metadata-cache.metafile-content-ttl duration
Expand Down Expand Up @@ -880,7 +880,7 @@ Usage of ./cmd/mimir/mimir:
-frontend.results-cache.memcached.max-item-size int
The maximum size of an item stored in memcached. Bigger items are not stored. If set to 0, no maximum size is enforced. (default 1048576)
-frontend.results-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-frontend.scheduler-address string
DNS hostname used for finding query-schedulers.
-frontend.scheduler-dns-lookup-period duration
Expand Down
8 changes: 4 additions & 4 deletions cmd/mimir/help.txt.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,21 +122,21 @@ Usage of ./cmd/mimir/mimir:
-blocks-storage.bucket-store.chunks-cache.memcached.addresses string
Comma separated list of memcached addresses. Supported prefixes are: dns+ (looked up as an A/AAAA query), dnssrv+ (looked up as a SRV query, dnssrvnoa+ (looked up as a SRV query, with no A/AAAA lookup made after that).
-blocks-storage.bucket-store.chunks-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-blocks-storage.bucket-store.index-cache.backend string
The index cache backend type. Supported values: inmemory, memcached. (default "inmemory")
-blocks-storage.bucket-store.index-cache.inmemory.max-size-bytes uint
Maximum size in bytes of in-memory index cache used to speed up blocks index lookups (shared between all tenants). (default 1073741824)
-blocks-storage.bucket-store.index-cache.memcached.addresses string
Comma separated list of memcached addresses. Supported prefixes are: dns+ (looked up as an A/AAAA query), dnssrv+ (looked up as a SRV query, dnssrvnoa+ (looked up as a SRV query, with no A/AAAA lookup made after that).
-blocks-storage.bucket-store.index-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-blocks-storage.bucket-store.metadata-cache.backend string
Backend for metadata cache, if not empty. Supported values: memcached.
-blocks-storage.bucket-store.metadata-cache.memcached.addresses string
Comma separated list of memcached addresses. Supported prefixes are: dns+ (looked up as an A/AAAA query), dnssrv+ (looked up as a SRV query, dnssrvnoa+ (looked up as a SRV query, with no A/AAAA lookup made after that).
-blocks-storage.bucket-store.metadata-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-blocks-storage.bucket-store.sync-dir string
Directory to store synchronized TSDB index headers. (default "tsdb-sync")
-blocks-storage.filesystem.dir string
Expand Down Expand Up @@ -354,7 +354,7 @@ Usage of ./cmd/mimir/mimir:
-frontend.results-cache.memcached.addresses string
Comma separated list of memcached addresses. Supported prefixes are: dns+ (looked up as an A/AAAA query), dnssrv+ (looked up as a SRV query, dnssrvnoa+ (looked up as a SRV query, with no A/AAAA lookup made after that).
-frontend.results-cache.memcached.timeout duration
The socket read/write timeout. (default 100ms)
The socket read/write timeout. (default 200ms)
-frontend.scheduler-address string
DNS hostname used for finding query-schedulers.
-frontend.split-queries-by-interval duration
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/configuration/config-file-reference.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3541,7 +3541,7 @@ The `memcached_config` configures the Memcached-based caching backend. The suppo

# The socket read/write timeout.
# CLI flag: -<prefix>.memcached.timeout
[timeout: <duration> | default = 100ms]
[timeout: <duration> | default = 200ms]

# [advanced] The maximum number of idle connections that will be maintained per
# address.
Expand Down
4 changes: 0 additions & 4 deletions operations/mimir-tests/test-defaults-generated.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -748,7 +748,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.gcs.bucket-name=blocks-bucket
- -consul.hostname=consul.default.svc.cluster.local:8500
Expand Down Expand Up @@ -1314,15 +1313,13 @@ spec:
- -blocks-storage.bucket-store.chunks-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.chunks-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.ignore-blocks-within=10h
- -blocks-storage.bucket-store.index-cache.backend=memcached
- -blocks-storage.bucket-store.index-cache.memcached.addresses=dnssrvnoa+memcached-index-queries.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.index-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.index-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.index-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.index-cache.memcached.max-item-size=5242880
- -blocks-storage.bucket-store.index-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.index-header-lazy-loading-enabled=true
- -blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout=60m
- -blocks-storage.bucket-store.max-chunk-pool-bytes=12884901888
Expand All @@ -1332,7 +1329,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.gcs.bucket-name=blocks-bucket
- -runtime-config.file=/etc/mimir/overrides.yaml
Expand Down
5 changes: 0 additions & 5 deletions operations/mimir-tests/test-disable-chunk-streaming-generated.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -825,7 +825,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down Expand Up @@ -1057,7 +1056,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down Expand Up @@ -1639,15 +1637,13 @@ spec:
- -blocks-storage.bucket-store.chunks-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.chunks-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.ignore-blocks-within=10h
- -blocks-storage.bucket-store.index-cache.backend=memcached
- -blocks-storage.bucket-store.index-cache.memcached.addresses=dnssrvnoa+memcached-index-queries.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.index-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.index-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.index-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.index-cache.memcached.max-item-size=5242880
- -blocks-storage.bucket-store.index-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.index-header-lazy-loading-enabled=true
- -blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout=60m
- -blocks-storage.bucket-store.max-chunk-pool-bytes=12884901888
Expand All @@ -1657,7 +1653,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down
5 changes: 0 additions & 5 deletions operations/mimir-tests/test-query-sharding-generated.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -824,7 +824,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down Expand Up @@ -1060,7 +1059,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down Expand Up @@ -1642,15 +1640,13 @@ spec:
- -blocks-storage.bucket-store.chunks-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.chunks-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.ignore-blocks-within=10h
- -blocks-storage.bucket-store.index-cache.backend=memcached
- -blocks-storage.bucket-store.index-cache.memcached.addresses=dnssrvnoa+memcached-index-queries.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.index-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.index-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.index-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.index-cache.memcached.max-item-size=5242880
- -blocks-storage.bucket-store.index-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.index-header-lazy-loading-enabled=true
- -blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout=60m
- -blocks-storage.bucket-store.max-chunk-pool-bytes=12884901888
Expand All @@ -1660,7 +1656,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down
5 changes: 0 additions & 5 deletions operations/mimir-tests/test-shuffle-sharding-generated.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -825,7 +825,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down Expand Up @@ -1062,7 +1061,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.addresses=dnssrvnoa+memcached-metadata.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.metadata-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down Expand Up @@ -1649,15 +1647,13 @@ spec:
- -blocks-storage.bucket-store.chunks-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.chunks-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.chunks-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.ignore-blocks-within=10h
- -blocks-storage.bucket-store.index-cache.backend=memcached
- -blocks-storage.bucket-store.index-cache.memcached.addresses=dnssrvnoa+memcached-index-queries.default.svc.cluster.local:11211
- -blocks-storage.bucket-store.index-cache.memcached.max-async-concurrency=50
- -blocks-storage.bucket-store.index-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.index-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.index-cache.memcached.max-item-size=5242880
- -blocks-storage.bucket-store.index-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.index-header-lazy-loading-enabled=true
- -blocks-storage.bucket-store.index-header-lazy-loading-idle-timeout=60m
- -blocks-storage.bucket-store.max-chunk-pool-bytes=12884901888
Expand All @@ -1667,7 +1663,6 @@ spec:
- -blocks-storage.bucket-store.metadata-cache.memcached.max-get-multi-concurrency=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-idle-connections=100
- -blocks-storage.bucket-store.metadata-cache.memcached.max-item-size=1048576
- -blocks-storage.bucket-store.metadata-cache.memcached.timeout=200ms
- -blocks-storage.bucket-store.sync-dir=/data/tsdb
- -blocks-storage.bucket-store.sync-interval=15m
- -blocks-storage.gcs.bucket-name=blocks-bucket
Expand Down
Loading