Refine size-your-shards wording #89081
Conversation
Clarify that the limits in the docs are absolute maxima that will avoid things just breaking but won't necessarily give great performance.
DaveCTurner
added
>docs
DaveCTurner
requested review from
jakommo,
original-brownbear and
henningandersen
elasticsearchmachine
added
Team:Distributed (Obsolete)
|
Pinging @elastic/es-docs (Team:Docs) |
|
Pinging @elastic/es-distributed (Team:Distributed) |
There was a problem hiding this comment.
Thanks a lot @DaveCTurner ❤️ This makes it much more clear now !
| @@ -175,17 +175,24 @@ index prirep shard store | |||
|
|
|||
| [discrete] | |||
| [[shard-count-recommendation]] | |||
| ==== Aim for 3000 indices or fewer per GB of heap memory on each master node | |||
| ==== Aim for fewer than 3000 indices per GB of heap memory on each master node | |||
There was a problem hiding this comment.
I think we should reverse the sentence here to say
Aim for 1 GB of heap memory on each master per 3000 indices
It is a recommendation for handling the necessary amount of indices, not a recommendation for how many indices they should have.
There was a problem hiding this comment.
I prefer this order for a couple of reasons:
-
IMO it's more common that the user knows the size of their nodes and is trying to work out a suitable sharding and retention strategy to match. For instance on Cloud there's only a few different possible master heap sizes.
-
We want to phrase this as a bound, not a target, and I find it less natural to say "aim for more than 1GB of heap per 3000 indices".
There was a problem hiding this comment.
About 1, this seems to be the wrong way around and I'd like to encourage that in the heading.
About 2, we can perhaps call it:
| ==== Aim for fewer than 3000 indices per GB of heap memory on each master node | |
| ==== Masters should have at least 1 GB of heap per 3000 indices. |
| each dedicated master node should have at least 4GB of heap. For non-dedicated | ||
| master nodes, the same rule holds and should be added to the heap requirements | ||
| of the other roles of each node. | ||
| As a general rule of thumb, you should have fewer than 3000 indices per GB of |
There was a problem hiding this comment.
Can we also reverse the sentence here to derive the GB from number of indices?
| @@ -175,17 +175,24 @@ index prirep shard store | |||
|
|
|||
| [discrete] | |||
| [[shard-count-recommendation]] | |||
| ==== Aim for 3000 indices or fewer per GB of heap memory on each master node | |||
| ==== Aim for fewer than 3000 indices per GB of heap memory on each master node | |||
There was a problem hiding this comment.
About 1, this seems to be the wrong way around and I'd like to encourage that in the heading.
About 2, we can perhaps call it:
| ==== Aim for fewer than 3000 indices per GB of heap memory on each master node | |
| ==== Masters should have at least 1 GB of heap per 3000 indices. |
DaveCTurner
added
auto-merge-without-approval
Clarify that the limits in the docs are absolute maxima that will avoid things just breaking but won't necessarily give great performance.
Clarify that the limits in the docs are absolute maxima that will avoid things just breaking but won't necessarily give great performance.
Clarify that the limits in the docs are absolute maxima that will avoid things just breaking but won't necessarily give great performance.
Clarify that the limits in the docs are absolute maxima that will avoid
things just breaking but won't necessarily give great performance.