Configure a limit on direct memory usage #41954
Assignees
Labels
Comments
DaveCTurner
added
>enhancement
resiliency
:Core/Infra/Core
|
Pinging @elastic/es-core-infra |
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this issue
May 10, 2019
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates elastic#41954
DaveCTurner
added a commit
that referenced
this issue
May 10, 2019
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates #41954
DaveCTurner
added a commit
that referenced
this issue
May 10, 2019
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates #41954
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this issue
May 27, 2019
This commit slightly reworks the recommendations in the docs about setting the heap size: * the "rules of thumb" are actually instructions that should be followed * the reason for setting `Xmx` to 50% of the heap size is more subtle than just leaving space for the filesystem cache * it is normal to see Elasticsearch using more memory than `Xmx` * replace `cutoff` and `limit` with `threshold` since all three terms are used interchangeably * since we recommend setting `Xmx` equal to `Xms`, avoid talking about setting `Xmx` in isolation Relates elastic#41954
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Sometimes a node can use substantially more memory than configured by the
-Xmxsetting. This setting limits the size of the JVM heap, but there are other memory consumers that are not controlled by this setting. The documentation says the following:This recommendation can result in running out of physical RAM, because we might allocate another 50% of the RAM as direct buffers, plus JVM overheads, OS overheads, etc. The consequence on Linux is often that Elasticsearch is killed by the OOM killer.
Direct memory is a major consumer of off-heap memory. The maximum amount of direct memory can be controlled with the
-XX:MaxDirectMemorySizesetting. At the moment this setting is unset by default. We discussed this today and decided against asking users to configure this setting, because it is easy to misconfigure and misconfiguration can lead to bad situations such as heavy GCs and OOMs.We decided to automatically set
-XX:MaxDirectMemorySizeto half of the-Xmxsetting, unless explicitly set by the user, in which case we would respect that setting.The text was updated successfully, but these errors were encountered: