Recycle pages used by outgoing publications #77407
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Today `PublicationTransportHandler.PublicationContext` allocates a bunch of memory for serialized cluster states and diffs, but it uses a plain `BytesStreamOutput` which means that the backing pages are allocated by the `BigArrays#NON_RECYCLING_INSTANCE`. With this commit we pass in a proper `BigArrays` so that the pages being used can be recycled.
DaveCTurner
added
>enhancement
:Distributed Coordination/Cluster Coordination
elasticmachine
added
the
Team:Distributed (Obsolete)
|
Pinging @elastic/es-distributed (Team:Distributed) |
|
This failed in CI after merging, see https://gradle-enterprise.elastic.co/s/o2issgehbeciw (despite repeated passes locally, although with a couple of changes since the soak test). I've reverted it and reopened this as a PR to investigate further later. |
|
It turned out to be NBD, one test needed a final |
|
Gentle ping for a review @original-brownbear, it's just a couple of small changes over what was already reviewed so should be a quick one. |
original-brownbear
approved these changes
Sep 13, 2021
DaveCTurner
added
the
auto-merge-without-approval
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Sep 13, 2021
Today we protect against releasing a request from a remote node until its handler has completed, but we do not have the same protection for requests from the local node. This commit adds the missing refcounting. Relates elastic#77407 Closes elastic#77634
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this pull request
Sep 13, 2021
Today we protect against releasing a request from a remote node until its handler has completed, but we do not have the same protection for requests from the local node. This commit adds the missing refcounting. Relates elastic#77407 Closes elastic#77634
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Today
PublicationTransportHandler.PublicationContextallocates a bunchof memory for serialized cluster states and diffs, but it uses a plain
BytesStreamOutputwhich means that the backing pages are allocated bythe
BigArrays#NON_RECYCLING_INSTANCE. With this commit we pass in aproper
BigArraysso that the pages being used can be recycled.Re-run of #77317