[kbn/optimizer] poll parent process to avoid zombie processes #67059
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Definitely seeing the child processes cleaned up with this. |
spalger
added
release_note:skip
|
Pinging @elastic/kibana-operations (Team:Operations) |
tylersmalley
approved these changes
May 20, 2020
|
If you don't foresee any issues, I think we should also backport to 7.7 to ensure we don't run into this down the road if we ever have to release another 7.7 build or for plugin authors. |
💚 Build SucceededHistory
To update your PR or re-run it, just comment with: |
spalger
pushed a commit
to spalger/kibana
that referenced
this pull request
May 20, 2020
spalger
pushed a commit
to spalger/kibana
that referenced
this pull request
May 20, 2020
spalger
pushed a commit
to spalger/kibana
that referenced
this pull request
May 20, 2020
spalger
pushed a commit
that referenced
this pull request
May 20, 2020
spalger
pushed a commit
that referenced
this pull request
May 20, 2020
spalger
pushed a commit
that referenced
this pull request
May 20, 2020
jloleysens
added a commit
to jloleysens/kibana
that referenced
this pull request
May 20, 2020
…ent/add-support-in-url-for-hidden-toggle * 'master' of github.com:elastic/kibana: (49 commits) [Uptime] Improve responsiveness details page (elastic#67034) skip flaky suite (elastic#66669) Revert "Integration of a static filesystem for the node_modules (elastic#47998)" (elastic#67124) Support api_integration/kibana/stats against remote hosts (elastic#53000) chore(NA): add module name mapper for src plugins on x-pack (elastic#67103) Change the error message on TSVB in order to be more user friendly (elastic#67090) [kbn/optimizer] poll parent process to avoid zombie processes (elastic#67059) [Visualize] Lazy load default editor, fix duplicated styles (elastic#66732) Bump styled-component dependencies (elastic#66611) Bump react-markdown dependencies (elastic#66615) Fix Core docs links (elastic#66977) Timelion graph is not refreshing content after searching or filtering (elastic#67023) Remove `--xpack.endpoint.enabled=true` from README.md file (elastic#67053) Move apm tutorial from apm plugin into apm_oss plugin (elastic#66432) [Logs UI] Restore call to `UsageCollector.countLogs` (elastic#67051) Remove unused license check result from LP Security plugin (elastic#66966) [Saved Objects] adds support for including hidden types in saved objects client (elastic#66879) [Discover] Deangularize timechart header (elastic#66532) [Discover] Improve and unskip a11y context view test (elastic#66959) [SIEM] Refactor Timeline.timelineType draft to Timeline.status draft (elastic#66864) ... # Conflicts: # x-pack/plugins/index_management/__jest__/client_integration/helpers/home.helpers.ts
gmmorris
added a commit
to gmmorris/kibana
that referenced
this pull request
May 21, 2020
* master: [apm] Annotation API documentation (elastic#65963) [Uptime] Improve responsiveness details page (elastic#67034) skip flaky suite (elastic#66669) Revert "Integration of a static filesystem for the node_modules (elastic#47998)" (elastic#67124) Support api_integration/kibana/stats against remote hosts (elastic#53000) chore(NA): add module name mapper for src plugins on x-pack (elastic#67103) Change the error message on TSVB in order to be more user friendly (elastic#67090) [kbn/optimizer] poll parent process to avoid zombie processes (elastic#67059) [Visualize] Lazy load default editor, fix duplicated styles (elastic#66732) Bump styled-component dependencies (elastic#66611) Bump react-markdown dependencies (elastic#66615) Fix Core docs links (elastic#66977) Timelion graph is not refreshing content after searching or filtering (elastic#67023) Remove `--xpack.endpoint.enabled=true` from README.md file (elastic#67053) Move apm tutorial from apm plugin into apm_oss plugin (elastic#66432) [Logs UI] Restore call to `UsageCollector.countLogs` (elastic#67051) Remove unused license check result from LP Security plugin (elastic#66966)
gmmorris
added a commit
to gmmorris/kibana
that referenced
this pull request
May 21, 2020
* master: (21 commits) [Alerting] Hides the `alert` SavedObjects type (elastic#66719) skip flaky suite (elastic#66869) fix visual baseline tests [kbn/optimizer] require fsevents on macos (elastic#67147) [APM] Fix obscured service map connections (elastic#67129) [apm] Annotation API documentation (elastic#65963) [Uptime] Improve responsiveness details page (elastic#67034) skip flaky suite (elastic#66669) Revert "Integration of a static filesystem for the node_modules (elastic#47998)" (elastic#67124) Support api_integration/kibana/stats against remote hosts (elastic#53000) chore(NA): add module name mapper for src plugins on x-pack (elastic#67103) Change the error message on TSVB in order to be more user friendly (elastic#67090) [kbn/optimizer] poll parent process to avoid zombie processes (elastic#67059) [Visualize] Lazy load default editor, fix duplicated styles (elastic#66732) Bump styled-component dependencies (elastic#66611) Bump react-markdown dependencies (elastic#66615) Fix Core docs links (elastic#66977) Timelion graph is not refreshing content after searching or filtering (elastic#67023) Remove `--xpack.endpoint.enabled=true` from README.md file (elastic#67053) Move apm tutorial from apm plugin into apm_oss plugin (elastic#66432) ...
spalger
added a commit
to spalger/kibana
that referenced
this pull request
May 22, 2020
…rocesses (elastic#67059)"" This reverts commit dcfa485.
spalger
added a commit
to spalger/kibana
that referenced
this pull request
May 22, 2020
…cesses (elastic#67059) (elastic#67119)" This reverts commit e6a5a9d.
spalger
added a commit
to spalger/kibana
that referenced
this pull request
May 22, 2020
…cesses (elastic#67059) (elastic#67118)" This reverts commit 46f916d.
spalger
added a commit
that referenced
this pull request
May 23, 2020
* Revert "[7.7] [kbn/optimizer] remove disconnect listener (#67161) (#67185)" This reverts commit a1c7cf1. * Revert "[7.7] [kbn/optimizer] poll parent process to avoid zombie processes (#67059) (#67119)" This reverts commit e6a5a9d. Co-authored-by: spalger <[email protected]> Co-authored-by: Elastic Machine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We're doing some debugging, trying to find potential explanations for why the memory usage of the optimizer on relatively beefy machines seems to be way higher than expected. Part of that debugging lead to realizing that zombie workers still seems possible, so this PR adds a polling mechanism to workers that will ask the parent process to send pongs back every 5 seconds or so. If a response is not received within 5 second the worker will exit. Additionally, if the "disconnect" event is observed in the worker, or the
process.connectedorprocess.sendproperties are set to falsy values, the worker will also exit.