Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Optimizer] Fix a stack overflow with watch_cache when it attempts to delete very large folders. #54457

Merged
merged 5 commits into from
Jan 11, 2020

Conversation

chrisdavies
Copy link
Contributor

@chrisdavies chrisdavies commented Jan 10, 2020

This happens occasionally when the .cache/ui_bundles has a lot of files. The reason is that globby spreads all of the file paths as arguments to a function call: sindresorhus/globby#141

This is a quick workaround for the offending call site.

You can reproduce the bug by running node and then running a script like this (after replacing the placeholders):

const fs = require('fs');

for (let i = 1; i < 80000; ++i) {
  fs.writeFileSync(
    `PREFIX/kibana/data/built_assets/.cache/ui_bundles/BUNDLE_PATH/babel/test_${i}.json`,
    '{}'
  );
}
  • Replace PREFIX/ with whatever is appropriate for your system
  • Replace BUNDLE_PATH with whatever the existing path is on your system
  • Change the yarn.lock file in some way yarn add ramda or something
  • Run node scripts/functional_tests_server to see the error.

@chrisdavies chrisdavies requested a review from a team as a code owner January 10, 2020 14:30
@chrisdavies chrisdavies requested a review from spalger January 10, 2020 14:31
@chrisdavies chrisdavies added the release_note:skip Skip the PR/issue when compiling release notes label Jan 10, 2020
@chrisdavies chrisdavies deleted the optimizer/max_stack_size_bug branch January 10, 2020 14:54
@chrisdavies chrisdavies reopened this Jan 10, 2020
@chrisdavies
Copy link
Contributor Author

retest

@chrisdavies
Copy link
Contributor Author

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

💚 Build Succeeded

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Contributor

@tylersmalley tylersmalley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks the fix!

@chrisdavies chrisdavies merged commit 8ef5609 into elastic:master Jan 11, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jan 13, 2020
* master: (69 commits)
  [Graph] Fix various a11y issues (elastic#54097)
  Add ApplicationService app status management (elastic#50223)
  logs in one time (elastic#54447)
  Deprecate using `elasticsearch.ssl.certificate` without `elasticsearch.ssl.key` and vice versa (elastic#54392)
  [Optimizer] Fix a stack overflow with watch_cache when it attempts to delete very large folders. (elastic#54457)
  Security - Role Mappings UI (elastic#53620)
  [SIEM] [Detection engine] Permission II (elastic#54292)
  Allow User to Cleanup Repository from UI  (elastic#53047)
  [Detection engine] Some UX for rule creation (elastic#54471)
  share specific instances of some ui packages (elastic#54079)
  [ML] APM modules configs for RUM Javascript and NodeJS (elastic#53792)
  [APM] Delay rendering invalid license notification (elastic#53924)
  [Graph] Improve error message on graph requests (elastic#54230)
  [ILM] Kibana should allow a min_age setting of 0ms in ILM policy phases (elastic#53719)
  Unit Tests for common/lib (elastic#53736)
  [Graph] Only show explorable fields (elastic#54101)
  remove linting rule exception for markdown (elastic#54232)
  [Monitoring] Fetch shard data more efficiently (elastic#54028)
  [Maps] Add hiddenLayers option to embeddable map input (elastic#54355)
  Pass termOrder and hasTermsAgg properties to serializeThresholdWatch function (elastic#54391)
  ...
gmmorris added a commit to gmmorris/kibana that referenced this pull request Jan 13, 2020
* master: (69 commits)
  [Graph] Fix various a11y issues (elastic#54097)
  Add ApplicationService app status management (elastic#50223)
  logs in one time (elastic#54447)
  Deprecate using `elasticsearch.ssl.certificate` without `elasticsearch.ssl.key` and vice versa (elastic#54392)
  [Optimizer] Fix a stack overflow with watch_cache when it attempts to delete very large folders. (elastic#54457)
  Security - Role Mappings UI (elastic#53620)
  [SIEM] [Detection engine] Permission II (elastic#54292)
  Allow User to Cleanup Repository from UI  (elastic#53047)
  [Detection engine] Some UX for rule creation (elastic#54471)
  share specific instances of some ui packages (elastic#54079)
  [ML] APM modules configs for RUM Javascript and NodeJS (elastic#53792)
  [APM] Delay rendering invalid license notification (elastic#53924)
  [Graph] Improve error message on graph requests (elastic#54230)
  [ILM] Kibana should allow a min_age setting of 0ms in ILM policy phases (elastic#53719)
  Unit Tests for common/lib (elastic#53736)
  [Graph] Only show explorable fields (elastic#54101)
  remove linting rule exception for markdown (elastic#54232)
  [Monitoring] Fetch shard data more efficiently (elastic#54028)
  [Maps] Add hiddenLayers option to embeddable map input (elastic#54355)
  Pass termOrder and hasTermsAgg properties to serializeThresholdWatch function (elastic#54391)
  ...
mistic pushed a commit that referenced this pull request Jan 14, 2020
… delete very large folders. (#54457) (#54535)

Co-authored-by: Elastic Machine <[email protected]>
@mistic
Copy link
Member

mistic commented Jan 14, 2020

7.x: 63d0b60

jkelastic pushed a commit to jkelastic/kibana that referenced this pull request Jan 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release_note:skip Skip the PR/issue when compiling release notes v7.6.0 v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants