Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(core): Reduce memory usage in the Webhook node #4640

Merged
merged 1 commit into from
Nov 24, 2022

Conversation

netroy
Copy link
Member

@netroy netroy commented Nov 17, 2022

This PR changes the Webhook node and the BinaryDataManager to avoid loading the binary payload even in memory, preventing any OOM crashes on large binary uploads

TODO:

  • update the binary download mechanism to use streaming instead of loading the base64 encoded data in memory
  • make everything work in BinaryDataManager's default mode

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team node/improvement New feature or request labels Nov 17, 2022
@netroy netroy force-pushed the N8N-4571-webhook-memory-optimization branch 4 times, most recently from e706a2d to 3eaeb0e Compare November 23, 2022 13:53
@netroy netroy marked this pull request as ready for review November 23, 2022 13:53
@netroy netroy force-pushed the N8N-4571-webhook-memory-optimization branch from 3eaeb0e to 46e8ef4 Compare November 23, 2022 13:56
@netroy netroy requested a review from ahsanv November 23, 2022 13:56
Copy link
Member

@ahsanv ahsanv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks alright!

@netroy netroy merged commit 07e4743 into master Nov 24, 2022
@netroy netroy deleted the N8N-4571-webhook-memory-optimization branch November 24, 2022 15:54
@netroy netroy added the Upcoming Release Will be part of the upcoming release label Nov 24, 2022
@janober janober added Upcoming Release Will be part of the upcoming release and removed Upcoming Release Will be part of the upcoming release labels Nov 24, 2022
MiloradFilipovic added a commit that referenced this pull request Nov 25, 2022
* master:
  refactor: Improve typings for element-ui imports (no-changelog) (#4678)
  fix(core): Mark binary data to be deleted when pruning executions (#4713)
  test: Add e2e tests for workflow/credential migrations when enabling UM (#4719)
  ci: Setup cypress tasks for resetting DB, and setting up an owner (#4717)
  fix: Fix Windows development environments (no-changelog) (#4720)
  refactor(core): Reduce memory usage in the Webhook node (#4640)
  📚 Update CHANGELOG.md and main package.json to 0.204.0
  🔖 Release [email protected]
  ⬆️ Set [email protected], [email protected], [email protected] and [email protected] on n8n
  🔖 Release [email protected]
  ⬆️ Set [email protected] and [email protected] on n8n-editor-ui
  🔖 Release [email protected]
  🔖 Release [email protected]
  ⬆️ Set [email protected] and [email protected] on n8n-nodes-base
  🔖 Release [email protected]
  ⬆️ Set [email protected] and [email protected] on n8n-node-dev
  🔖 Release [email protected]
  ⬆️ Set [email protected] on n8n-core
  🔖 Release [email protected]
  fix: Apply SpecialNodeParameters to the types generated for the frontend (no-changelog) (#4715)
@janober
Copy link
Member

janober commented Dec 7, 2022

Got released with [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team node/improvement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants