feat(Elasticsearch Node): Add bulk operations for Elasticsearch #9940
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Stretch goals:
|
n8n-assistant
bot
added
community
pemontto
force-pushed
the
elasticsearch-bulk
branch
from
92811f5 to
5eb31c9
Compare
pemontto
force-pushed
the
elasticsearch-bulk
branch
2 times, most recently
from
b0593e2 to
179d9f3
Compare
pemontto
force-pushed
the
elasticsearch-bulk
branch
from
179d9f3 to
fc76c5d
Compare
Merged
|
Got released with |
MiloradFilipovic
added a commit
that referenced
this pull request
Aug 7, 2024
* master: refactor(core): Centralize scaling mode (no-changelog) (#9835) fix(editor): Remove body padding from storybook previews (no-changelog) (#10317) feat(MySQL Node): Return decimal types as numbers (#10313) 🚀 Release 1.54.0 (#10315) feat(Elasticsearch Node): Add bulk operations for Elasticsearch (#9940) feat(Stripe Trigger Node): Add Stripe webhook descriptions based on the workflow ID and name (#9956) feat(MongoDB Node): Add projection to query options on Find (#9972) fix(Invoice Ninja Node): Fix payment types (#10196) feat(HTTP Request Tool Node): Use DynamicStructuredTool with models supporting it (no-changelog) (#10246) feat: Return scopes on executions (no-changelog) (#10310) feat(Webflow Node): Update to use the v2 API (#9996) feat(Lemlist Trigger Node): Update Trigger events (#10311) feat(Calendly Trigger Node): Update event names (no-changelog) (#10129) refactor(core): Reorganize webhook related components under src/webhooks (no-changelog) (#10296) docs: Fix links to license files in readme (no-changelog) (#10257) fix(editor): Update design system Avatar component to show initials also when only firstName or lastName is given (#10308) fix(editor): Update tags filter/editor to not show non existing tag as a selectable option (#10297) fix(editor): Update project tabs test (no-changelog) (#10300) fix(core): VM2 sandbox should not throw on `new Promise` (#10298) # Conflicts: # packages/design-system/src/components/N8nAvatar/Avatar.vue
pemontto
commented
Aug 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add support for Elasticsearch bulk operations to:
It defaults to 50 items per batch, but should be easy enough to make configurable. Depending on data size it could be reasonable to do batches of 100, 200, 500.
On 247 items the node time went from over a minute (75s) to 2s.
Old
With _bulk
Errors
It should handle item indices and error correctly
Review / Merge checklist
release/backport(if the PR is an urgent fix that needs to be backported)