Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Switch binary filesystem mode to nested path structure #7307

Merged

Conversation

ivov
Copy link
Contributor

@ivov ivov commented Sep 29, 2023

Depends on #7253 | Story: PAY-863

This PR introduces filesystem-v2 to store binary data in the filesystem in the same format as s3.

valya
valya previously approved these changes Oct 4, 2023
Copy link
Contributor

@valya valya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple questions but other than that, I think it's good.

packages/core/src/BinaryData/FileSystem.manager.ts Outdated Show resolved Hide resolved
@cypress
Copy link

cypress bot commented Oct 4, 2023

Passing run #2444 ↗︎

0 250 0 0 Flakiness 0

Details:

🌳 pay-863-switch-binary-filesystem-mode-to-nested-path-structure 🖥️ browsers:n...
Project: n8n Commit: 3a1fe52195
Status: Passed Duration: 08:51 💡
Started: Oct 10, 2023 7:19 AM Ended: Oct 10, 2023 7:28 AM

Review all test suite changes for PR #7307 ↗︎

@github-actions
Copy link
Contributor

github-actions bot commented Oct 4, 2023

✅ All Cypress E2E specs passed

Base automatically changed from pay-848-integrate-object-store-service-as-binary-data-manager to master October 5, 2023 13:25
@ivov ivov dismissed valya’s stale review October 5, 2023 13:25

The base branch was changed.

krynble
krynble previously approved these changes Oct 5, 2023
Copy link
Contributor

@krynble krynble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested the feature for backwards compatibility and new behavior, all working nicely! Thanks 💯

@github-actions
Copy link
Contributor

github-actions bot commented Oct 6, 2023

Great PR! Please pay attention to the following items before merging:

Files matching packages/**:

  • If fixing bug, added test to cover scenario.
  • If addressing forum or Github issue, added link to description.

Files matching packages/**/*.ts:

  • Added unit tests to cover new or updated functionality.

Make sure to check off this list before asking for review.

@github-actions
Copy link
Contributor

✅ All Cypress E2E specs passed

@ivov ivov merged commit 0847623 into master Oct 10, 2023
55 checks passed
@ivov ivov deleted the pay-863-switch-binary-filesystem-mode-to-nested-path-structure branch October 10, 2023 08:06
MiloradFilipovic added a commit that referenced this pull request Oct 10, 2023
* master: (42 commits)
  fix(editor): Make workflow history button available only for dev builds (#7392)
  refactor: Upgrade to pnpm 8.9 (no-changelog) (#7393)
  ci: Identify NPM releases as `stable` (no-changelog)
  fix(editor): Implement canvas zoom UX improvements (#7376)
  feat(core): Switch binary filesystem mode to nested path structure (#7307)
  fix(editor): Fix completions for `.json` on quoted node name in Code node (#7382)
  fix(Notion Node): Handle empty values correctly for Notion selects + multi selects (#7383)
  feat(editor): Make PDF and Audio binary-data viewable in the UI (#7367)
  fix(Google Sheets Node): Fix "Maximum call stack size exceeded" error on too many rows (#7384)
  refactor(core): Refactor nodes loading (no-changelog) (#7283)
  fix(core): Add an option to enable postgres ssl with default certs (#6889)
  feat(editor): Workflow history [WIP]- Add restore and clone into new workflow actions (no-changelog) (#7359)
  fix(HTML Node): Update property fields to not use expressions on drag (#7379)
  fix: Add role check for upgrade path (#7374)
  fix(editor): Remove excess margin below run data editor (#7372)
  fix(Google Drive Trigger Node): Add Shared Drives support (#7369)
  feat(core): Add Job Summary to Worker response (#7360)
  feat(Execute Workflow Node): Run once for each item mode (#7289)
  refactor(core): Move `copyInputItems` to node helpers (no-changelog) (#7299)
  refactor(core): Create controller for binary data (no-changelog) (#7363)
  ...

# Conflicts:
#	cypress/e2e/5-ndv.cy.ts
@github-actions github-actions bot mentioned this pull request Oct 11, 2023
netroy added a commit that referenced this pull request Oct 11, 2023
# [1.11.0](https://github.com/n8n-io/n8n/compare/[email protected]@1.11.0)
(2023-10-11)


### Bug Fixes

* **core:** Add an option to enable postgres ssl with default certs
([#6889](#6889))
([789e1e7](789e1e7))
* **core:** Fix error on missing paired item data
([#7399](#7399))
([47e8953](47e8953))
* **core:** Missing pairing info
([#7326](#7326))
([e2c3c7a](e2c3c7a))
* **core:** Prevent object deletion request on no prefix match
([#7366](#7366))
([63e11e4](63e11e4))
* **editor:** Fix completions for `.json` on quoted node name in Code
node ([#7382](#7382))
([86e7ec7](86e7ec7))
* **editor:** Implement canvas zoom UX improvements
([#7376](#7376))
([7e06b31](7e06b31))
* **editor:** Make workflow history button available only for dev builds
([#7392](#7392))
([7ed466d](7ed466d))
* **editor:** Remove excess margin below run data editor
([#7372](#7372))
([3fa2764](3fa2764))
* **editor:** Sanitize HTML binary-data before rendering in the UI
([#7400](#7400))
([2b075bf](2b075bf))
* **editor:** Use display option's @Version specifier
([#7351](#7351))
([afbf0c3](afbf0c3))
* **Google BigQuery Node:** Location default to jobReference
([#7354](#7354))
([97bb703](97bb703))
* **Google Drive Trigger Node:** Add Shared Drives support
([#7369](#7369))
([3e7a4d3](3e7a4d3))
* **Google Sheets Node:** Fix "Maximum call stack size exceeded" error
on too many rows ([#7384](#7384))
([732b15a](732b15a))
* **HTML Node:** Update property fields to not use expressions on drag
([#7379](#7379))
([77643e5](77643e5))
* **Notion Node:** Handle empty values correctly for Notion selects +
multi selects ([#7383](#7383))
([fbcd1d4](fbcd1d4))
* **Set Node:** Increase search priority
([#7358](#7358))
([e5ad1e7](e5ad1e7))
* **Webhook Node:** Backward compatible form-data parsing for non-array
files ([#7385](#7385))
([6479eb1](6479eb1))


### Features

* **core:** Add Job Summary to Worker response
([#7360](#7360))
([b8608ce](b8608ce))
* **core:** Integrate object store as binary data manager
([#7253](#7253))
([1a661e6](1a661e6))
* **core:** Switch binary filesystem mode to nested path structure
([#7307](#7307))
([0847623](0847623))
* **editor:** Make PDF and Audio binary-data viewable in the UI
([#7367](#7367))
([8187be1](8187be1))
* **editor:** Support autologin for upgrade path
([#7316](#7316))
([1dfa052](1dfa052))
* **Execute Workflow Node:** Run once for each item mode
([#7289](#7289))
([c8c14ca](c8c14ca))
* **Item Lists Node:** Split merge binary data
([#7297](#7297))
([965db8f](965db8f))
* **Loop Over Items (Split in Batches) Node:** Automatically add a loop
+ rename ([#7228](#7228))
([7b773cc](7b773cc))
* **Notion Node:** Fetch child blocks recursively
([#7304](#7304))
([193181a](193181a))

Co-authored-by: netroy <[email protected]>
@janober
Copy link
Member

janober commented Oct 11, 2023

Got released with [email protected]

elsmr pushed a commit that referenced this pull request Oct 19, 2023
# [1.11.0](https://github.com/n8n-io/n8n/compare/[email protected]@1.11.0)
(2023-10-11)


### Bug Fixes

* **core:** Add an option to enable postgres ssl with default certs
([#6889](#6889))
([789e1e7](789e1e7))
* **core:** Fix error on missing paired item data
([#7399](#7399))
([47e8953](47e8953))
* **core:** Missing pairing info
([#7326](#7326))
([e2c3c7a](e2c3c7a))
* **core:** Prevent object deletion request on no prefix match
([#7366](#7366))
([63e11e4](63e11e4))
* **editor:** Fix completions for `.json` on quoted node name in Code
node ([#7382](#7382))
([86e7ec7](86e7ec7))
* **editor:** Implement canvas zoom UX improvements
([#7376](#7376))
([7e06b31](7e06b31))
* **editor:** Make workflow history button available only for dev builds
([#7392](#7392))
([7ed466d](7ed466d))
* **editor:** Remove excess margin below run data editor
([#7372](#7372))
([3fa2764](3fa2764))
* **editor:** Sanitize HTML binary-data before rendering in the UI
([#7400](#7400))
([2b075bf](2b075bf))
* **editor:** Use display option's @Version specifier
([#7351](#7351))
([afbf0c3](afbf0c3))
* **Google BigQuery Node:** Location default to jobReference
([#7354](#7354))
([97bb703](97bb703))
* **Google Drive Trigger Node:** Add Shared Drives support
([#7369](#7369))
([3e7a4d3](3e7a4d3))
* **Google Sheets Node:** Fix "Maximum call stack size exceeded" error
on too many rows ([#7384](#7384))
([732b15a](732b15a))
* **HTML Node:** Update property fields to not use expressions on drag
([#7379](#7379))
([77643e5](77643e5))
* **Notion Node:** Handle empty values correctly for Notion selects +
multi selects ([#7383](#7383))
([fbcd1d4](fbcd1d4))
* **Set Node:** Increase search priority
([#7358](#7358))
([e5ad1e7](e5ad1e7))
* **Webhook Node:** Backward compatible form-data parsing for non-array
files ([#7385](#7385))
([6479eb1](6479eb1))


### Features

* **core:** Add Job Summary to Worker response
([#7360](#7360))
([b8608ce](b8608ce))
* **core:** Integrate object store as binary data manager
([#7253](#7253))
([1a661e6](1a661e6))
* **core:** Switch binary filesystem mode to nested path structure
([#7307](#7307))
([0847623](0847623))
* **editor:** Make PDF and Audio binary-data viewable in the UI
([#7367](#7367))
([8187be1](8187be1))
* **editor:** Support autologin for upgrade path
([#7316](#7316))
([1dfa052](1dfa052))
* **Execute Workflow Node:** Run once for each item mode
([#7289](#7289))
([c8c14ca](c8c14ca))
* **Item Lists Node:** Split merge binary data
([#7297](#7297))
([965db8f](965db8f))
* **Loop Over Items (Split in Batches) Node:** Automatically add a loop
+ rename ([#7228](#7228))
([7b773cc](7b773cc))
* **Notion Node:** Fetch child blocks recursively
([#7304](#7304))
([193181a](193181a))

Co-authored-by: netroy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants