feat(core): Add support for ready hooks, and credentials overwrite endpoint in workers #6954

netroy · 2023-08-17T09:58:01Z

No description provided.

github-actions · 2023-08-17T09:58:29Z

Great PR! Please pay attention to the following items before merging:

Files matching packages/**:

If fixing bug, added test to cover scenario.
If addressing forum or Github issue, added link to description.

Files matching packages/**/*.ts:

Added unit tests to cover new or updated functionality.

Make sure to check off this list before asking for review.

krynble

LGTM, my only concern is that this code is located under the if (config.getEnv('queue.health.active')) block, and I think this should be separate, wdyt?

Also is config.getEnv('credentials.overwrite.endpoint') set by env variable or by the hooks file? If the latter, can we assure it has been executed at this point?

ahsanv · 2023-08-17T10:04:14Z

packages/cli/src/commands/worker.ts

-			});
+			let presetCredentialsLoaded = false;
+			const endpointPresetCredentials = config.getEnv('credentials.overwrite.endpoint');
+			if (endpointPresetCredentials !== '') {


its basically a copy of code here right? lgmt 👍

yeah. I'd like to refactor worker to also use AbstractServer, but that's a much larger change that I'd like to avoid in this PR.

netroy · 2023-08-17T10:08:43Z

LGTM, my only concern is that this code is located under the if (config.getEnv('queue.health.active')) block, and I think this should be separate, wdyt?

Without the if (config.getEnv('queue.health.active')) block, there is no http server to add the endpoint to. We should probably revisit this check, and update it to make more sense. but in this PR, I'd like to make the least amount of changes to get credentials overwrite endpoint working in workers.

Also is config.getEnv('credentials.overwrite.endpoint') set by env variable or by the hooks file? If the latter, can we assure it has been executed at this point?

It's set by env variables.

codecov · 2023-08-17T10:21:19Z

Codecov Report

Patch coverage has no change and project coverage change: -0.01% ⚠️

Comparison is base (832d087) 25.05% compared to head (a6501e7) 25.05%.
Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6954      +/-   ##
==========================================
- Coverage   25.05%   25.05%   -0.01%     
==========================================
  Files        3149     3149              
  Lines      191918   191933      +15     
  Branches    21111    21112       +1     
==========================================
- Hits        48090    48088       -2     
- Misses     142847   142864      +17     
  Partials      981      981

Files Changed	Coverage Δ
packages/cli/src/commands/worker.ts	`0.00% <0.00%> (ø)`

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

packages/cli/src/commands/worker.ts

…dpoint in workers (#6954)

github-actions · 2023-08-17T11:24:08Z

⚠️ Some Cypress E2E specs are failing, please fix them before merging

cypress · 2023-08-17T11:24:12Z

Passing run #1910 ↗︎

0	226	0	0	0

Details:

🌳 CredentialsOverwrites-in-workers 🖥️ browsers:node18.12.0-chrome107 🤖 netroy...
Project: n8n	Commit: `a6501e77b8`
Status: Passed	Duration: 07:56 💡
Started: Aug 17, 2023 1:31 PM	Ended: Aug 17, 2023 1:38 PM

_{This comment has been generated by cypress-bot as a result of this
project's GitHub integration settings.}

…dpoint in workers

github-actions · 2023-08-17T13:39:01Z

✅ All Cypress E2E specs passed

…dpoint in workers (#6954)

# [0.237.0](https://github.com/n8n-io/n8n/compare/[email protected]@0.237.0) (2023-08-17) ### Bug Fixes * **API:** Fix issue with workflow setting not supporting newer nanoids ([#6699](#6699)) ([2e64c9d](2e64c9d)) * **AwsS3 Node:** Fix issue if bucket name contains a '.' ([#6542](#6542)) ([219f897](219f897)) * **core:** Add missing primary key on the `execution_data` table on postgres ([#6797](#6797)) ([1c5f98d](1c5f98d)) * **core:** Fix `continueOnFail` for expression error in Set ([#6939](#6939)) ([2a3937f](2a3937f)) * **core:** Fix property existence checks on AugmentObject ([#6842](#6842)) ([3e4483a](3e4483a)) * **core:** Log crash causes to console when sentry is disabled ([#6890](#6890)) ([f19c9c1](f19c9c1)) * **core:** Restrict read/write file paths access ([#6582](#6582)) ([6e6b90e](6e6b90e)) * **core:** Serialize BigInts ([#6805](#6805)) ([17ce49b](17ce49b)) * **core:** Update frontend urls when using the `--tunnel` option ([#6898](#6898)) ([ec3c066](ec3c066)) * **core:** Update packages to address CVE-2023-2142 and CVE-2020-28469 ([#6844](#6844)) ([6dd0850](6dd0850)) * Correct typos in Taiga and ServiceNow nodes ([#6814](#6814)) ([b9f52ec](b9f52ec)) * **editor:** Fix code node’s content property to be reactive ([#6931](#6931)) ([f3b380d](f3b380d)) * **editor:** Prevent Code node linter from erroring on `null` parse ([#6934](#6934)) ([baada92](baada92)) * **Email Trigger (IMAP) Node:** UTF-8 attachments are not correctly named ([#6856](#6856)) ([f3f1c14](f3f1c14)) * **Email Trigger (IMAP) Node:** Fix connection issue with unexpected spaces in host ([#6886](#6886)) ([35c29ac](35c29ac)) * Fix issue with key based credentials not being read correctly ([#6824](#6824)) ([2459b0a](2459b0a)) * Fix issue with key formatting if null or undefined ([#6924](#6924)) ([59d33db](59d33db)) * Fix issue with key formatting introduced in 1.2.0 ([#6896](#6896)) ([8423153](8423153)) * **FTP Node:** List recursive ignore . and .. to prevent infinite loops ([#6707](#6707)) ([6a787dd](6a787dd)) * **GitLab Trigger Node:** Fix trigger activation 404 error ([#6711](#6711)) ([8c5145b](8c5145b)) * **Gmail Trigger Node:** Early returns in case of no data ([#6727](#6727)) ([df2f1d6](df2f1d6)) * **Google BigQuery Node:** Error description improvement ([#6715](#6715)) ([261d73e](261d73e)) * **GoToWebinar Node:** Fix issue with timezone incorrectly being required ([#6865](#6865)) ([2f61e34](2f61e34)) * **HTTP Request Node:** Improve error handling for TCP socket errors when `Continue On Fail` is enabled ([#6925](#6925)) ([e8e4c9e](e8e4c9e)) * **Lemlist Node:** Fix pagination issues with campaigns and activities ([#6734](#6734)) ([73eddc6](73eddc6)) * **Salesforce Node:** Fix Account update owner operation ([#6958](#6958)) ([e71ff76](e71ff76)) * **Telegram Trigger Node:** Add guard to 'include' call on null or undefined ([#6730](#6730)) ([c3c845c](c3c845c)) * **Todoist Node:** Fix issue with section id being ignored ([#6799](#6799)) ([e53f4a0](e53f4a0)) ### Features * Add support for not requiring SMTP auth with user management ([#3742](#3742)) ([fc97d18](fc97d18)) * **core:** Add support for ready hooks, and credentials overwrite endpoint in workers ([#6954](#6954)) ([e5986c5](e5986c5)) * **editor:** Add "Download" button if JSON data is to large ([#6850](#6850)) ([3d815cf](3d815cf)) * Enable parallel processing on multiple queue nodes ([#6295](#6295)) ([e45461a](e45461a)) Co-authored-by: netroy <[email protected]>

# [1.4.0](https://github.com/n8n-io/n8n/compare/[email protected]@1.4.0) (2023-08-23) ### Bug Fixes * **core:** Add recoveryInProgress flag file ([#6962](#6962)) ([7b96820](7b96820)) * **core:** Fix `continueOnFail` for expression error in Set ([#6939](#6939)) ([d4fac05](d4fac05)) * **core:** Fix `import:workflow` command ([#6996](#6996)) ([8c38d85](8c38d85)) * **core:** Replace throw with warning when deactivating a non-active workflow ([#6969](#6969)) ([b6a00fe](b6a00fe)) * **core:** Set up OAuth2 cred test ([#6960](#6960)) ([4fc69b7](4fc69b7)) * **editor:** Do not flag dynamic load options issue on expression ([#6932](#6932)) ([60a1ef0](60a1ef0)) * **editor:** Ensure community node install button tracks user agreement ([#6976](#6976)) ([0ddfc73](0ddfc73)) * **editor:** Fix parsing for single quoted resolvables ([#6982](#6982)) ([f32e993](f32e993)) * **editor:** Fix Remove all fields not removing values in resource mapper ([#6940](#6940)) ([e6cff3f](e6cff3f)) * **editor:** Prevent Code node linter from erroring on `null` parse ([#6934](#6934)) ([40d3a29](40d3a29)) * **Google Sheets Node:** Fix short sheet name interpreted as range ([#6989](#6989)) ([00268a0](00268a0)) * **Google Sheets Trigger Node:** Support sheet names with non-latin characters ([#6970](#6970)) ([052dd7c](052dd7c)) * **GraphQL Node:** Improve error handling ([#6955](#6955)) ([41db637](41db637)) * **Mautic Node:** Fix issue with owner not being set correctly ([#6991](#6991)) ([64b950f](64b950f)) * **Salesforce Node:** Fix Account update owner operation ([#6958](#6958)) ([9b27878](9b27878)) * **Shopify Node:** Fix pagination when using options ([#6972](#6972)) ([475d9c9](475d9c9)) * **Webhook Node:** Backward compatible form-data parsing for non-array fields ([#6967](#6967)) ([9455bcf](9455bcf)) ### Features * **core:** Add a warning to error workflows that cannot be started due to permission or settings ([#6961](#6961)) ([67b88f7](67b88f7)) * **core:** Add support for ready hooks, and credentials overwrite endpoint in workers ([#6954](#6954)) ([8f8a1de](8f8a1de)) * **editor:** Show banner for non-production licenses ([#6943](#6943)) ([413570c](413570c)) * Remove PostHog event calls ([#6915](#6915)) ([270946a](270946a)) * **Send Email Node:** Add support for sending text and html email simultaneously ([#6978](#6978)) ([3860d41](3860d41)) Co-authored-by: krynble <[email protected]>

janober · 2023-08-30T09:02:09Z

Got released with [email protected]

n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Aug 17, 2023

netroy force-pushed the CredentialsOverwrites-in-workers branch from d6b2d8e to 9b51e08 Compare August 17, 2023 10:02

krynble suggested changes Aug 17, 2023

View reviewed changes

ahsanv reviewed Aug 17, 2023

View reviewed changes

netroy commented Aug 17, 2023

View reviewed changes

packages/cli/src/commands/worker.ts Show resolved Hide resolved

netroy added a commit that referenced this pull request Aug 17, 2023

feat(core): Add support for ready hooks, and credentials overwrite en…

4ef6094

…dpoint in workers (#6954)

krynble approved these changes Aug 17, 2023

View reviewed changes

netroy added 3 commits August 17, 2023 13:26

feat(core): Add support for ready hooks, and credentials overwrite en…

b8922fc

…dpoint in workers

workers don't need to generate frontend types

859395d

add bodyParser

a6501e7

netroy force-pushed the CredentialsOverwrites-in-workers branch from fa8c59f to a6501e7 Compare August 17, 2023 11:26

netroy requested review from krynble and ahsanv August 17, 2023 11:26

ahsanv approved these changes Aug 17, 2023

View reviewed changes

netroy merged commit 8f8a1de into master Aug 17, 2023

netroy deleted the CredentialsOverwrites-in-workers branch August 17, 2023 13:42

netroy added a commit that referenced this pull request Aug 17, 2023

feat(core): Add support for ready hooks, and credentials overwrite en…

411258f

…dpoint in workers (#6954)

github-actions bot mentioned this pull request Aug 17, 2023

🚀 Release 0.236.4 #6963

Closed

netroy added a commit that referenced this pull request Aug 17, 2023

feat(core): Add support for ready hooks, and credentials overwrite en…

e5986c5

…dpoint in workers (#6954)

github-actions bot mentioned this pull request Aug 17, 2023

🚀 Release 0.237.0 #6965

Merged

This was referenced Aug 23, 2023

🚀 Release 1.4.0 #7000

Closed

🚀 Release 1.4.0 #7001

Merged

janober added the Released label Aug 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(core): Add support for ready hooks, and credentials overwrite endpoint in workers #6954

feat(core): Add support for ready hooks, and credentials overwrite endpoint in workers #6954

netroy commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

krynble left a comment

ahsanv Aug 17, 2023

netroy Aug 17, 2023

netroy commented Aug 17, 2023

codecov bot commented Aug 17, 2023 •

edited

Loading

github-actions bot commented Aug 17, 2023

cypress bot commented Aug 17, 2023 •

edited

Loading

github-actions bot commented Aug 17, 2023

janober commented Aug 30, 2023

feat(core): Add support for ready hooks, and credentials overwrite endpoint in workers #6954

feat(core): Add support for ready hooks, and credentials overwrite endpoint in workers #6954

Conversation

netroy commented Aug 17, 2023

github-actions bot commented Aug 17, 2023

krynble left a comment

Choose a reason for hiding this comment

ahsanv Aug 17, 2023

Choose a reason for hiding this comment

netroy Aug 17, 2023

Choose a reason for hiding this comment

netroy commented Aug 17, 2023

codecov bot commented Aug 17, 2023 • edited Loading

Codecov Report

github-actions bot commented Aug 17, 2023

cypress bot commented Aug 17, 2023 • edited Loading

Passing run #1910 ↗︎

github-actions bot commented Aug 17, 2023

janober commented Aug 30, 2023

codecov bot commented Aug 17, 2023 •

edited

Loading

cypress bot commented Aug 17, 2023 •

edited

Loading