Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(core): Flatten Redis pubsub class hierarchy (no-changelog) #10616

Merged
merged 22 commits into from
Sep 17, 2024

Conversation

ivov
Copy link
Contributor

@ivov ivov commented Aug 29, 2024

To set up a publisher and subscriber for scaling mode, we currently rely on six interrelated classes: RedisService, RedisServiceBase, RedisServiceBaseSender, RedisServiceBaseReceiver, RedisServicePubSubPublisher, RedisServicePubSubSubscriber

This PR simplifies this six-class setup into two small classes Publisher and Subscriber, tightens typings, adds documentation, removes dead code, and deletes all six original classes.

Further improvements are marked as TODOs, outside the scope of this PR.

Follow-up to: #10566

@n8n-assistant n8n-assistant bot added core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team labels Aug 29, 2024
@ivov ivov force-pushed the flatten-redis-hierarchy branch from e88ea1f to 15a2a16 Compare September 16, 2024 08:18
@ivov ivov marked this pull request as ready for review September 16, 2024 08:59
Comment on lines +33 to +35
getClient() {
return this.client;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to expose the client publicly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For testing only, it's an open discussion on what's best practice for this.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I keep wavering between the both options. On one hand it's nice to test certain things that require private parts, but then again tests should test the public interface or it's too tied to implementation details 🤷 No need to change this now

packages/cli/src/scaling/pubsub/publisher.service.ts Outdated Show resolved Hide resolved
packages/cli/src/scaling/pubsub/publisher.service.ts Outdated Show resolved Hide resolved
packages/cli/src/scaling/pubsub/subscriber.service.ts Outdated Show resolved Hide resolved
@@ -29,7 +17,6 @@ export abstract class OrchestrationHandlerService {
}

async shutdown() {
await this.redisSubscriber?.destroy();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Who takes care of destroying the subscriber now?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here:

// @TODO: Use `@OnShutdown()` decorator
async shutdown() {
if (!this.isInitialized) return;
if (this.isMultiMainSetupEnabled) await this.multiMainSetup.shutdown();
this.publisher.shutdown();
this.subscriber.shutdown();
this.isInitialized = false;
}

@ivov ivov requested a review from tomi September 17, 2024 09:16
@ivov
Copy link
Contributor Author

ivov commented Sep 17, 2024

Once there are no more code changes, I'll do another round of manual testing.

tomi
tomi previously approved these changes Sep 17, 2024
Copy link
Contributor

@tomi tomi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job, much cleaner now 💯

Copy link

cypress bot commented Sep 17, 2024

n8n    Run #6914

Run Properties:  status check passed Passed #6914  •  git commit 6b17bd095f: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 ivov 🗃️ e2e/*
Project n8n
Branch Review flatten-redis-hierarchy
Run status status check passed Passed #6914
Run duration 04m 39s
Commit git commit 6b17bd095f: 🌳 🖥️ browsers:node18.12.0-chrome107 🤖 ivov 🗃️ e2e/*
Committer Iván Ovejero
View all properties for this run ↗︎

Test results
Tests that failed  Failures 0
Tests that were flaky  Flaky 0
Tests that did not run due to a developer annotating a test with .skip  Pending 0
Tests that did not run due to a failure in a mocha hook  Skipped 0
Tests that passed  Passing 430
View all changes introduced in this branch ↗︎

Copy link
Contributor

✅ All Cypress E2E specs passed

@ivov
Copy link
Contributor Author

ivov commented Sep 17, 2024

Retested manually that these are still sent and received:

  • reloadLicense
  • reloadExternalSecrets
  • restartEventBus
  • community-package-install
  • community-package-uninstall

@tomi If you can please reapprove 🙏🏻

@tomi
Copy link
Contributor

tomi commented Sep 17, 2024

@tomi If you can please reapprove 🙏🏻

Will do 👍 You can use the re-request review to notify me:

image

Copy link
Contributor

✅ All Cypress E2E specs passed

@ivov ivov merged commit aa00d9c into master Sep 17, 2024
33 checks passed
@ivov ivov deleted the flatten-redis-hierarchy branch September 17, 2024 13:45
ivov added a commit that referenced this pull request Sep 18, 2024
- Move `/redis` under scaling mode
- Move channel constants to `scaling/constants.ts`
- Delete `redis-service-base-classes.ts` - unused since #10616
- Delete `WorkerJobStatusSummary` - unused
- Move pubsub map to `pubsub.types.ts`
- Rename `PubSubMessage` to `JobReport` for consistency
@janober
Copy link
Member

janober commented Sep 18, 2024

Got released with [email protected]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Enhancement outside /nodes-base and /editor-ui n8n team Authored by the n8n team Released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants