Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CommandProcessor Bulk Clear #1764 #1927

Merged
merged 5 commits into from
Feb 7, 2022
Merged

CommandProcessor Bulk Clear #1764 #1927

merged 5 commits into from
Feb 7, 2022

Conversation

preardon
Copy link
Member

This should allow the Command Processor to Bulk Clear from the Outbox and send Messages to Transports in Batches

@iancooper I would appreciate you're thoughts on this when you have a second before I get too much deeper, however I understand that this is for v9.next

@iancooper
Copy link
Member

This looks fairly solid and I don't have any issues with how this is working. We may want to make sure that a publisher confirm works with this, but that would mean we would need to update the transport for RMQ etc, and I am fine with doing that after you get ASB in

@preardon
Copy link
Member Author

@iancooper thanks for this, I will continue down this path and get a PR in

@preardon preardon force-pushed the 1764-BulkOperations branch 2 times, most recently from a6c80a1 to 8feaf0a Compare February 6, 2022 12:56
@preardon
Bulk clear proof of Concept
c21c4d9 
  - Add Bulk Get, and Bulk Mark as dispatched to Outbox
  - Add Bulk Clear to Command Processor
  - Add IAmABulkMessageProdcerAsync Interface
@preardon
Updated Outboxes for Bulk
b02e5bd 
  - Updated MsSql, MySql, and Sqlite to Support the new Async contract
  -  DynamoDb and EventStore were updated in a minimal way as it doesn't appear that these are being used Around them
@preardon
Updated Sweeper to use Bulk if available
383544b
@preardon
Fix renamed file
a027427
@preardon preardon marked this pull request as ready for review February 6, 2022 15:43
@preardon
Copy link
Member Author

preardon commented Feb 6, 2022 •
edited
Loading

This Adds the following Functionality

  • Outbox - Get Multiple by ID
  • Outbox Mark Multiple as Dispatched by ID
  • Command Processor - Bulk Clear
  • New Interface for Producers that support Bulk
  • Outbox Sweeper - Ability to set to Use Bulk

There is not yet any support for Synchronous Bulk due to the IAsyncEnumerable being used for Async, I could bring the logic back into the ExternalBus service if you wanted me to support this, please let me know and I'll get it in ASAP

@iancooper this should be a MVP to allow ASB to dispatch as Bulk, let me know what you think

iancooper
iancooper approved these changes Feb 7, 2022
View reviewed changes
Copy link
Member

@iancooper iancooper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this works. What we might be able to look at is the same model we use elsewhere - launch a thread to run the bulk update and use a mutex to prevent re-entry if a bulk clear is already running. But I can point you in the direction of our prior art there.

@preardon preardon merged commit ff6d314 into BrighterCommand:master Feb 7, 2022
@preardon preardon deleted the 1764-BulkOperations branch February 7, 2022 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@iancooper iancooper iancooper approved these changes

@DevJonny DevJonny Awaiting requested review from DevJonny DevJonny is a code owner

@holytshirt holytshirt Awaiting requested review from holytshirt holytshirt is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@preardon @iancooper