Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(NODE-6337): implement client bulk write batching #4248

Merged
merged 11 commits into from
Oct 1, 2024

Conversation

durran
Copy link
Member

@durran durran commented Sep 25, 2024

Description

Implements batching of client bulk writes.

What is changing?

  • Updates DocumentSequence to be able to serialize itself and return its documents byte length when pushing to it.
  • Updates the command builder to batch bulk write commands based on the max message size or max batch size.
  • Updates the result merger to properly link indexes back to the original user provided models.
  • Exposes maxMessageSizeBytes and maxWriteBatchSize up to the topology level.
  • Implements CRUD prose tests #3 and #4
Is there new documentation needed for these changes?

None

What is the motivation for this change?

NODE-6337

Double check the following

  • Ran npm run check:lint script
  • Self-review completed using the steps outlined here
  • PR title follows the correct format: type(NODE-xxxx)[!]: description
    • Example: feat(NODE-1234)!: rewriting everything in coffeescript
  • Changes are covered by tests
  • New TODOs have a related JIRA ticket

@durran durran force-pushed the NODE-6337-bulk-write-batch branch from 35e62b7 to b8fcf14 Compare September 25, 2024 15:10
@durran durran force-pushed the NODE-6337-bulk-write-batch branch 3 times, most recently from 0c4de74 to 9f63397 Compare September 25, 2024 23:20
@durran durran marked this pull request as ready for review September 26, 2024 14:07
@baileympearson baileympearson self-assigned this Sep 26, 2024
@baileympearson baileympearson added the Primary Review In Review with primary reviewer, not yet ready for team's eyes label Sep 26, 2024
@baileympearson baileympearson self-requested a review September 26, 2024 14:11
src/sdam/server_description.ts Outdated Show resolved Hide resolved
src/operations/client_bulk_write/executor.ts Outdated Show resolved Hide resolved
src/cmap/commands.ts Show resolved Hide resolved
src/cmap/commands.ts Outdated Show resolved Hide resolved
src/operations/client_bulk_write/command_builder.ts Outdated Show resolved Hide resolved
src/cmap/commands.ts Outdated Show resolved Hide resolved
src/operations/client_bulk_write/command_builder.ts Outdated Show resolved Hide resolved
@durran durran force-pushed the NODE-6337-bulk-write-batch branch from d4d4bc6 to 1951d97 Compare September 30, 2024 20:57
@durran durran force-pushed the NODE-6337-bulk-write-batch branch from 1951d97 to 0ed7ce8 Compare September 30, 2024 21:06
Copy link
Contributor

@baileympearson baileympearson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small changes!

src/operations/client_bulk_write/client_bulk_write.ts Outdated Show resolved Hide resolved
src/sdam/server_description.ts Outdated Show resolved Hide resolved
@durran durran requested a review from baileympearson October 1, 2024 14:42
@baileympearson baileympearson merged commit d56e235 into main Oct 1, 2024
22 of 25 checks passed
@baileympearson baileympearson deleted the NODE-6337-bulk-write-batch branch October 1, 2024 19:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Primary Review In Review with primary reviewer, not yet ready for team's eyes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants