feat(NODE-6390): Add timeoutMS support to auto encryption #4265

aditi-khare-mongoDB · 2024-10-02T21:30:00Z

Description

Auto encryption encrypt and decrypt helpers now respect timeoutMS.

What is changing?

Added timeoutContext to stateMachine.execute call in AutoEncrypter.encrypt/decrypt
Integration tests for StateMachine helpers respecting timeoutMS
Integration tests for CryptoConnection.command respecting timeoutMS
Spec unit test for Auto Encryption

Is there new documentation needed for these changes?

No

What is the motivation for this change?

CSOT

Double check the following

Ran npm run check:lint script
Self-review completed using the steps outlined here
PR title follows the correct format: type(NODE-xxxx)[!]: description
- Example: feat(NODE-1234)!: rewriting everything in coffeescript
Changes are covered by tests
New TODOs have a related JIRA ticket

…ver selection

#4157)

Co-authored-by: Warren James <[email protected]>

…4186)

…#4237)

…ver selection

#4157)

Co-authored-by: Warren James <[email protected]>

…4186)

…#4237)

lint fix prose test 2

Co-authored-by: Warren James <[email protected]>

baileympearson · 2024-10-21T21:37:35Z

test/integration/client-side-encryption/driver.test.ts

+    const metadata: MongoDBMetadataUI = {
+      requires: {
+        mongodb: '>=4.2.0',
+        clientSideEncryption: '>=6.1.0'


Suggested change

clientSideEncryption: '>=6.1.0'

mongodb-client-encryption 6.1.0 shouldn't be required for these tests

Removed it!

baileympearson

Just a few small comments!

baileympearson · 2024-10-21T21:38:24Z

test/integration/client-side-encryption/driver.test.ts

+      });
+    });
+
+    context('when client is not provided timeoutMS and command hangs', function () {


Suggested change

context('when client is not provided timeoutMS and command hangs', function () {

context('when client is not provided timeoutMS and auto encryption is delayed', function () {

Made the change! - editted to match the test name at line 545 since they test the same scenario - just with and without timeoutMS

test/integration/client-side-encryption/driver.test.ts

baileympearson · 2024-10-21T21:41:51Z

test/integration/client-side-encryption/driver.test.ts

+        });
+
+        it(
+          'the command should not fail due to a server error',


Suggested change

'the command should not fail due to a server error',

'the command succeeds',

good catch!

baileympearson · 2024-10-21T21:42:37Z

test/integration/client-side-operations-timeout/client_side_operations_timeout.prose.test.ts

        client = new MongoClient(`mongodb://localhost:${mongocryptdTestPort}/?timeoutMS=1000`, {
-          monitorCommands: true
+          family: 6,
+          monitorCommands: true,
+          serverSelectionTimeoutMS: 2000


Looks like they're still there

baileympearson · 2024-10-22T16:29:49Z

test/integration/client-side-encryption/driver.test.ts

+    };
+
+    context('when client is provided timeoutMS and command hangs', function () {
+      let encryptedClient: MongoClient;


Can we make the test titles more precise? command hangs is a bit vague - specifically we're concerned with autoEncryption timing out so the test should make that clear.

Suggested change

let encryptedClient: MongoClient;

context('when an auto encrypted client is configured with timeoutMS and autoEncryption takes longer than timeoutMS', function () {

it('the command times out', function() {} )

Yep, I can make the title more specific!

baileympearson · 2024-10-22T16:32:05Z

test/integration/client-side-encryption/driver.test.ts

+      });
+
+      afterEach(async function () {
+        encryptedClient.close();


Suggested change

encryptedClient.close();

await encryptedClient.close();

good catch, my bad!

baileympearson · 2024-10-22T16:35:29Z

test/integration/client-side-encryption/driver.test.ts

+    };
+
+    describe('#markCommand', function () {
+      context('when provided timeoutContext and command hangs', function () {


Suggested change

context('when provided timeoutContext and command hangs', function () {

context('when csot is enabled and markCommand() takes longer than the remaining timeoutMS', function() {

we always provide a timeoutContext to markCommand() (and other state machine helper functions that we test here, so this comment is relevant for listCollections and getKeys). also, let's be precise with the exact scenario we're testing here - that the command fails if it takes longer than the remaining timeoutMS

sure i can change it. my original thinking is that timeoutContext is only registered when it's non-nullish but ig also in the case that the timeoutContext is non-csot.

baileympearson · 2024-10-22T16:37:45Z

test/integration/client-side-encryption/driver.test.ts

+        const err = await encryptedClient
+          .db('test')
+          .collection('test')
+          .insertOne({ a: 1 })
+          .catch(e => e);
+        expect(err).to.be.instanceOf(MongoOperationTimeoutError);


We realized last week that it is important to measure the duration of the command when we expect it to time out. If we expect it to time out in timeoutMS, then we need to confirm it actually took no more than timeoutMS to complete.

There's some flexibility here, because CSOT isn't a hard gaurantee, but in other tests we've measured the duration (there's a measureDuration helper) and we assert its within a range of timeoutMS. Can we do the same here?

This applies to other places we're testing CSOT too

added it in to all 5 csot enabled testing cases from this PR

baileympearson · 2024-10-22T16:38:33Z

test/integration/client-side-encryption/driver.test.ts

+        await setupClient.close();
+      });
+
+      context('when provided timeoutContext and command hangs', function () {


same comment as markCommand's describe block

Co-authored-by: Warren James <[email protected]> Co-authored-by: Neal Beeken <[email protected]> Co-authored-by: Bailey Pearson <[email protected]>

W-A-James and others added 30 commits September 12, 2024 13:43

feat(NODE-6090): Implement CSOT logic for connection checkout and ser…

3c2ec0a

…ver selection

test(NODE-6120): Implement Unified test runner changes for CSOT (#4121)

909578f

refactor(NODE-6187): refactor to use TimeoutContext abstraction (#4131)

e101750

refactor(NODE-6230): executeOperation to use iterative retry mechanism (

e4efd3f

#4157)

feat(NODE-5682): set maxTimeMS on commands and preempt I/O (#4174)

22082c9

Co-authored-by: Warren James <[email protected]>

feat(NODE-6231): Add CSOT behaviour for retryable reads and writes (#…

bf95fa4

…4186)

feat(NODE-6312): add error transformation for server timeouts (#4192)

c63d102

feat(NODE-6313): add CSOT support to sessions and transactions (#4199)

1eab23d

feat(NODE-6304): add CSOT support for non-tailable cursors (#4195)

4c4b0a9

fix(NODE-6374): MongoOperationTimeoutError inherits MongoRuntimeError (…

558d416

…#4237)

test: remove empty skipped context blocks (#4238)

3ed4a14

feat(NODE-5844): add iscryptd to ServerDescription (#4239)

d3438ea

temp

ff561e3

temp

164780c

chore: plumb timeoutMS around more

12a7e2e

feat(NODE-6090): Implement CSOT logic for connection checkout and ser…

999f23d

…ver selection

test(NODE-6120): Implement Unified test runner changes for CSOT (#4121)

0355404

refactor(NODE-6187): refactor to use TimeoutContext abstraction (#4131)

5ef3d69

refactor(NODE-6230): executeOperation to use iterative retry mechanism (

7139b8f

#4157)

feat(NODE-5682): set maxTimeMS on commands and preempt I/O (#4174)

acfb4fc

Co-authored-by: Warren James <[email protected]>

feat(NODE-6231): Add CSOT behaviour for retryable reads and writes (#…

4efff95

…4186)

feat(NODE-6312): add error transformation for server timeouts (#4192)

1997f81

feat(NODE-6313): add CSOT support to sessions and transactions (#4199)

cc3ef8f

feat(NODE-6304): add CSOT support for non-tailable cursors (#4195)

38affae

fix(NODE-6374): MongoOperationTimeoutError inherits MongoRuntimeError (…

738188b

…#4237)

test: remove empty skipped context blocks (#4238)

c4a7c2c

feat(NODE-5844): add iscryptd to ServerDescription (#4239)

5aa6d4c

chore: allow clientBulkWrite to use TimeoutContext (#4251)

17a2fde

Merge branch 'NODE-6090' into NODE-6389

aead2f1

half testing

88ca990

lint fix prose test 2

aditi-khare-mongoDB and others added 8 commits October 16, 2024 17:35

requested changes

f2572c9

lint

0dab0ca

requested changes

2a6a2d1

drop collection

9530160

test

2088e63

feat(NODE-6403): add CSOT support to client bulk write (#4261)

313eaa0

Co-authored-by: Warren James <[email protected]>

chore: fix a few flaky CSOT tests (#4278)

07cffc7

feat(NODE-6421): add support for timeoutMS to explain helpers (#4268)

c3f31da

baileympearson force-pushed the NODE-6090 branch from d224fff to c3f31da Compare October 21, 2024 15:09

aditi-khare-mongoDB and others added 2 commits October 21, 2024 11:27

Merge branch 'NODE-6090' into NODE-6390/auto-encryption-csot

b37eb63

added schmea map

5b1164f

aditi-khare-mongoDB requested a review from baileympearson October 21, 2024 20:20

baileympearson force-pushed the NODE-6090 branch from c3f31da to 4fd4b24 Compare October 21, 2024 20:52

Merge branch 'NODE-6090' into NODE-6390/auto-encryption-csot

186c059

baileympearson reviewed Oct 21, 2024

View reviewed changes

baileympearson requested changes Oct 21, 2024

View reviewed changes

baileympearson requested changes Oct 22, 2024

View reviewed changes

aditi-khare-mongoDB added 3 commits October 22, 2024 15:28

requested changes

aa9a077

consolidate meta data

df6bb05

fixed failing test

fec8887

aditi-khare-mongoDB requested a review from baileympearson October 23, 2024 20:38

aditi-khare-mongoDB and others added 2 commits October 24, 2024 11:25

lint fix

a0ebd93

Merge branch 'NODE-6090' into NODE-6390/auto-encryption-csot

b56ced1

baileympearson approved these changes Oct 25, 2024

View reviewed changes

baileympearson merged commit bc05c4b into NODE-6090 Oct 25, 2024
18 of 27 checks passed

baileympearson deleted the NODE-6390/auto-encryption-csot branch October 25, 2024 17:01

nbbeeken added a commit that referenced this pull request Nov 1, 2024

feat(NODE-6390): Add timeoutMS support to auto encryption (#4265)

656bb01

Co-authored-by: Warren James <[email protected]> Co-authored-by: Neal Beeken <[email protected]> Co-authored-by: Bailey Pearson <[email protected]>

nbbeeken added a commit that referenced this pull request Nov 5, 2024

feat(NODE-6390): Add timeoutMS support to auto encryption (#4265)

3054eaf

Co-authored-by: Warren James <[email protected]> Co-authored-by: Neal Beeken <[email protected]> Co-authored-by: Bailey Pearson <[email protected]>

dariakp pushed a commit that referenced this pull request Nov 6, 2024

feat(NODE-6390): Add timeoutMS support to auto encryption (#4265)

55e08e7

Co-authored-by: Warren James <[email protected]> Co-authored-by: Neal Beeken <[email protected]> Co-authored-by: Bailey Pearson <[email protected]>

github-actions bot mentioned this pull request Nov 6, 2024

chore(main): release 6.11.0 [skip-ci] #4317

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(NODE-6390): Add timeoutMS support to auto encryption #4265

feat(NODE-6390): Add timeoutMS support to auto encryption #4265

aditi-khare-mongoDB commented Oct 2, 2024 •

edited by nbbeeken

Loading

baileympearson Oct 21, 2024

aditi-khare-mongoDB Oct 22, 2024

baileympearson left a comment

baileympearson Oct 21, 2024 •

edited

Loading

aditi-khare-mongoDB Oct 22, 2024 •

edited

Loading

baileympearson Oct 21, 2024

aditi-khare-mongoDB Oct 22, 2024

baileympearson Oct 21, 2024

baileympearson Oct 22, 2024

aditi-khare-mongoDB Oct 22, 2024

baileympearson Oct 22, 2024

aditi-khare-mongoDB Oct 22, 2024

baileympearson Oct 22, 2024

aditi-khare-mongoDB Oct 22, 2024

baileympearson Oct 22, 2024

baileympearson Oct 22, 2024

aditi-khare-mongoDB Oct 22, 2024

baileympearson Oct 22, 2024

	context('when client is not provided timeoutMS and command hangs', function () {
	context('when client is not provided timeoutMS and auto encryption is delayed', function () {

	'the command should not fail due to a server error',
	'the command succeeds',

	let encryptedClient: MongoClient;
	context('when an auto encrypted client is configured with timeoutMS and autoEncryption takes longer than timeoutMS', function () {
	it('the command times out', function() {} )

	context('when provided timeoutContext and command hangs', function () {
	context('when csot is enabled and markCommand() takes longer than the remaining timeoutMS', function() {

feat(NODE-6390): Add timeoutMS support to auto encryption #4265

feat(NODE-6390): Add timeoutMS support to auto encryption #4265

Conversation

aditi-khare-mongoDB commented Oct 2, 2024 • edited by nbbeeken Loading

Description

What is changing?

Is there new documentation needed for these changes?

What is the motivation for this change?

Double check the following

Choose a reason for hiding this comment

Choose a reason for hiding this comment

baileympearson left a comment

Choose a reason for hiding this comment

baileympearson Oct 21, 2024 • edited Loading

Choose a reason for hiding this comment

aditi-khare-mongoDB Oct 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aditi-khare-mongoDB commented Oct 2, 2024 •

edited by nbbeeken

Loading

baileympearson Oct 21, 2024 •

edited

Loading

aditi-khare-mongoDB Oct 22, 2024 •

edited

Loading