refactor!: only store topology on MongoClient #2594

HanaPearlman · 2020-10-21T20:38:08Z

Description

What changed?
A topology is no longer stored on Db or Collection; it's only on MongoClient. Each class only stores its parent and accesses the topology through that parent.

Methods that require a topology can now run into cases where the topology is undefined; an error is thrown during these cases.

This work is a step towards having a single, immutable topology on MongoClient which can never be null.

NODE-2850

test/unit/collection.test.js

src/operations/execute_operation.ts

reggi

Just one thought about adding getters / public API.

reggi · 2020-10-22T15:05:13Z

src/collection.ts

+  /**
+   * The current topology of the collection.
+   */
+  get topology(): Topology | undefined {


This here exposes topology as a top-level API, I'm generally OK with this, but it is something to acknowledge. We're usually conscious about what we allow as apart of the stable API. We can't make it private because we need to call it internally collection.topology. Calling the .s.properties is pretty annoying. Our users have always been able to call .s.topology but I don't think we consider that stable.

If we don't use getter's we can use deep properties this.s.client.s.db.s.topology 😢

Side thought may not need breaking change ! here?

I really like it, and I'd also be interested to add getters in all the places we're using topology / parent / db / client.

get db () { return this.s.db; } get topology () { return this.db.topology; }

This would help in that we can always just call this.topology from within a given class.

I think the getters idea is really good! Am happy to add that.

Hm, I'm not sure we should take this approach. My main question is: who is this feature helping? Do we expect users to need to refer to the topology or the db of a Collection, or are we adding something here that only helps us (the maintainers)?

For starters, I don't think we should make it easy to access the Topology type, since we want that to be private API that we can change at will. Leaving the topology in the s is unsavory, but at least let's us make decisions later about how we want to continue to shield users from accessing private implementation.

If it's too hard to stomach this.s.db.client.topology, we could consider a new getTopology util method. Given how boilerplate this all is, I'm not sure it's worth the effort to make it look clean though.

This is a good point. It seems like this would just be for the maintainers' benefit.

One thing to note is that a topology getter already exists on the Db class. Is this something we would consider removing?

For now I have gone with a getTopology util method on the Collection class. Since a topology field exists on MongoClient and a topology getter exists on Db, it's a bit inconsistent. To address this, I think we could remove the topology getter on Db and (potentially) replace it with a getTopology util.

src/collection.ts

src/utils.ts

emadum

LGTM 👍

nbbeeken

One 🧅 that's optional lgtm!

nbbeeken · 2020-11-04T19:39:28Z

src/sdam/topology.ts

@@ -57,6 +57,7 @@ import { DestroyOptions, Connection } from '../cmap/connection';
 import { RunCommandOperation } from '../operations/run_command';
 import type { CursorOptions } from '../cursor/cursor';
 import type { MongoClientOptions } from '../mongo_client';
+import { Db, Collection, MongoClient } from '..';


Suggested change

import { Db, Collection, MongoClient } from '..';

import { Db } from '../db';

import { Collection } from '../collection';

import { MongoClient } from '../mongo_client';

Just an 🧅 I think internally we should import from the file directly

+1, also you may run into issues with require "cycles" with this approach (noted below).

mbroadst · 2020-11-05T12:09:41Z

src/change_stream.ts

    } else {
      throw new TypeError(
        'parent provided to ChangeStream constructor is not an instance of Collection, Db, or MongoClient'
      );
    }

+    const topology = getTopology(parent);
+    if (!topology) throw new MongoClientClosedError();
+    this.topology = topology;


we should be able to use getTopology to remove the need to attach a topology property on ChangeStream as well

mbroadst · 2020-11-05T12:12:46Z

src/db.ts

   * @param options - Optional settings for Db construction
   */
-  constructor(databaseName: string, topology: Topology, options?: DbOptions) {
+  constructor(databaseName: string, client: MongoClient, options?: DbOptions) {


Suggested change

constructor(databaseName: string, client: MongoClient, options?: DbOptions) {

constructor(client: MongoClient, databaseName: string, options?: DbOptions) {

nit, but if we're changing this anyway, let's try to go from most fundamental element to most optional element in the parameter ordering (this mirrors how we store things in our private s object)

mbroadst · 2020-11-05T12:14:17Z

src/db.ts

@@ -194,8 +194,8 @@ export class Db implements OperationParent {
  }

  // Topology
-  get topology(): Topology {
-    return this.s.topology;
+  get topology(): Topology | undefined {


aren't we removing all topology properties here?

mbroadst · 2020-11-05T12:17:05Z

src/collection.ts

@@ -636,8 +646,10 @@ export class Collection implements OperationParent {
      throw new TypeError('`options` parameter must not be function');
    }

+    const topology = getTopology(this);
+    if (!topology) throw new MongoClientClosedError('Collection.prototype.find');


Rather than create a custom error message and throwing them in place, it looks like you can actually just move the error inside getTopology. getTopology should throw if it's unable to find the topology, with a MongoError which says that the client has been closed. Additionally, there is no need to include the name of the method in the error message string because you'll get that context in the stack trace anyway.

mbroadst · 2020-11-05T12:21:20Z

src/operations/execute_operation.ts

+  operation: T,
+  callback?: Callback<TResult>
+): Promise<TResult> | void {
+  if (!topology) throw new MongoClientClosedError();


getTopology should be changed to always return a Topology, not Topology | undefined - that method will throw if one is not found. Likewise, you can remove this topology == null check since we now validate the presence of a Topology in getTopology

mbroadst · 2020-11-05T12:23:20Z

test/functional/mongo_client.test.js

@@ -220,8 +220,8 @@ describe('MongoClient', function () {
      client.connect(function (err, client) {
        expect(err).to.not.exist;
        var db = client.db(configuration.db);
-        expect(db).nested.property('s.topology.s.options.connectTimeoutMS').to.equal(0);
-        expect(db).nested.property('s.topology.s.options.socketTimeoutMS').to.equal(0);
+        expect(db).nested.property('s.client.topology.s.options.connectTimeoutMS').to.equal(0);


reuse getTopology here

test/unit/collection.test.js

mbroadst · 2020-11-05T12:27:07Z

src/sdam/topology.ts

@@ -1053,3 +1054,10 @@ export class ServerCapabilities {
    return this.maxWireVersion >= 5;
  }
 }
+
+/** @internal */
+export function getTopology(provider: MongoClient | Db | Collection): Topology | undefined {


Another way to implement this is to use ducktyping. It would remove the need for having to import these types into the file directly, which will prevent cycles in module loading:

export function getTopology(provider: MongoClient | Db | Collection): Topology { if (`topology` in provider) return provider.topology; if ('s' in provider && 'client' in provider.s && 'topology' in provider.s.client) return provider.s.client.topology; ...

mbroadst · 2020-11-05T12:27:56Z

src/sdam/topology.ts

@@ -57,6 +57,7 @@ import { DestroyOptions, Connection } from '../cmap/connection';
 import { RunCommandOperation } from '../operations/run_command';
 import type { CursorOptions } from '../cursor/cursor';
 import type { MongoClientOptions } from '../mongo_client';
+import { Db, Collection, MongoClient } from '..';


+1, also you may run into issues with require "cycles" with this approach (noted below).

mbroadst · 2020-11-05T12:30:39Z

src/error.ts

+ * @public
+ * @category Error
+ */
+export class MongoClientClosedError extends MongoError {


It's uncommon for us to define a single-purpose error like this, and I think (as I said above) we can get away with moving the logic (and creation of the error) to one location inside getTopology. In the future I would generally say that single purpose errors may indicate an issue in code structure

mbroadst

👏 👏 very nice work here, LGTM

HanaPearlman added 6 commits October 21, 2020 14:23

only store topology on client, first pass

6c4982b

pull error up before execute op

e0cf415

add client closed error

42f7883

use new custom error

51929a5

pull up error from executeLegacyOperation

cc16e89

small fixes

631db82

HanaPearlman commented Oct 21, 2020

View reviewed changes

test/unit/collection.test.js Outdated Show resolved Hide resolved

push down error check to decorate with collation

147e78b

HanaPearlman commented Oct 21, 2020

View reviewed changes

src/operations/execute_operation.ts Outdated Show resolved Hide resolved

push error check to execute operation

7a50664

HanaPearlman force-pushed the NODE-2850/master/topology-on-client branch from 5797beb to 7a50664 Compare October 21, 2020 21:36

HanaPearlman marked this pull request as ready for review October 22, 2020 12:57

HanaPearlman requested review from mbroadst, nbbeeken, reggi and emadum October 22, 2020 12:58

reggi suggested changes Oct 22, 2020

View reviewed changes

HanaPearlman added 5 commits October 28, 2020 15:53

Merge branch 'master' into NODE-2850/master/topology-on-client

803e11c

do not add topology getter to collection

048c4b7

add some method names during client closed errors

642d882

be consistent with db topology access

416d3ce

two options inheritance fixes in Db

d2a1e75

mbroadst reviewed Nov 2, 2020

View reviewed changes

src/collection.ts Outdated Show resolved Hide resolved

create getTopology util

588cbec

HanaPearlman force-pushed the NODE-2850/master/topology-on-client branch from 749efd0 to 588cbec Compare November 2, 2020 15:31

HanaPearlman commented Nov 2, 2020

View reviewed changes

src/utils.ts Outdated Show resolved Hide resolved

HanaPearlman requested a review from mbroadst November 2, 2020 16:02

move getTopology to topology file

884c551

HanaPearlman force-pushed the NODE-2850/master/topology-on-client branch from 9844bb5 to 884c551 Compare November 3, 2020 14:03

emadum approved these changes Nov 3, 2020

View reviewed changes

nbbeeken approved these changes Nov 4, 2020

View reviewed changes

nbbeeken reviewed Nov 4, 2020

View reviewed changes

mbroadst suggested changes Nov 5, 2020

View reviewed changes

HanaPearlman force-pushed the NODE-2850/master/topology-on-client branch 2 times, most recently from c27692b to 80d7565 Compare November 5, 2020 16:20

HanaPearlman requested a review from mbroadst November 5, 2020 16:26

implement changes

e218676

HanaPearlman force-pushed the NODE-2850/master/topology-on-client branch from 80d7565 to e218676 Compare November 5, 2020 17:02

mbroadst approved these changes Nov 6, 2020

View reviewed changes

HanaPearlman merged commit 33fa6b2 into mongodb:master Nov 6, 2020

HanaPearlman deleted the NODE-2850/master/topology-on-client branch November 6, 2020 14:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor!: only store topology on MongoClient #2594

refactor!: only store topology on MongoClient #2594

HanaPearlman commented Oct 21, 2020 •

edited

Loading

reggi left a comment

reggi Oct 22, 2020 •

edited

Loading

HanaPearlman Oct 22, 2020

mbroadst Oct 22, 2020

HanaPearlman Oct 22, 2020

HanaPearlman Oct 28, 2020

emadum left a comment

nbbeeken left a comment

nbbeeken Nov 4, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst Nov 5, 2020

mbroadst left a comment

-import { Db, Collection, MongoClient } from '..';
+import { Db } from '../db';
+import { Collection } from '../collection';
+import { MongoClient } from '../mongo_client';

	constructor(databaseName: string, client: MongoClient, options?: DbOptions) {
	constructor(client: MongoClient, databaseName: string, options?: DbOptions) {

refactor!: only store topology on MongoClient #2594

refactor!: only store topology on MongoClient #2594

Conversation

HanaPearlman commented Oct 21, 2020 • edited Loading

Description

reggi left a comment

Choose a reason for hiding this comment

reggi Oct 22, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emadum left a comment

Choose a reason for hiding this comment

nbbeeken left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mbroadst left a comment

Choose a reason for hiding this comment

HanaPearlman commented Oct 21, 2020 •

edited

Loading

reggi Oct 22, 2020 •

edited

Loading