DRIVERS-2789 Convert Server Selection Spec to Markdown (#1506)

* DRIVERS-2789 Convert Server Selection Spec to Markdown * fix links * fix link
mongodb · Feb 8, 2024 · bcde8cd · bcde8cd
1 parent a5e5641
commit bcde8cd
Show file tree

Hide file tree

Showing 19 changed files with 2,021 additions and 2,546 deletions.
diff --git a/markdown_link_config.json b/markdown_link_config.json
@@ -5,6 +5,9 @@
       },
       {
         "pattern": "^https://github.com/10gen/mongo-enterprise-modules"
+      },
+      {
+        "pattern": "^#"
       }
     ]
 }
diff --git a/source/change-streams/change-streams.rst b/source/change-streams/change-streams.rst
@@ -687,7 +687,7 @@ Drivers MUST raise an error on the first document received without a resume toke
 
 A change stream MUST attempt to resume a single time if it encounters any resumable error per `Resumable Error`_.  A change stream MUST NOT attempt to resume on any other type of error.
 
-In addition to tracking a resume token, change streams MUST also track the read preference specified when the change stream was created. In the event of a resumable error, a change stream MUST perform server selection with the original read preference using the `rules for server selection <https://github.com/mongodb/specifications/blob/master/source/server-selection/server-selection.rst#rules-for-server-selection>`_ before attempting to resume.
+In addition to tracking a resume token, change streams MUST also track the read preference specified when the change stream was created. In the event of a resumable error, a change stream MUST perform server selection with the original read preference using the `rules for server selection <../server-selection/server-selection.md#rules-for-server-selection>`_ before attempting to resume.
 
 Single Server Topologies
 ^^^^^^^^^^^^^^^^^^^^^^^^

diff --git a/source/client-side-encryption/client-side-encryption.rst b/source/client-side-encryption/client-side-encryption.rst
@@ -1810,7 +1810,7 @@ Connecting to mongocryptd_
 If the crypt_shared_ library is loaded, the driver MUST NOT attempt to connect
 to mongocryptd_. (Refer: `Detecting crypt_shared Availability`_).
 
-Single-threaded drivers MUST connect with `serverSelectionTryOnce=false <../server-selection/server-selection.rst#serverselectiontryonce>`_
+Single-threaded drivers MUST connect with `serverSelectionTryOnce=false <../server-selection/server-selection.md#serverselectiontryonce>`_
 , connectTimeoutMS=10000, and MUST bypass `cooldownMS <../server-discovery-and-monitoring/server-discovery-and-monitoring.rst#cooldownms>`__ when connecting to mongocryptd. See `Why are serverSelectionTryOnce and cooldownMS disabled for single-threaded drivers connecting to mongocryptd?`_.
 
 If the ClientEncryption is configured with mongocryptdBypassSpawn=true,
@@ -2481,7 +2481,7 @@ means server selection fails if a topology scan fails the first time (i.e. it
 will not make repeat attempts until serverSelectionTimeoutMS expires). This
 behavior is overridden since there may be a small delay between spawning
 mongocryptd (which the driver may be responsible for) and for mongocryptd to
-listen on sockets. See the Server Selection spec description of `serverSelectionTryOnce <../server-selection/server-selection.rst#serverselectiontryonce>`_.
+listen on sockets. See the Server Selection spec description of `serverSelectionTryOnce <../server-selection/server-selection.md#serverselectiontryonce>`_.
 
 Similarly, single threaded clients will by default wait for 5 second cooldown
 period after failing to connect to a server before making another attempt.

diff --git a/source/crud/crud.rst b/source/crud/crud.rst
@@ -2252,14 +2252,14 @@ assume the backing mongos is 5.0+), drivers MUST use the available read
 preference.
 
 Drivers SHOULD augment their
-`server selection algorithm <..../server-selection/server-selection.rst#server-selection-algorithm>`_
+`server selection algorithm <../server-selection/server-selection.md#server-selection-algorithm>`_
 such that this logic can be enforced within a single server selection attempt.
 
 Drivers MUST discern the read preference used to select a server for the
 operation, which SHALL be used for specifying the
 `$readPreference global command argument <../message/OP_MSG.rst#global-command-arguments>`_
 and
-`passing read preference to mongos and load balancers <../server-selection/server-selection.rst#passing-read-preference-to-mongos-and-load-balancers>`_
+`passing read preference to mongos and load balancers <../server-selection/server-selection.md#passing-read-preference-to-mongos-and-load-balancers>`_
 (if applicable).
 
 

diff --git a/source/find_getmore_killcursors_commands.rst b/source/find_getmore_killcursors_commands.rst
@@ -160,7 +160,7 @@ The **secondaryOk** flag SHOULD not be set for all follow-up **getMore** and **k
 
 More detailed information about the interaction of the **secondaryOk** with **OP_QUERY** can be found in the Server Selection Spec `Passing a Read Preference`_.
 
-.. _Passing a Read Preference: https://github.com/mongodb/specifications/blob/master/source/server-selection/server-selection.rst#passing-read-preference-to-mongos
+.. _Passing a Read Preference: ./server-selection/server-selection.md#passing-read-preference-to-mongos
 
 Behavior of Limit, skip and batchSize
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -511,7 +511,7 @@ This format is general for all commands when executing against a Mongos proxy.
 
 More in depth information about passing read preferences to Mongos can be found in the Server Selection Specification `Server Selection Specification`_.
 
-.. _Server Selection Specification: https://github.com/mongodb/specifications/blob/master/source/server-selection/server-selection.rst#passing-read-preference-to-mongos
+.. _Server Selection Specification: ./server-selection/server-selection.md#passing-read-preference-to-mongos
 
 Changelog
 =========

diff --git a/source/load-balancers/load-balancers.rst b/source/load-balancers/load-balancers.rst
@@ -158,7 +158,7 @@ For read and write operations, the single server in the topology MUST always be
 
 During command construction, the LoadBalancer server MUST be treated like a mongos and
 drivers MUST add a $readPreference field to the command when required by
-`Passing read preference to mongos and load balancers <../server-selection/server-selection.rst#passing-read-preference-to-mongos-and-load-balancers>`_.
+`Passing read preference to mongos and load balancers <../server-selection/server-selection.md#passing-read-preference-to-mongos-and-load-balancers>`_.
 
 
 Connection Pooling

diff --git a/source/logging/logging.rst b/source/logging/logging.rst
@@ -171,7 +171,7 @@ driver-specific messages they produce.
      - ``MONGODB_LOG_TOPOLOGY``
 
    * - serverSelection
-     - `Server Selection <../server-selection/server-selection.rst>`__
+     - `Server Selection <../server-selection/server-selection.md>`__
      - ``MONGODB_LOG_SERVER_SELECTION``
 
    * - connection

diff --git a/source/max-staleness/max-staleness.rst b/source/max-staleness/max-staleness.rst
@@ -481,7 +481,7 @@ Clients with minWireVersion < 5 MUST throw an error if maxStalenessSeconds is se
 and any available server in the topology has maxWireVersion less than 5.
 
 An available server is defined in the `Server Selection
-<../server-selection/server-selection.rst#terms>`_
+<../server-selection/server-selection.md#terms>`_
 specification.
 
 Servers began reporting lastWriteDate in wire protocol version 5,

diff --git a/source/run-command/run-command.rst b/source/run-command/run-command.rst
@@ -68,7 +68,7 @@ The following represents how a runCommand API SHOULD be exposed.
        *
        * @defaultValue ReadPreference(mode: primary)
        *
-       * @see https://github.com/mongodb/specifications/blob/master/source/server-selection/server-selection.rst#read-preference
+       * @see ../server-selection/server-selection.md#read-preference
        */
       readPreference?: ReadPreference;
 
@@ -113,7 +113,7 @@ ReadPreference
 For the purposes of server selection RunCommand MUST assume all commands are read operations.
 To facilitate server selection the RunCommand operation MUST accept an optional ``readPreference`` option.
 
-* See Server Selection's section on `Use of read preferences with commands <https://github.com/mongodb/specifications/blob/master/source/server-selection/server-selection.rst#use-of-read-preferences-with-commands>`_
+* See Server Selection's section on `Use of read preferences with commands <../server-selection/server-selection.md#use-of-read-preferences-with-commands>`_
 
 If the provided ReadPreference is NOT ``{mode: primary}`` and the selected server is NOT a standalone, the command sent MUST include the ``$readPreference`` global command argument.
 

diff --git a/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst b/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst
@@ -182,7 +182,7 @@ A server is judged "suitable" for an operation if the client can use it
 for a particular operation.
 For example, a write requires a standalone, primary, or mongos.
 Suitability is fully specified in the `Server Selection Spec
-<https://github.com/mongodb/specifications/blob/master/source/server-selection/server-selection.rst>`_.
+<../server-selection/server-selection.md>`_.
 
 address
 ```````

diff --git a/source/server-discovery-and-monitoring/server-monitoring.rst b/source/server-discovery-and-monitoring/server-monitoring.rst
@@ -1236,7 +1236,7 @@ Changelog
 
 .. Section for links.
 
-.. _Server Selection Spec: /source/server-selection/server-selection.rst
+.. _Server Selection Spec: ../server-selection/server-selection.md
 .. _main SDAM spec: server-discovery-and-monitoring.rst
 .. _Server Discovery And Monitoring: server-discovery-and-monitoring.rst
 .. _server API version: /source/versioned-api/versioned-api.rst
@@ -1247,7 +1247,7 @@ Changelog
 .. _updateRSFromPrimary: server-discovery-and-monitoring.rst#updateRSFromPrimary
 .. _Network error when reading or writing: server-discovery-and-monitoring.rst#network-error-when-reading-or-writing
 .. _connection handshake: mongodb-handshake/handshake.rst
-.. _localThresholdMS: /source/server-selection/server-selection.rst#localThresholdMS
+.. _localThresholdMS: ../server-selection/server-selection.md#localThresholdMS
 .. _SDAM Monitoring spec: server-discovery-and-monitoring-logging-and-monitoring.rst#heartbeats
 .. _OP_MSG Spec: /source/message/OP_MSG.rst
 .. _OP_MSG exhaustAllowed flag: /source/message/OP_MSG.rst#exhaustAllowed

diff --git a/source/server-selection/server-selection-tests.md b/source/server-selection/server-selection-tests.md
@@ -0,0 +1,250 @@
+# Server Selection Test Plan
+
+- Status: Accepted
+
+- Minimum Server Version: 2.4
+
+See also the YAML test files and their accompanying README in the "tests" directory.
+
+______________________________________________________________________
+
+## ReadPreference Document Validation
+
+While there are no YAML tests for this section, clients implementing this spec SHOULD perform validations on
+ReadPreference documents provided by the user. Specifically, documents with the following values should raise an error:
+
+- Mode PRIMARY and non-empty tag set
+
+## Calculating Round Trip Time
+
+Drivers implementing server selection MUST test that RTT values are calculated correctly. YAML tests for RTT
+calculations can be found in the "tests" directory and they test for correctness in the following scenarios:
+
+- first RTT: new average RTT equals measurement
+- subsequent measurements: new average RTT is calculated using the new measurement and the previous average as described
+  in the spec.
+
+Additionally, drivers SHOULD ensure that their implementations reject negative RTT values.
+
+Lastly, drivers SHOULD ensure that average RTT for a given ServerDescription is reset to 0 if that server is
+disconnected (ie a network error occurs during a `hello` or legacy hello call). Upon reconnect, the first new RTT value
+should become the average RTT for this server.
+
+The RTT tests are intentionally simplified to test the implementation of the EWMA algorithm without imposing any
+additional conditions on drivers that might affect architecture. For some drivers, RTT tests might require mocks; for
+others, it might just require unit tests.
+
+## Server Selection
+
+The following test cases can be found in YAML form in the "tests" directory. Each test lists a TopologyDescription
+representing a set of servers, a ReadPreference document, and sets of servers returned at various stages of the server
+selection process. These sets are described below. Note that it is not required to test for correctness at every step.
+
+| Test Case           | Description                                                                                                                                        |
+| ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `suitable_servers`  | the set of servers matching all server selection logic.                                                                                            |
+| `in_latency_window` | the subset of `suitable_servers` that falls within the allowable latency window (required). NOTE: tests use the default localThresholdMS of 15 ms. |
+
+Drivers implementing server selection MUST test that their implementations correctly return **one** of the servers in
+`in_latency_window`. Drivers SHOULD test against the full set of servers in `in_latency_window` and against
+`suitable_servers` if possible.
+
+### Topology Type Single
+
+- The single server is always selected.
+
+### Topology Type ReplicaSetNoPrimary
+
+**Reads**
+
+- PRIMARY
+  - no server selected
+- PRIMARY_PREFERRED
+  - Matching tags: select any eligible secondary
+  - Non-matching tags: no server selected
+- SECONDARY
+  - Matching tags: select any eligible secondary
+  - Non-matching tags: no server selected
+- SECONDARY_PREFERRED
+  - Matching tags: select any eligible secondary
+  - Non-matching tags: no server selected
+- NEAREST
+  - Matching tags: select any eligible secondary
+  - Non-matching tags: no server selected
+
+**Writes**
+
+- Writes must go to a primary, no server can be selected.
+
+### Topology Type ReplicaSetWithPrimary
+
+**Reads**
+
+- PRIMARY
+  - primary is selected **NOTE:** it is an error to provide tags with mode PRIMARY. See "ReadPreference Document
+    Validation."
+- PRIMARY_PREFERRED
+  - Matching tags: primary is selected
+  - Non-matching tags: primary is selected
+- SECONDARY
+  - Matching tags: select any eligible secondary
+  - Non-matching tags: no server selected
+- SECONDARY_PREFERRED
+  - Matching tags: select any eligible secondary
+  - Non-matching tags: primary is selected
+- NEAREST
+  - Matching tags: select any eligible server
+  - Non-matching tags: no server selected
+
+**Writes**
+
+- Primary is selected.
+
+### Topology Type Sharded
+
+**Reads**
+
+- Select any mongos.
+
+**Writes**
+
+- Select any mongos.
+
+### Topology Type Unknown
+
+**Reads**
+
+- No server is selected.
+
+**Writes**
+
+- No server is selected.
+
+## Passing ReadPreference to Mongos
+
+While there are no YAML tests for this, drivers are strongly encouraged to test in a way specific to their
+implementation that ReadPreference is correctly passed to Mongos in the following scenarios:
+
+- PRIMARY
+  - the SecondaryOk wire protocol flag is NOT set
+  - $readPreference is NOT used
+- PRIMARY_PREFERRED
+  - the SecondaryOk wire protocol flag is set
+  - $readPreference is used
+- SECONDARY
+  - the SecondaryOk wire protocol flag is set
+  - $readPreference is used
+- SECONDARY_PREFERRED
+  - the SecondaryOk wire protocol flag is set
+  - if `tag_sets` or `hedge` are specified $readPreference is used, otherwise $readPreference is NOT used
+- NEAREST
+  - the SecondaryOk wire protocol flag is set
+  - $readPreference is used
+
+## Random Selection Within Latency Window (single-threaded drivers)
+
+The Server Selection spec mandates that single-threaded drivers select a server at random from the set of suitable
+servers that are within the latency window. Drivers implementing the spec SHOULD test their implementations in a
+language-specific way to confirm randomness.
+
+For example, the following topology description, operation, and read preference will return a set of three suitable
+servers within the latency window:
+
+```
+    topology_description:
+      type: ReplicaSetWithPrimary
+      servers:
+      - &secondary_1
+        address: b:27017
+        avg_rtt_ms: 5
+        type: RSSecondary
+        tags: {}
+      - &secondary_2
+        address: c:27017
+        avg_rtt_ms: 10
+        type: RSSecondary
+        tags: {}
+      - &primary
+        address: a:27017
+        avg_rtt_ms: 6
+        type: RSPrimary
+        tags: {}
+    operation: read
+    read_preference:
+      mode: Nearest
+      tags: {}
+    in_latency_window:
+    - *primary
+    - *secondary_1
+    - *secondary_2
+```
+
+Drivers SHOULD check that their implementation selects one of `primary`, `secondary_1`, and `secondary_2` at random.
+
+## operationCount-based Selection Within Latency Window (multi-threaded or async drivers)
+
+The Server Selection spec mandates that multi-threaded or async drivers select a server from within the latency window
+according to their operationCounts. There are YAML tests verifying that drivers implement this selection correctly which
+can be found in the `tests/in_window` directory. Multi-threaded or async drivers implementing the spec MUST use them to
+test their implementations.
+
+The YAML tests each include some information about the servers within the late ncy window. For each case, the driver
+passes this information into whatever function it uses to select from within the window. Because the selection algorithm
+relies on randomness, this process MUST be repeated 2000 times. Once the 2000 selections are complete, the runner
+tallies up the number of times each server was selected and compares those counts to the expected results included in
+the test case. Specifics of the test format and how to run the tests are included in the tests README.
+
+### Prose Test
+
+Multi-threaded and async drivers MUST also implement the following prose test:
+
+1. Configure a sharded cluster with two mongoses. Use a 4.2.9 or newer server version.
+
+2. Enable the following failpoint against exactly one of the mongoses:
+
+   ```
+   {
+      configureFailPoint: "failCommand",
+      mode: { times: 10000 },
+      data: {
+          failCommands: ["find"],
+          blockConnection: true,
+          blockTimeMS: 500,
+          appName: "loadBalancingTest",
+      },
+   }
+   ```
+
+3. Create a client with both mongoses' addresses in its seed list, appName="loadBalancingTest", and
+   localThresholdMS=30000.
+
+   - localThresholdMS is set to a high value to help avoid a single mongos being selected too many times due to a random
+     spike in latency in the other mongos.
+
+4. Using CMAP events, ensure the client's connection pools for both mongoses have been saturated, either via setting
+   minPoolSize=maxPoolSize or executing operations.
+
+   - This helps reduce any noise introduced by connection establishment latency during the actual server selection
+     tests.
+
+5. Start 10 concurrent threads / tasks that each run 10 `findOne` operations with empty filters using that client.
+
+6. Using command monitoring events, assert that fewer than 25% of the CommandStartedEvents occurred on the mongos that
+   the failpoint was enabled on.
+
+7. Disable the failpoint.
+
+8. Start 10 concurrent threads / tasks that each run 100 `findOne` operations with empty filters using that client.
+
+9. Using command monitoring events, assert that each mongos was selected\
+   roughly 50% of the time (within +/- 10%).
+
+## Application-Provided Server Selector
+
+The Server Selection spec allows drivers to configure registration of a server selector function that filters the list
+of suitable servers. Drivers implementing this part of the spec MUST test that:
+
+- The application-provided server selector is executed as part of the server selection process when there is a nonzero
+  number of candidate or eligible servers. For example, execute a test against a replica set: Register a server selector
+  that selects the suitable server with the highest port number. Execute 10 queries with nearest read preference and,
+  using command monitoring, assert that all the operations execute on the member with the highest port number.
-Original file line number
+Diff line change
@@ Expand Up / @@ -5,6 +5,9 @@ @@
           },
           {
             "pattern": "^https://github.com/10gen/mongo-enterprise-modules"
+          },
+          {
+            "pattern": "^#"
           }
         ]
     }