forked from mongodb/specifications
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
DRIVERS-2789 Convert Polling SRV Records to Markdown
- Loading branch information
Showing
6 changed files
with
243 additions
and
317 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
298 changes: 118 additions & 180 deletions
298
...ng-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,238 +1,176 @@ | ||
| .. role:: javascript(code) | ||
| :language: javascript | ||
| # Polling SRV Records for mongos Discovery | ||
|
|
||
| ======================================== | ||
| Polling SRV Records for mongos Discovery | ||
| ======================================== | ||
| - Status: Accepted | ||
| - Minimum Server Version: N/A | ||
|
|
||
| :Status: Accepted | ||
| :Minimum Server Version: N/A | ||
| ______________________________________________________________________ | ||
|
|
||
| .. contents:: | ||
| ## Abstract | ||
|
|
||
| -------- | ||
| Currently the [Initial DNS Seedlist Discovery](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md) | ||
| functionality provides a static seedlist when a MongoClient is constructed. Periodically polling the DNS SRV records | ||
| would allow for the mongos proxy list to be updated without having to change client configuration. | ||
|
|
||
| Abstract | ||
| ======== | ||
| This specification builds on top of the original Initial DNS Seedlist Discovery specification, and modifies the | ||
| [Server Discovery and Monitoring](../server-discovery-and-monitoring/server-discovery-and-monitoring.md) specification's | ||
| definition of monitoring a set of mongos servers in a Sharded TopologyType. | ||
|
|
||
| Currently the `Initial DNS Seedlist Discovery`_ functionality provides a static | ||
| seedlist when a MongoClient is constructed. Periodically polling the DNS SRV | ||
| records would allow for the mongos proxy list to be updated without having to | ||
| change client configuration. | ||
| ## META | ||
|
|
||
| This specification builds on top of the original Initial DNS Seedlist | ||
| Discovery specification, and modifies the `Server Discovery and Monitoring`_ | ||
| specification's definition of monitoring a set of mongos servers in a Sharded | ||
| TopologyType. | ||
| The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and | ||
| "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://www.ietf.org/rfc/rfc2119.txt). | ||
|
|
||
| .. _`Initial DNS Seedlist Discovery`: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md | ||
| .. _`Server Discovery and Monitoring`: ../server-discovery-and-monitoring/server-discovery-and-monitoring.md | ||
| ## Specification | ||
|
|
||
| META | ||
| ==== | ||
| ### Terms | ||
|
|
||
| The keywords “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, | ||
| “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be | ||
| interpreted as described in `RFC 2119 <https://www.ietf.org/rfc/rfc2119.txt>`_. | ||
| #### rescan, rescanning | ||
|
|
||
| Specification | ||
| ============= | ||
| A rescan is the periodic scan of all DNS SRV records to discover a new set of mongos hosts. | ||
|
|
||
| Terms | ||
| ----- | ||
| #### rescanSRVIntervalMS | ||
|
|
||
| rescan, rescanning | ||
| ~~~~~~~~~~~~~~~~~~ | ||
| An internal value representing how often the DNS SRV records should be queried for. | ||
|
|
||
| A rescan is the periodic scan of all DNS SRV records to discover a new set of | ||
| mongos hosts. | ||
| ### Implementation | ||
|
|
||
| rescanSRVIntervalMS | ||
| ~~~~~~~~~~~~~~~~~~~ | ||
| If the initial topology was created through a `mongodb+srv://` URI, then drivers MUST implement this specification by | ||
| periodically rescanning the SRV DNS records. There MUST NOT be an option to turn this behaviour off. | ||
|
|
||
| An internal value representing how often the DNS SRV records should be queried | ||
| for. | ||
| Drivers MUST NOT implement this specification if they do not adhere fully to the | ||
| [Initial DNS Seedlist Discovery](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md) specification. | ||
|
|
||
| Implementation | ||
| -------------- | ||
| If the initial topology was created through a ``mongodb+srv://`` URI, then | ||
| drivers MUST implement this specification by periodically rescanning the SRV | ||
| DNS records. There MUST NOT be an option to turn this behaviour off. | ||
| Drivers MUST NOT implement this specification if they do not adhere fully to | ||
| the `Initial DNS Seedlist Discovery`_ specification. | ||
| This feature is only available when the Server Discovery has determined that | ||
| the TopologyType is Sharded, or Unknown. Drivers MUST NOT rescan SRV DNS | ||
| records when the Topology is not Sharded (i.e. Single, ReplicaSetNoPrimary, or | ||
| This feature is only available when the Server Discovery has determined that the TopologyType is Sharded, or Unknown. | ||
| Drivers MUST NOT rescan SRV DNS records when the Topology is not Sharded (i.e. Single, ReplicaSetNoPrimary, or | ||
| ReplicaSetWithPrimary). | ||
|
|
||
| The discovery of a set of mongos servers is explained in the seedlist_ | ||
| discovery section of the original specification. The behaviour of the periodic | ||
| rescan is similar, but not identical to the behaviour of initial seedlist | ||
| discovery. Periodic scan MUST follow these rules: | ||
| - The driver will query the DNS server for SRV records on | ||
| ``{hostname}.{domainname}``, prefixed with the SRV service name and protocol. | ||
| The SRV service name is provided in the srvServiceName_ URI option and | ||
| defaults to ``mongodb``. The protocol is always ``tcp``. After prefixing, the | ||
| URI should look like: ``_{srvServiceName}._tcp.{hostname}.{domainname}``. | ||
| - A driver MUST verify that the host names returned through SRV records have | ||
| the same parent ``{domainname}``. When this verification fails, a driver: | ||
| The discovery of a set of mongos servers is explained in the | ||
| [seedlist](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#seedlist-discovery) discovery section of | ||
| the original specification. The behaviour of the periodic rescan is similar, but not identical to the behaviour of | ||
| initial seedlist discovery. Periodic scan MUST follow these rules: | ||
|
|
||
| - The driver will query the DNS server for SRV records on `{hostname}.{domainname}`, prefixed with the SRV service name | ||
| and protocol. The SRV service name is provided in the | ||
| [srvServiceName](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvservicename) URI option and | ||
| defaults to `mongodb`. The protocol is always `tcp`. After prefixing, the URI should look like: | ||
| `_{srvServiceName}._tcp.{hostname}.{domainname}`. | ||
| - A driver MUST verify that the host names returned through SRV records have the same parent `{domainname}`. When this | ||
| verification fails, a driver: | ||
| - MUST NOT add such a non-compliant host name to the topology | ||
| - MUST NOT raise an error | ||
| - SHOULD log the non-compliance, including the host name | ||
| - MUST NOT initiate a connection to any such host | ||
| - If the DNS request returns no verified hosts in SRV records, no SRV records | ||
| at all, or a DNS error happens, the driver: | ||
| - If the DNS request returns no verified hosts in SRV records, no SRV records at all, or a DNS error happens, the | ||
| driver: | ||
| - MUST NOT change the topology | ||
| - MUST NOT raise an error | ||
| - SHOULD log this situation, including the reason why the DNS records | ||
| could not be found, if possible | ||
| - MUST temporarily set *rescanSRVIntervalMS* to *heartbeatFrequencyMS* until | ||
| at least one verified SRV record is obtained. | ||
| - For all verified host names, as returned through the DNS SRV query, the | ||
| driver: | ||
| - MUST remove all hosts that are part of the topology, but are no longer | ||
| in the returned set of valid hosts | ||
| - MUST NOT remove all hosts, and then re-add the ones that were returned. | ||
| Hosts that have not changed, MUST be left alone and unchanged. | ||
| - If srvMaxHosts_ is zero or greater than or equal to the number of valid | ||
| hosts, each valid new host MUST be added to the topology as Unknown. | ||
| - If srvMaxHosts_ is greater than zero and less than the number of valid | ||
| hosts, valid new hosts MUST be randomly selected and added to the topology | ||
| as Unknown until the topology has ``srvMaxHosts`` hosts. Drivers MUST use | ||
| the same randomization algorithm as they do for `initial selection`_. | ||
| - Priorities and weights in SRV records MUST continue to be ignored, and MUST | ||
| NOT dictate which mongos server is used for new connections. | ||
| The rescan needs to happen periodically. As SRV records contain a TTL value, | ||
| this value can be used to indicate when a rescan needs to happen. Different | ||
| SRV records can have different TTL values. The *rescanSRVIntervalMS* value MUST | ||
| be set to the lowest of the individual TTL values associated with the | ||
| different SRV records in the most recent rescan, but MUST NOT be lower | ||
| than *60 seconds*. If a driver is unable to access the TTL values of SRV | ||
| records, it MUST rescan every 60 seconds. | ||
| Drivers SHOULD endeavour to rescan and obtain a new list of mongos servers | ||
| every *rescanSRVIntervalMS* value. The *rescanSRVIntervalMS* period SHOULD be | ||
| calculated from the **end** of the previous rescan (or the **end** of the | ||
| initial DNS seedlist discovery scan). | ||
| .. _seedlist: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#seedlist-discovery | ||
| .. _srvMaxHosts: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvmaxhosts | ||
| .. _srvServiceName: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvservicename | ||
| .. _`initial selection`: ../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#querying-dns | ||
| Multi-Threaded Drivers | ||
| ---------------------- | ||
| A threaded driver MUST use a separate monitoring thread for scanning the DNS | ||
| records so that DNS lookups don't block other operations. | ||
| Single-Threaded Drivers | ||
| ----------------------- | ||
| - SHOULD log this situation, including the reason why the DNS records could not be found, if possible | ||
| - MUST temporarily set *rescanSRVIntervalMS* to *heartbeatFrequencyMS* until at least one verified SRV record is | ||
| obtained. | ||
| - For all verified host names, as returned through the DNS SRV query, the driver: | ||
| - MUST remove all hosts that are part of the topology, but are no longer in the returned set of valid hosts | ||
| - MUST NOT remove all hosts, and then re-add the ones that were returned. Hosts that have not changed, MUST be left | ||
| alone and unchanged. | ||
| - If [srvMaxHosts](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvmaxhosts) is zero or greater | ||
| than or equal to the number of valid hosts, each valid new host MUST be added to the topology as Unknown. | ||
| - If [srvMaxHosts](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#srvmaxhosts) is greater than | ||
| zero and less than the number of valid hosts, valid new hosts MUST be randomly selected and added to the topology as | ||
| Unknown until the topology has `srvMaxHosts` hosts. Drivers MUST use the same randomization algorithm as they do for | ||
| [initial selection](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md#querying-dns). | ||
| - Priorities and weights in SRV records MUST continue to be ignored, and MUST NOT dictate which mongos server is used | ||
| for new connections. | ||
|
|
||
| The rescan needs to happen periodically. As SRV records contain a TTL value, this value can be used to indicate when a | ||
| rescan needs to happen. Different SRV records can have different TTL values. The *rescanSRVIntervalMS* value MUST be set | ||
| to the lowest of the individual TTL values associated with the different SRV records in the most recent rescan, but MUST | ||
| NOT be lower than *60 seconds*. If a driver is unable to access the TTL values of SRV records, it MUST rescan every 60 | ||
| seconds. | ||
|
|
||
| Drivers SHOULD endeavour to rescan and obtain a new list of mongos servers every *rescanSRVIntervalMS* value. The | ||
| *rescanSRVIntervalMS* period SHOULD be calculated from the **end** of the previous rescan (or the **end** of the initial | ||
| DNS seedlist discovery scan). | ||
|
|
||
| ### Multi-Threaded Drivers | ||
|
|
||
| A threaded driver MUST use a separate monitoring thread for scanning the DNS records so that DNS lookups don't block | ||
| other operations. | ||
|
|
||
| ### Single-Threaded Drivers | ||
|
|
||
| The rescan MUST happen **before** scanning all servers as part of the normal | ||
| scanning_ functionality, but only if *rescanSRVIntervalMS* has passed. | ||
| .. _scanning: ../server-discovery-and-monitoring/server-discovery-and-monitoring.md#scanning | ||
| [scanning](../server-discovery-and-monitoring/server-discovery-and-monitoring.md#scanning) functionality, but only if | ||
| *rescanSRVIntervalMS* has passed. | ||
|
|
||
| Test Plan | ||
| ========= | ||
| ## Test Plan | ||
|
|
||
| See README.rst in the accompanying `test directory`_. | ||
| See README.rst in the accompanying [test directory](tests). | ||
|
|
||
| .. _`test directory`: tests | ||
| ## Motivation for Change | ||
|
|
||
| Motivation for Change | ||
| ===================== | ||
| The original [Initial DNS Seedlist Discovery](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md) | ||
| specification only regulates the initial list of mongos hosts to be used instead of a single hostname from a connection | ||
| URI. Although this makes the initial configuration of a set of mongos servers a lot easier, it does not provide a method | ||
| for updating the list of mongos servers in the topology. | ||
|
|
||
| The original `Initial DNS Seedlist Discovery`_ specification only regulates | ||
| the initial list of mongos hosts to be used instead of a single hostname from | ||
| a connection URI. Although this makes the initial configuration of a set of | ||
| mongos servers a lot easier, it does not provide a method for updating the | ||
| list of mongos servers in the topology. | ||
| Since the introduction of the ``mongodb+srv://`` schema to provide an initial | ||
| seedlist, some users have requested additional functionality to be able to | ||
| update the configured list of mongos hosts that make up the initially seeded | ||
| Since the introduction of the `mongodb+srv://` schema to provide an initial seedlist, some users have requested | ||
| additional functionality to be able to update the configured list of mongos hosts that make up the initially seeded | ||
| topology: | ||
|
|
||
| - https://jira.mongodb.org/browse/JAVA-2927 | ||
| - <https://jira.mongodb.org/browse/JAVA-2927> | ||
|
|
||
| Design Rationale | ||
| ================ | ||
| ## Design Rationale | ||
|
|
||
| From the scope document | ||
| ----------------------- | ||
| ### From the scope document | ||
|
|
||
| Should DNS polling use heartbeatFrequencyMS or DNS cache TTLs? | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
| #### Should DNS polling use heartbeatFrequencyMS or DNS cache TTLs? | ||
|
|
||
| We have selected to use lowest TTLs among all DNS SRV records, with a caveat | ||
| that the rescan frequency is not lower than 60 seconds. | ||
| We have selected to use lowest TTLs among all DNS SRV records, with a caveat that the rescan frequency is not lower than | ||
| 60 seconds. | ||
|
|
||
| Should DNS polling also have a "fast polling" mode when no servers are available? | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
| #### Should DNS polling also have a "fast polling" mode when no servers are available? | ||
|
|
||
| We have not opted to have a "fast polling" mode, but we did include a | ||
| provision that a rescan needs to happen when DNS records are not available. In | ||
| that case, a rescan would happen every *heartbeatFrequencyMS*. The rationale | ||
| being that polling DNS really often really fast does not make a lot of sense | ||
| due to DNS caching, which often uses the TTL already anyway, but when we have | ||
| no TTL records to reference we still need a fallback frequency. | ||
| We have not opted to have a "fast polling" mode, but we did include a provision that a rescan needs to happen when DNS | ||
| records are not available. In that case, a rescan would happen every *heartbeatFrequencyMS*. The rationale being that | ||
| polling DNS really often really fast does not make a lot of sense due to DNS caching, which often uses the TTL already | ||
| anyway, but when we have no TTL records to reference we still need a fallback frequency. | ||
|
|
||
| For the design | ||
| -------------- | ||
| ### For the design | ||
|
|
||
| No option to turn off periodic rescanning | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
| #### No option to turn off periodic rescanning | ||
|
|
||
| The design does not allow for an option to turn off the periodic rescanning of | ||
| SRV records on the basis that we try to have as few options as possible: the | ||
| "no knobs" philosophy. | ||
| The design does not allow for an option to turn off the periodic rescanning of SRV records on the basis that we try to | ||
| have as few options as possible: the "no knobs" philosophy. | ||
|
|
||
| Backwards Compatibility | ||
| ======================= | ||
| ## Backwards Compatibility | ||
|
|
||
| This specification changes the behaviour of server monitoring by introducing a | ||
| repeating DNS lookup of the SRV records. Although this is an improvement in | ||
| the ``mongodb+srv://`` scheme it can nonetheless break expectations with users | ||
| that were familiar with the old behaviour. We do not expect this to negatively | ||
| impact users. | ||
| This specification changes the behaviour of server monitoring by introducing a repeating DNS lookup of the SRV records. | ||
| Although this is an improvement in the `mongodb+srv://` scheme it can nonetheless break expectations with users that | ||
| were familiar with the old behaviour. We do not expect this to negatively impact users. | ||
|
|
||
| Reference Implementation | ||
| ======================== | ||
| ## Reference Implementation | ||
|
|
||
| Reference implementations are made for the following drivers: | ||
|
|
||
| - Perl | ||
| - C# | ||
|
|
||
| Security Implication | ||
| ==================== | ||
| ## Security Implication | ||
|
|
||
| This specification has no security implications beyond the ones associated | ||
| with the original `Initial DNS Seedlist Discovery`_ specification. | ||
| This specification has no security implications beyond the ones associated with the original | ||
| [Initial DNS Seedlist Discovery](../initial-dns-seedlist-discovery/initial-dns-seedlist-discovery.md) specification. | ||
|
|
||
| Future work | ||
| =========== | ||
| ## Future work | ||
|
|
||
| No future work is expected. | ||
|
|
||
| Changelog | ||
| ========= | ||
| ## Changelog | ||
|
|
||
| - 2024-08-22: Migrated from reStructuredText to Markdown. | ||
|
|
||
| - 2022-10-05: Revise spec front matter and reformat changelog. | ||
|
|
||
| - 2021-10-14: Specify behavior for `srvMaxHosts` MongoClient option. | ||
|
|
||
| :2022-10-05: Revise spec front matter and reformat changelog. | ||
| :2021-10-14: Specify behavior for ``srvMaxHosts`` MongoClient option. | ||
| :2021-09-15: Clarify that service name only defaults to ``mongodb``, and should | ||
| be defined by the ``srvServiceName`` URI option. | ||
| - 2021-09-15: Clarify that service name only defaults to `mongodb`, and should\ | ||
| be defined by the `srvServiceName` URI | ||
| option. |
4 changes: 4 additions & 0 deletions
4
...g-srv-records-for-mongos-discovery/polling-srv-records-for-mongos-discovery.rst
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
|
|
||
| .. note:: | ||
| This specification has been converted to Markdown and renamed to | ||
| `polling-srv-records-for-mongos-discovery.md <polling-srv-records-for-mongos-discovery.md>`_. |
Oops, something went wrong.