DRIVERS-2789 Convert CMAP Spec to Markdown (#1500)

* Cleanup

* convert to MD tables

* fix migration script and update links

* update script to handle absolute paths

* update script to handle all path types

* undo comment out

.pre-commit-config.yaml

@@ -34,6 +34,7 @@ repos:
   rev: v3.11.2
   hooks:
     - id: markdown-link-check
+      args: ["-c", "markdown_link_config.json"]
 
 - repo: https://github.com/rstcheck/rstcheck
   rev: v6.2.0

markdown_link_config.json

@@ -0,0 +1,7 @@
+{
+    "ignorePatterns": [
+      {
+        "pattern": "^/source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst#"
+      }
+    ]
+}

scripts/migrate_to_md.py

@@ -92,36 +92,47 @@
 # (https://github.com/mongodb/specifications/blob/master/source/...)
 # and rewrite them to use appropriate md links.
 # If the link is malformed we ignore and print an error.
-pattern = re.compile(f'(<.*{path.parent.name}/{path.name}[>#])')
+rel_pattern = re.compile(f'(\.\.\S*/{path.name})')
+md_pattern = re.compile(f'(\(http\S*/{path.name})')
+rst_pattern = re.compile(f'(<http\S*/{path.name})')
+abs_pattern = re.compile(f'(/source/\S*/{path.name})')
 for p in Path("source").rglob("*"):
-    if not '.rst' in p.name:
+    if p.suffix not in ['.rst', '.md']:
         continue
     found = False
     with p.open() as fid:
         lines = fid.readlines()
     new_lines = []
+    relpath = os.path.relpath(md_file, start=p.parent)
     for line in lines:
-        match = re.search(pattern, line)
-        if not match:
-            new_lines.append(line)
-            continue
-        matchstr = match.groups()[0]
-        if matchstr[0].startswith('https'):
-            if not substr.startswith('<https://github.com/mongodb/specifications/blob/master/source/'):
-                print('*** Error in link: ', substr, p)
-                new_lines.append(line)
-                continue
-        found = True
-        relpath = os.path.relpath(path.parent.parent, start=p.parent)
-        new_name = path.name.replace('.rst', '.md')
-        new_substr = f'<{relpath}/{path.parent.name}/{new_name}{matchstr[-1]}'
-        new_lines.append(line.replace(matchstr, new_substr))
+        new_line = line
+        if re.search(rel_pattern, line):
+            matchstr = re.search(rel_pattern, line).groups()[0]
+            new_line = line.replace(matchstr, relpath)
+        elif re.search(md_pattern, line):
+            matchstr = re.search(md_pattern, line).groups()[0]
+            if not matchstr.startswith('(https://github.com/mongodb/specifications/blob/master/source'):
+                print('*** Error in link: ', matchstr, p)
+            else:
+                new_line = line.replace(matchstr, f'({relpath}')
+        elif re.search(rst_pattern, line):
+            matchstr = re.search(rst_pattern, line).groups()[0]
+            if not matchstr.startswith('<https://github.com/mongodb/specifications/blob/master/source'):
+                print('*** Error in link: ', matchstr, p)
+            else:
+                new_line = line.replace(matchstr, f'<{relpath}')
+        elif re.search(abs_pattern, line):
+            matchstr = re.search(abs_pattern, line).groups()[0]
+            new_line = line.replace(matchstr, relpath)
+
+        if new_line != line:
+            found = True
+        new_lines.append(new_line)
 
     if found:
         with p.open('w') as fid:
             fid.writelines(new_lines)
-        print(f'Update link(s) in {p.name}')
-
+        print(f'Update link(s) in {p}')
 
 
 print('Created markdown file:')

source/client-side-encryption/client-side-encryption.rst

@@ -2511,7 +2511,7 @@ These operations MUST NOT reuse the same connection pool as the parent
 situations.
 
 Drivers supporting a connection pool (see `CMAP specification
-</source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst>`_)
+<../connection-monitoring-and-pooling/connection-monitoring-and-pooling.md>`_)
 support an option for limiting the connection pool size: ``maxPoolSize``.
 
 Drivers need to check out a connection before serializing the command. If the

source/client-side-encryption/tests/README.rst

@@ -1133,7 +1133,7 @@ Repeat the steps from the "Via bypassAutoEncryption" test, replacing "bypassAuto
 9. Deadlock Tests
 ~~~~~~~~~~~~~~~~~
 
-.. _Connection Monitoring and Pooling: /source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst
+.. _Connection Monitoring and Pooling: ../../connection-monitoring-and-pooling/connection-monitoring-and-pooling.md
 
 The following tests only apply to drivers that have implemented a connection pool (see the `Connection Monitoring and Pooling`_ specification).
 

source/command-logging-and-monitoring/command-logging-and-monitoring.rst

@@ -408,7 +408,7 @@ The following key-value pairs MUST be included in all command messages:
    * - driverConnectionId
      - Int64
      - The driver's ID for the connection used for the command. Note this is NOT the same as ``CommandStartedEvent.connectionId`` defined above,
-       but refers to the `connectionId` defined in the  `connection monitoring and pooling specification <../connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst>`_.
+       but refers to the `connectionId` defined in the  `connection monitoring and pooling specification <../connection-monitoring-and-pooling/connection-monitoring-and-pooling.md>`_.
        Unlike ``CommandStartedEvent.connectionId`` this field MUST NOT contain the host/port; that information MUST be in the following fields,
        ``serverHost`` and ``serverPort``. This field is optional for drivers that do not implement CMAP if they do have an equivalent concept of
        a connection ID.