abs tol unable to handle non-real numerical noise

[dimpase]

E.g.

--- a/src/sage/doctest/parsing.py
+++ b/src/sage/doctest/parsing.py
@@ -1461,7 +1461,7 @@ class SageOutputChecker(doctest.OutputChecker):
             ...
             RuntimeError
             sage: 1  # abs tol 2
-            -0.5
+            -0.5+0.0001*I
             sage: print("0.9999")    # rel tol 1e-4
             1.0
             sage: print("1.00001")   # abs tol 1e-5

gives

sage -t --warn-long 20.5 --random-seed=37938279076849498617774420336933883040 src/sage/doctest/parsing.py
**********************************************************************
File "src/sage/doctest/parsing.py", line 1463, in sage.doctest.parsing.SageOutputChecker.check_output
Failed example:
    1  # abs tol 2
Expected:
    -0.5+0.0001*I
Got:
    1
**********************************************************************
1 item had failures:
   1 of  55 in sage.doctest.parsing.SageOutputChecker.check_output
    [322 tests, 1 failure, 0.17 s]
----------------------------------------------------------------------
sage -t --warn-long 20.5 --random-seed=37938279076849498617774420336933883040 src/sage/doctest/parsing.py  # 1 doctest failed

Indeed, RIF is used as data to check tolerances, no wonder things go wrong here.
By right, one needs a complex abs tol and rel tol.

Originally posted by @dimpase in #36509 (comment)

[jhpalmieri]

It's not precisely that it can't handle complex numerical noise, but there can't be a mismatch between the output type and the expected output type. One of the failing examples on OS X looks like

File "src/sage/tests/books/computational-mathematics-with-sagemath/mpoly_doctest.py", line 398, in sage.tests.books.computational-mathematics-with-sagemath.mpoly_doctest
Failed example:
    [CDF['x'](p(y=ys[0][0])).roots() for p in J.gens()] # abs tol 2e-15
Expected:
    [[(-0.5999999999999999, 1), (0.6000000000000001, 1)], [(0.6000000000000001, 1), (2.600000000000001, 1)]]
Got:
    [[(-0.5999999999999998, 1), (0.5999999999999999, 1)],
     [(0.6000000000000001 - 6.162975822039155e-33*I, 1),
      (2.6 + 3.0814879110195774e-32*I, 1)]]

If we change the expected output like this, it passses:

diff --git a/src/sage/tests/books/computational-mathematics-with-sagemath/mpoly_doctest.py b/src/sage/tests/books/computational-mathematics-with-sagemath/mpoly_doctest.py
index bef4a2b6c6..ab4be4fbbc 100644
--- a/src/sage/tests/books/computational-mathematics-with-sagemath/mpoly_doctest.py
+++ b/src/sage/tests/books/computational-mathematics-with-sagemath/mpoly_doctest.py
@@ -398,7 +398,7 @@ Sage example in ./mpoly.tex, line 1882::
   sage: ys = CDF['y'](Jy.0).roots(); ys # abs tol 2e-15
   [(-0.8, 1), (0.0, 1), (0.8, 1)]
   sage: [CDF['x'](p(y=ys[0][0])).roots() for p in J.gens()] # abs tol 2e-15
-  [[(-0.5999999999999999, 1), (0.6000000000000001, 1)], [(0.6000000000000001, 1), (2.600000000000001, 1)]]
+  [[(-0.5999999999999999, 1), (0.6000000000000001, 1)], [(0.6000000000000001 + 0*I, 1), (2.600000000000001 + 0*I, 1)]]
 
 Sage example in ./mpoly.tex, line 1911::
 

As far as I understand, the tolerance checker will check each real number in the output with the corresponding number that Sage actually produces, so if Sage produces a complex number (which gets parsed to a pair of reals), then the doctest has to actually provide a complex number (a pair of reals) to which to compare it. This is certainly a flaw in the doctesting procedure. It would be nice, at least for abs tol, if we could just take the absolute value of the difference between the expected output and the actual output and compare to the tolerance.

[vbraun]

I've implemented the "complex abs tol" proposal in #38433, please review