[tests] tests on Python 3.13 are very slow

[picnixz]

The issue is likely due to something happening on 3.13. I did not find a related issue yet but the tests on 3.13 require 15 minutes (!!!)

Possibly related: python/cpython#117108

[picnixz]

Some timings (I've simplified the tests but look at this):

pytest tests/test_builders/test_build_html_5_output.py::test_html5_output --durations=3

Python 3.10.13 (optimized build)

0.41s call     tests/test_builders/test_build_html_5_output.py::test_html5_output[images.html-.//img[@src='_images/img.png']-]
0.07s setup    tests/test_builders/test_build_html_5_output.py::test_html5_output[images.html-.//img[@src='_images/img.png']-]
0.01s setup    tests/test_builders/test_build_html_5_output.py::test_html5_output[search.html-.//meta[@name='robots'][@content='noindex']-]

Python 3.13.0a5+ (heads/main:63d6f2623e, Mar 22 2024, 14:15:43) [GCC 7.5.0] on linux

0.92s call     tests/test_builders/test_build_html_5_output.py::test_html5_output[images.html-.//img[@src='_images/img.png']-]
0.14s setup    tests/test_builders/test_build_html_5_output.py::test_html5_output[images.html-.//img[@src='_images/img.png']-]
0.10s setup    tests/test_builders/test_build_html_5_output.py::test_html5_output[search.html-.//meta[@name='robots'][@content='noindex']-]

# with --enable-optimizations (it's slower)
0.98s call     tests/test_builders/test_build_html_5_output.py::test_html5_output[images.html-.//img[@src='_images/img.png']-]
0.14s setup    tests/test_builders/test_build_html_5_output.py::test_html5_output[images.html-.//img[@src='_images/img.png']-]
0.11s setup    tests/test_builders/test_build_html_5_output.py::test_html5_output[search.html-.//meta[@name='robots'][@content='noindex']-]

[picnixz]

The test slowdown appeared since yesterday so something happened but I'm not sure what. It's apparently not related to the deadsnake version since the latest main branch has this issue.

[hugovk]

Yes, this seems to be a CPython thing, our doctests have increased from ~9 mins to ~40 mins.

[picnixz]

Well, good to know (well not good since we wait more). I'll leave it opened until it's patched upstream (to remind myself).

[picnixz]

Now that python/cpython#117120 is merged, this should be fixed. However, since Sphinx is based on docutils and we could also have a lot of circular references, the memory usage could also blow up (as shown on python/cpython#117120 (comment)).

@hugovk Is it (or is it planned to be) possible to disable the incremental gc introduced in Python 3.13 and keep the old gc strategy?

[chrisjsewell]

Interesting insight into the garbage collector, and how finely tuned its performance needs to be 😅

[hugovk]

Now that python/cpython#117120 is merged, this should fixed. However, since Sphinx is based on docutils and we could also have a lot of circular references, the memory usage could also blow up (as shown on python/cpython#117120 (comment)).

If you see a memory usage, please do report on the issue or create a new one. This is exactly what the alpha releases are for, and thank you for already testing them!

@hugovk Is it (or is it planned to be) possible to disable the incremental gc introduced in Python 3.13 and keep the old gc strategy?

I don't think so, and don't see it in What's New or changelog, but I'm not following this area closely.

[picnixz]

Quick update: deadsnake version is still the one from yesterday morning so it does not contain the fix yet: Python 3.13.0a5+ (main, Mar 22 2024, 08:49:13) [GCC 11.4.0]. Our tests are likely to be back after the next nightly build.

[picnixz]

With the new nightly build, tests are back to their usual timings !