Autodoc failing to import *some* modules, only on RTD

[anicusan]

Details

Hello, I have a rather odd situation where essentially the same Sphinx conf / code I used for another Python library (pept) works perfectly, but for Coexist produces warnings only on readthedocs.

My .readthedocs.yml file is quite vanilla:

version: 2

sphinx:
  configuration: docs/source/conf.py

formats: all

python:
  install:
    - requirements: requirements.txt
    - requirements: requirements_extra.txt
    - method: pip
      path: .
  system_packages: true

• Read the Docs project URL: https://coexist.readthedocs.io/en/latest/
• Build URL (if applicable): https://readthedocs.org/api/v2/build/15104962.txt
• Read the Docs username (if applicable): anicusan

Expected Result

Creating the docs locally works perfectly, with no warnings produced. I tested on new, empty environments with Python 3.7 and 3.9; the library is written in pure Python, no C extensions or deeply-nested module hierarchies. I expected correct autosummaries on RTD too.

Actual Result

Looking at the build logs, the library installation went fine; there are no import errors: Successfully installed coexist-0.1.0. However, when generating the docstrings with autosummary & numpydoc, I get multiple warnings only on RTD, e.g. WARNING: autosummary: failed to import coexist.plots.access; however, other objects at the same nesting level are found, e.g. writing output... [ 44%] manual/generated/coexist.schedulers.Scheduler.

In summary:

• Locally everything works fine, even in fresh environments.
• On RTD the library is installed correctly, and some library functions are found by autosummary.
• However, many other functions get autosummary: failed to import, even at the same nesting level.

No amount of fiddling, Googling and reading documentation seemed to solve this.
Any help would be greatly appreciated.

Best wishes,
Leonard

[astrojuanlu]

Hi @anicusan , I suspect the problem might be that autosummary produces misleading warnings: sphinx-doc/sphinx#7989 Therefore, I suspect that there is some missing dependency or error that arises on import, even if coexist is installed correctly.

[humitos]

However, when generating the docstrings with autosummary & numpydoc, I get multiple warnings only on RTD,

It may be worth checking that pip list returns exactly the same package versions on both, local and on Read the Docs. I enabled to show the pip list in the build output of your project. Let us know if you find any difference there.

[anicusan]

Hi @astrojuanlu and @humitos , a million thanks, I found the issue - stupidly forgetting to add plotly to requirements.txt! The pip list command (for others reading this, this is what the logs looked like) was essential for finding the error. Would it be possible to always show pip list when building on RTD?

If you found this issue via Google: you might have a dependency missing deep inside the module imports (if you have e.g. multiple .py files or submodules) and the library may end up in a partially initialised state before the import error happened. In such cases, the library might look like it installed successfully, but autosummary errors only on the parent modules of the script where the missing library import failed. Rather confusing indeed, but installing only the packages listed in "requirements.txt" - and nothing else, and no --system-site-packages - will show the error when trying to import the library in e.g. the Python interpreter. Weirdly, on IPython I only get ModuleNotFoundError: No module named 'coexist'; it's only on the Python interpreter itself that I see ModuleNotFoundError: No module named 'plotly'. What a joy.

[humitos]

Would it be possible to always show pip list when building on RTD?

There is some conversation about this in #5006 and #6992 that you be interested in. I think this could be part of the new templates work we are doing.