Pdoc fails to import shapely / geopandas and therefore does not run.

[bubalis]

Problem Description

I'm trying to create documentation for a geoprocessing routine. When I run pdoc on the module, it cannot import geopandas / shapely and therefore fails.

Steps to reproduce the behavior:

create file: test.py - one line: import geopandas as gpd
pdoc test.py #Fails
python test.py #Runs Fine

System Information

requirements.txt

pdoc: 10.0.1
Python: 3.9.7
Platform: Linux-4.19.128-microsoft-standard-x86_64-with-glibc2.31

Full error trace:

Warn: Error importing Flood_Demand_ES.test:
Traceback (most recent call last):
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/extract.py", line 190, in load_module
return importlib.import_module(module)
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1030, in _gcd_import
File "", line 1007, in _find_and_load
File "", line 986, in _find_and_load_unlocked
File "", line 680, in _load_unlocked
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/mnt/c/Users/benja/Flood_Demand_ES/test.py", line 1, in
import geopandas as gpd
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/geopandas/init.py", line 1, in
from geopandas._config import options # noqa
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/geopandas/_config.py", line 126, in
default_value=_default_use_pygeos(),
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/geopandas/_config.py", line 112, in _default_use_pygeos
import geopandas._compat as compat
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/geopandas/_compat.py", line 10, in
import shapely.geos
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/shapely/geos.py", line 104, in
free = load_dll('c', fallbacks=c_alt_paths).free
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/shapely/geos.py", line 60, in load_dll
raise OSError(
OSError: Could not find lib c or load any of its variants ['libc.musl-x86_64.so.1'].

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/init.py", line 433, in pdoc
all_modules[module_name] = doc.Module.from_name(module_name)
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/doc.py", line 364, in from_name
return cls(extract.load_module(name))
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/contextlib.py", line 79, in inner
return func(*args, **kwds)
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/extract.py", line 192, in load_module
raise RuntimeError(f"Error importing {module}") from e
RuntimeError: Error importing Flood_Demand_ES.test
(/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/init.py:435)
Traceback (most recent call last):
File "/home/bdube/miniconda3/envs/Flood/bin/pdoc", line 8, in
sys.exit(cli())
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/main.py", line 191, in cli
pdoc.pdoc(
File "/home/bdube/miniconda3/envs/Flood/lib/python3.9/site-packages/pdoc/init.py", line 438, in pdoc
raise RuntimeError("Unable to import any modules.")
RuntimeError: Unable to import any modules.

[mhils]

Thanks for the detailed report. I'm unable to reproduce this though, things are working just fine for me:

λ python --version
Python 3.10.2
λ python -m venv repro
λ repro/bin/pip install pdoc geopandas
[...]
λ echo "import geopandas as pd" > foo.py
λ repro/bin/pdoc foo.py -o /tmp
# no errors

I tried installing your requirements.txt file, but that fails on my system with some unrelated GDAL issues when installing Fiona.

[mhils]

Figured it out. This only affects shapely 1.7.1, 1.8 is unaffected. In 1.7.1, there's some code that calls ctypes.util.find_library. On Linux, find_library calls subprocess.Popen(['/sbin/ldconfig', '-p'], ...). However, pdoc blocks all Popen calls to disable any side effects when loading modules. Sounds like we need to make an exception for ldconfig. :)

[mhils]

This is fixed on main now, thanks again. 😃