Establish testing against declared "minimal" supported versions of dependencies

[yarikoptic]

Came up in discussion of datalad/datalad-container#131 (comment) -- to raise or not the minimal version for datalad "for the sake of tests". But the question is - either datalad-container is still compatible with 0.12 series, which it declares to be the lower supported version.

@jwodder -- do you know an easy way to make pip install try to install "minimal" versions of packages were versioned dependency is requested? I doubt that there is, but who knows since it is IMHO a common situation.

Similar examples: after my whining about "too strict" (==) versioned dependencies, pynwb team established testing against "minimally supported" as well. In their case they maintain dependencies in requirement files, so https://github.com/NeurodataWithoutBorders/pynwb/blob/dev/requirements-min.txt declares minimal versions, and then setup.py compiles them into install_requires, while tox.ini installs using -r requirements.txt.

I wonder what could be "our" approach... E.g. we could just compliment current pip install -e .[devel] with ad-hoc post-install downgrade to versions we find in setup.py? or make setup.py (upon env var set) restrict to minimal instead of >= with smth like

(git)lena:~datalad/datalad-container[master]git
$> git diff
diff --git a/setup.py b/setup.py
index b64f17f..0403122 100755
--- a/setup.py
+++ b/setup.py
@@ -64,6 +64,17 @@ requires = {
         'nose>=1.3.4',
     ],
 }
+
+
+def restrict_to_minimal(requires):
+    for name, reqs in requires.items():
+        for i, req in enumerate(reqs):
+            reqs[i] = req.replace('>=', '==')
+    return requires
+
+# TODO: based on env var
+requires = restrict_to_minimal(requires)
+
 requires['devel'] = sum(list(requires.values()), [])
 
 setup(

but that requires "per extension" tune up.

I think the best would be to provide a setup() shim which would take care about tuning up dependencies. Since would be useful not only for datalad-extensions testing, I guess it should be included in https://github.com/datalad/datalad-buildsupport.

WDYT @jwodder @mih @kyleam @bpoldrack ?

[jwodder]

do you know an easy way to make pip install try to install "minimal" versions of packages were versioned dependency is requested? I doubt that there is, but who knows since it is IMHO a common situation.

Do you mean install the release with the lowest version number in a given range? Pip does not have an option for that, though there is this feature request.

[yarikoptic]

yes, that feature request is what I want ;) I guess it would take awhile..

FTR - 0.12.0 datalad is not even pip installable :-/ 0.12.1 is

$> deactivate; py=3; d=venvs/dev$py-min; rm -rf $d; virtualenv --python=python$py $d && source $d/bin/activate && pip3 install  datalad==0.12.0
zsh: command not found: deactivate
created virtual environment CPython3.8.5.final.0-64 in 251ms
  creator CPython3Posix(dest=/tmp/venvs/dev3-min, clear=False, global=False)
  seeder FromAppData(download=False, chardet=latest, progress=latest, distlib=latest, contextlib2=latest, colorama=latest, ipaddr=latest, CacheControl=latest, setuptools=latest, lockfile=latest, six=latest, resolvelib=latest, pkg_resources=latest, toml=latest, certifi=latest, html5lib=latest, wheel=latest, msgpack=latest, pyparsing=latest, distro=latest, webencodings=latest, pep517=latest, pip=latest, urllib3=latest, packaging=latest, appdirs=latest, retrying=latest, idna=latest, requests=latest, via=copy, app_data_dir=/home/yoh/.local/share/virtualenv/seed-app-data/v1.0.1.debian)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
Collecting datalad==0.12.0
  Using cached datalad-0.12.0.tar.gz (1.3 MB)
    ERROR: Command errored out with exit status 1:
     command: /tmp/venvs/dev3-min/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/yoh/.tmp/pip-install-hflk_e22/datalad/setup.py'"'"'; __file__='"'"'/home/yoh/.tmp/pip-install-hflk_e22/datalad/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /home/yoh/.tmp/pip-pip-egg-info-sd8qmud0
         cwd: /home/yoh/.tmp/pip-install-hflk_e22/datalad/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/yoh/.tmp/pip-install-hflk_e22/datalad/setup.py", line 13, in <module>
        from _datalad_build_support.setup import (
    ModuleNotFoundError: No module named '_datalad_build_support'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[yarikoptic]

so next question would be -- what is the easiest way to "shim" setup call without changing setup.py ? ;)