Is reason for lower bound of scipy v1.2.1 known?

[matthewfeickert]

Hi. 👋 In PR #7680 scipy was added as an install requirements for JAX and a lower bound of

https://github.com/google/jax/blob/e0d0be2f1ee916b6b8c21372e14bd8bb2601b5aa/setup.py#L44

was placed on it (all good 👍).

Has scipy v1.2.1 been tested as the lower bound or was there a motivating reason given API or performance changes in that release? Or was this just chosen as a reasonable lower bound for modern SciPy?

If the latter, and a lower bound on scipy was found that passes the tests would the JAX team be interested in a PR to adjust it? Or is this not of interest / more trouble than it is worth to monitor at the moment?

This is a very minor question, which I could of course evaluate by building and testing JAX myself, but I thought I'd ask here first in case there was a known reason why the lower bound was chosen.

I looked through Issues with the search filter is:issue scipy 1.2.1 to see if this has been discussed already, but I couldn't find anything that seemed to be about the lower bound specifically.

[jakevdp]

My recollection is the reason is this: JAX requires Python 3.7 or newer. Explicit support for Python 3.7 was not added in scipy until version 1.3; we pin >=1.2.1 rather than >=1.3.0 because support for that particular scipy version is important to stakeholders.

Additionally, we have functionality in jax.scipy that wraps associated scipy functions; in the past we've had to have compatibility layers in the test suite & wrappers for supporting features & APIs that are not implemented in older scipy versions, and I think part of requiring newer scipy was the desire to drop these.

[matthewfeickert]

My recollection is the reason is this: JAX requires Python 3.7 or newer. Explicit support for Python 3.7 was not added in scipy until version 1.3; we pin >=1.2.1 rather than >=1.3.0 because support for that particular scipy version is important to stakeholders.

My question was more about going lower. I maintain a Python 3.7+ library that has a (tested on Python 3.7) lower bound of scipy v1.1.0 (which is the oldest to have Python 3.7 wheels), which made me curious where the lower bound here came from.

If I can test and find a lower bound that still passes tests (and any lower bound CI that exists), would that be something the team would be willing to accept as a PR?

[matthewfeickert]

I've gone ahead and tested this, and with

$ git diff upstream/main 
diff --git a/setup.py b/setup.py
index e40c7e13..d83489b3 100644
--- a/setup.py
+++ b/setup.py
@@ -41,7 +41,7 @@ setup(
         'absl-py',
         'numpy>=1.18',
         'opt_einsum',
-        'scipy>=1.2.1',
+        'scipy>=1.1.0',  # Oldest scipy with Python 3.7 wheels
         'typing_extensions',
     ],
     extras_require={

and then using the following constraints.txt as a requirements file

numpy==1.18.0
# scipy==1.1.0  # Fails LaxBackedScipyFftTests.testDct_shape=bool[2,5]_n=1_axis=0_norm=ortho 
scipy==1.2.0  # First to pass with numpy v1.18.0
# scipy==1.2.1
# scipy==1.2.2
# scipy==1.2.3
# scipy==1.3.0
# scipy==1.3.1
# scipy==1.3.2
# scipy==1.3.3
# scipy==1.4.0
# scipy==1.4.1
# scipy==1.5.0
# scipy==1.5.1  # First to pass with unrestricted numpy (at time of testing numpy==1.21.4)

I was able to check for CPython 3.7.10 that with taking the lower bound NumPy of numpy==1.18.0 the oldest SciPy that would pass with it was scipy==1.2.0.

$ python -m pip install -r constraints.txt && pytest --exitfirst -n auto tests

As that's just a patch release below the current

https://github.com/google/jax/blob/e0d0be2f1ee916b6b8c21372e14bd8bb2601b5aa/setup.py#L44

I don't see any motivation to change this bound. 👍

So the TL;DR of it is:

Is the reason for lower bound of scipy v1.2.1 known?

• Yes. scipy v1.2.0+ is required to pass the test suite with the lower supported bound of numpy==1.18.0.

Thanks for the fast response and info!