mypy running in 3.11 targeting 3.7 reports syntax errors for positional args in numpy

[tgamblin]

Bug Report

I'm developing in Python 3.11, but my project targets 3.6-3.11, and we set up mypy to target Python 3.7. mypy will fail if I have numpy installed locally.

To Reproduce

> python3 -m venv repro
> cd repro
> . bin/activate
(repro) > pip3 install numpy mypy
Collecting numpy
  Using cached numpy-1.24.1-cp311-cp311-macosx_11_0_arm64.whl (13.8 MB)
Collecting mypy
  Using cached mypy-0.991-cp311-cp311-macosx_11_0_arm64.whl (9.9 MB)
Collecting typing-extensions>=3.10
  Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
Collecting mypy-extensions>=0.4.3
  Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Installing collected packages: mypy-extensions, typing-extensions, numpy, mypy
Successfully installed mypy-0.991 mypy-extensions-0.4.3 numpy-1.24.1 typing-extensions-4.4.0

(repro) > echo "import numpy" > test.py
(repro) > mypy --python-version=3.7 test.py
lib/python3.11/site-packages/numpy/__init__.pyi:641: error: Positional-only parameters are only supported in Python 3.8 and greater  [syntax]
Found 1 error in 1 file (errors prevented further checking)

I actually set this up with pyproject.toml and use a Spack environment, but I simplified things with a CLI arg for the reproducer.

Expected Behavior

I expected not to get the syntax error, as I don't want to limit what developers have installed in their environment outside my project.

This seems almost like a duplicate of #13639, which was fixed in .980, but I'm targeting 3.7 explicitly and they weren't. So maybe my understanding is incorrect, and if I target 3.7 I'm supposed to get this error. But I feel like I should be able to develop with a newer python and have dev tools target an older one without this issue.

Actual Behavior

Got this error:

lib/python3.11/site-packages/numpy/__init__.pyi:641: error: Positional-only parameters are only supported in Python 3.8 and greater  [syntax]

Your Environment

• Mypy version used: 0.991
• Mypy command-line flags: --python-version=3.7
• Mypy configuration options from mypy.ini (and other config files):

[tool.mypy]
python_version = 3.7. # changing to 3.8 fixes this but I want to target 3.7

• Python version used: 3.11.0

[hauntsaninja]

Recent versions of numpy require Python 3.8 or newer

https://github.com/numpy/numpy/blob/68d7aadd30cb7a4f6f32e76715b38b644df18602/setup.py#L435

[tgamblin]

My project doesn’t actually use numpy; numpy just happens to be installed in the python (3.11) I’m using for development.

Grepping around, though, my project does import pytest, which can import numpy (optionally, from within a function).

Does mypy have to statically follow all imports? Is there a way to force it to skip numpy?

[hauntsaninja]

You can add a per-module follow imports setting. Something like the following in a mypy.ini:

[mypy-numpy.*]
follow_imports = skip

[tgamblin]

Awesome -- I was able to fix this with the following in pyproject.toml:

  [[tool.mypy.overrides]]
  module = 'pytest'
  follow_imports = 'skip'

If I tell it to skip numpy, it does not work because pytest has the import for numpy, and the stub seems to be read there. Thanks! I'll just close this -- as I think my particular issue is resolved.

I wonder whether, for larger projects, it would make sense to have a follow imports setting to specify a "core" set of packages (i.e. the ones I'm actively developing) and by default only follow imports from them. e.g.:

[tool.mypy]
only_follow_imports_from = ['mypackage.*', 'othermodule']

This would get you .pyi's, etc. from direct dependencies but not from transitive dependencies of your project. I could imagine it eventually being hard to track down which packages to skip on larger projects.

[hauntsaninja]

Okay, checked with my laptop. Since numpy ships stubs, apparently you also need to set follow_imports_for_stubs, so mypy.ini would contain:

[mypy-numpy.*]
follow_imports = skip
follow_imports_for_stubs = true