"unable to merge overloads" reported from site-packages

[loganod]

Bug Report

MyPy is reporting problems with dependencies in site-packages.

To Reproduce

To be honest, I'm having trouble reproducing a minimal example I can share. I don't understand why it's reporting on site-packages in the first place.

Expected Behavior

MyPy should only tell me about errors in the code I tell it to check - not it's dependencies.

Actual Behavior

.venv/lib/python3.6/site-packages/ddtrace/internal/utils/__init__.py: error: Source file found twice under different module names: "ddtrace.internal.utils" and "ddtrace.internal.utils.__init__"

Your Environment

• Mypy version used: mypy 0.931
• Mypy command-line flags: just a list of directories (none of which are .venv)
• Mypy configuration options from mypy.ini (and other config files):

follow_imports = "silent"
ignore_missing_imports = true
namespace_packages = true
exclude = "(.venv|...other stuff...)"

• Python version used: 3.6.7
• Operating system and version: OSX Big Sur

[NiklasRosenstein]

I'm experiencing the same issue. My project kraken-wrappr depends on kraken-core which vendors a bunch of libraries, such as dill. Dill is not typed nor are there type stubs available. For Mypy checks in kraken-core, I configure Mypy to ignore the dill package like so:

# pyproject.toml
[[tool.mypy.overrides]]
module = "kraken._vendor.dill.*"
ignore_errors = true
ignore_missing_imports = true

But in the kraken-wrapper package, Mypy spits out an error for kraken._vendor.dill, even though Mypy is only supposed to check the src/ directory of kraken-wrapper.

mypy| Daemon started
mypy| /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/kraken/_vendor/dill/__init__.py:287: note: In module imported here,
mypy| /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/kraken/cli/serialize.py:7: note: ... from here,
mypy| /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/kraken/cli/main.py:81: note: ... from here,
mypy| src/kraken/wrapper/buildenv.py:190: note: ... from here:
mypy| /opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/site-packages/kraken/_vendor/dill/_dill.py:1409: error: Condition can't be inferred, unable to merge overloads  [misc]
mypy|     if PyTextWrapperType:
mypy|     ^

You can find the CI here: https://github.com/kraken-build/kraken-wrapper/runs/7796703778?check_suite_focus=true

And reproduce it locally with

python3 -m venv .venv
source .venv/bin/activate
pip install kraken-core==0.8.0 mypy
echo "from kraken.cli.main import main" > main.py
mypy main.py

Gives you

.venv/lib/python3.10/site-packages/kraken/_vendor/dill/_dill.py:1409: error: Condition can't be inferred, unable to merge overloads
Found 1 error in 1 file (checked 1 source file)

(Mypy 0.971)

Sometimes I also see Mypy checking the site packages as if I passed them as paths to the CLI. I believe I see this mostly with the Mypy daemon though, and restarting it usually fixes it, so it may be slightly unrelated.

Adding this config to kraken-wrapper's pyproject.toml fixes the issue, but I would like to avoid that every consumer of the library has to add this to their config.

[[tool.mypy.overrides]]
module = "kraken._vendor.dill.*"
ignore_errors = true

Interestingly, adding # type: ignore to the top of the vendored _dill.py does not fix it.

[hauntsaninja]

Similar to NiklasRosenstein's example, there's also a case in
#12299 (comment) where we don't hide errors in third party modules that come from fastparse.
(This was fine when these were hard parse errors, but obviously the line has greyed since then)

[parhamfh]

Some anecdotal findings, I hit this issue in a container running my simply Python application in an image based on python:3.10.7-slim-bullseye (fe16e9fa64e9). It failed with this error:

> pipenv run mypy app --ignore-missing-imports --strict
#13 3.564 /root/.local/share/virtualenvs/app-xyz/lib/python3.10/site-packages/numpy/__init__.pyi:636: error: Positional-only parameters are only supported in Python 3.8 and greater

and removing this errror kept spawning further errors in .pyi files in the numpy module.

Changing base image to python:3.10.6-slim-bullseye resolved this issue

mypy version: mypy 0.971 (compiled: yes)

[hauntsaninja]

Technically unrelated to this, see the pinned issue here #13627

[NiklasRosenstein]

I'm encountering this error again, and I think it's because my project is compatible with Python 3.7+, but when it is installed in Python 3.10 it get's a newer version of its markdown-it-py dependency which uses the Walrus operator, hence I get the following in 3.10:

/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/markdown_it/main.py:8: note: In module imported here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/markdown_it/__init__.py:5: note: ... from here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/rich/markdown.py:6: note: ... from here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/rich/__main__.py:8: note: ... from here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/rich/pager.py:29: note: ... from here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/rich/console.py:59: note: ... from here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/httpx/_main.py:10: note: ... from here,
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/httpx/__init__.py:48: note: ... from here,
src/kraken/std/http/__init__.py:8: note: ... from here:
/home/coder/git/kraken/kraken-build/.venvs/3.11/lib/python3.11/site-packages/markdown_it/common/utils.py:106: error: Assignment expressions are only supported in Python 3.8 and greater  [syntax]
        if pat := DIGITAL_ENTITY_BASE10_RE.fullmatch(name):

The following does not fix it:

[[tool.mypy.overrides]]
module = "markdown_it.*"
ignore_errors = true

Pinning markdown-it-py<3.0.0 fixes it because Mypy can no longer complain about its 3.10 syntax.

[hauntsaninja]

Ah that's unfortunate. If your library still supports Python 3.7, one good way to test is to set up your Python 3.7 environment and run mypy --python-executable py37venv/bin/python .... Or since mypy still supports Python 3.7 for now (we'll drop support sometime this year), just install mypy into that environment.

This will work, because pip will avoid installing packages that are unsupported into an environment based on the python_requires metadata. (It might be the case that markdown-it-py doesn't declare python_requires correctly, in which case manually pinning is your only option)