Custom managers broken with mypy 1.5.0

[andersk]

Bug report

What's wrong

Mypy 1.5.0 was just released and breaks lots of django-stubs tests. Obviously that’s semi-expected since 1.5.0 is outside the django-stubs[compatible-mypy] bound of mypy==1.4.*. But upon investigating anyway, I think mypy is correct that there’s a django-stubs problem here. The failures began with

• Improve handling of attribute access on class objects python/mypy#14988

which fixes mypy to understand that metaclass attributes take precedence over attributes defined in regular classes, matching Python’s runtime behavior. Unfortunately, django-stubs was relying on the previous misunderstanding:

django-stubs/django-stubs/db/models/base.pyi

Lines 21 to 29 in 325006c

	class ModelBase(type):
	@property
	def objects(cls: type[_Self]) -> BaseManager[_Self]: ... # type: ignore[misc]
	@property
	def _default_manager(cls: type[_Self]) -> BaseManager[_Self]: ... # type: ignore[misc]
	@property
	def _base_manager(cls: type[_Self]) -> BaseManager[_Self]: ... # type: ignore[misc]
	class Model(metaclass=ModelBase):

django-stubs/django-stubs/contrib/contenttypes/models.pyi

Lines 14 to 18 in 325006c

	class ContentType(models.Model):
	id: int
	app_label: models.CharField
	model: models.CharField
	objects: ContentTypeManager

mypy 1.4.1 thinks ContentType.objects has type ContentTypeManager, but mypy 1.5.0 thinks ContentType.objects has type BaseManager[ContentType] from the metaclass.

This affects real projects too, such as Zulip (after bumping its pinned version of mypy):

corporate/models.py:80: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
zerver/actions/invites.py:458: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
zerver/actions/invites.py:469: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
zerver/tests/test_users.py:769: error: "BaseManager[ContentType]" has no attribute "clear_cache"  [attr-defined]
zerver/tests/test_signup.py:929: error: "BaseManager[ContentType]" has no attribute "clear_cache"  [attr-defined]
corporate/views/webhook.py:70: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
corporate/views/webhook.py:85: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]
corporate/views/webhook.py:100: error: "BaseManager[ContentType]" has no attribute "get_for_model"  [attr-defined]

How is that should be

We need to somehow convince mypy 1.5.0 that ContentType.objects has type ContentTypeManager.

System information

• OS: Linux (NixOS 23.11 x86-64)
• python version: 3.10.2
• django version: 4.2.4
• mypy version: 1.5.0
• django-stubs version: 4.2.3
• django-stubs-ext version: 4.2.2

[adambirds]

@andersk I'm getting this error which i think is caused by the same thing:

error: "BaseManager[User]" has no attribute "normalize_email"  [attr-defined]

so happy to help test any fix when its available as would like to switch to mypy 1.5.0 when possible.

[andersk]

#1649 seems to fix most of the failures.

[adambirds]

@andersk just installed from that PR and can confirm it fixed my error. Had no other errors switching to 1.5.0. Thanks

[orsinium]

confirmed to, with the fix from #1649 there are no other errors switching to mypy 1.5.0.

[johnthagen]

Related symptom from

• False positive "BaseManager[User]" has no attribute "create_user" on default User Model #1654

from django.contrib.auth.models import User

user = User.objects.create_user(username="user")

Generates a type checking error on Mypy 1.5.0:

main.py:3: error: "BaseManager[User]" has no attribute "create_user"  [attr-defined]

[tylerlaprade]

I just started seeing this error from Pyright (only in CI). I assume it's due to their latest release fixing the same bug that mypy fixed a couple weeks ago. Is there something I can do with my django-stubs usage to avoid this error? For now I'm just pinning my Pyright version in the GitHub Action, but then I miss out on other bugfixes.

[sobolevn]

@tylerlaprade use django-stubs[compatible-mypy]

[tylerlaprade]

@sobolevn, thanks for the response! However, my error is coming from Pyright, not from mypy, so it still persists even after I made that change.

[orsinium]

I don't think django-stubs is tested to work with pyright. I mean, it works with pyright in vscode for the purpose of autocomplete, but there are no tests to ensure that pyright works as a linter. Especially considering that django-stubs provides a mypy plugin to infer quite a lot of useful stuff but pyright does not support plugins and never going to.

I'd suggest using a more mature and reliable mypy for type checking on CI. If it's, for some reason, not an option for your project, take a look at django-types, a community fork specifically designed to be compatible with pyright.