Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot assign to field of Callable type #708

Closed
spkersten opened this issue Jun 6, 2015 · 48 comments · Fixed by #10548 or #13400
Closed

Cannot assign to field of Callable type #708

spkersten opened this issue Jun 6, 2015 · 48 comments · Fixed by #10548 or #13400
Labels
affects-typeshed Anything that blocks a typeshed change bug mypy got something wrong false-positive mypy gave an error on correct code priority-0-high size-large topic-descriptors Properties, class vs. instance attributes

Comments

@spkersten
Copy link
Contributor

Consider this example:

from typing import Callable

class A:
    f = None  # type: Callable[[int], int]

    def __init__(self, g: Callable[[int], int]) -> None:
        self.f = g

mypy reports:

trial.py: In member "__init__" of class "A":
trial.py, line 7: Cannot assign to a method
trial.py, line 7: Invalid method type
trial.py, line 7: Incompatible types in assignment (expression has type Callable[[int], int], variable has type Callable[[], int])

while I'd expect this to work.

Note that there is a second error in the reporting of the type of the variable.

Also, this passes type checking (correctly):

f = None  # type: Callable[[int], int]

def g(x: int) -> int: return 0

f = g
@spkersten
Copy link
Contributor Author

The mismatching types is probably because mypy thinks f is a method and therefore strips off the first argument (self).

@JukkaL JukkaL added bug mypy got something wrong priority labels Jun 7, 2015
@JukkaL
Copy link
Collaborator

JukkaL commented Jun 7, 2015

Hmm.. actually not sure what's the right thing to do here. Based on a quick experiment, the desired behavior seems to depend on what kind of a callable g is in your original example.

Consider this:

class A:
    f = None
def m(x):
    return x + 1
print(m(1))  # 2
A.f = m
print(A().f(1))  # Error (2 args given, expects 1)

But now look at this:

class A:
    f = None
class M:
    def __call__(self, x):
        return x + 1
m = M()
print(m(1) # 2
A.f = m
print(A().f(1))  # Okay => 2

@JukkaL JukkaL removed the priority label Jun 7, 2015
@JukkaL
Copy link
Collaborator

JukkaL commented Jun 7, 2015

Taking off priority label because the fix doesn't seem obvious.

@spkersten
Copy link
Contributor Author

Before reporting it I also thought for a while that I could fix it quickly 😄.

@tharvik
Copy link
Contributor

tharvik commented Jul 29, 2016

Is there any update? The issue is still present. For some real world example, it is used in the ctypes module (in the examples).

from typing import Any, Callable

class A():
    a = lambda: None  # type: Callable[..., None]

def f() -> None: return None

A().a = f

gives

/tmp/assign_callable.py:8: error: Cannot assign to a method
/tmp/assign_callable.py:8: error: Incompatible types in assignment (expression has type Callable[[], None], variable has type Callable[..., None])

Having Callable[..., None] or Callable[[], None] do not resolve it, having Callable[..., Any] remove the incompatible types line.

@gvanrossum
Copy link
Member

I'm sorry, I don't have any further updates.

@gnprice
Copy link
Collaborator

gnprice commented Aug 4, 2016

Thanks for bumping this with the new repro!

It's possible these two errors are different issues. The second looks like it may be a case of the Void/NoneTyp distinction (recently eliminated in our --strict-optional future: #1975). Would you try reproducing it on master and with --strict-optional?

The first error looks like the same thing as the original issue.

@gnprice
Copy link
Collaborator

gnprice commented Aug 4, 2016

@tharvik, would you also link more specifically to the bit in ctypes that refers to this? It's good to know about that use case, and it'd be interesting to see it more concretely.

@gnprice gnprice added this to the Undetermined priority milestone Aug 4, 2016
@tharvik
Copy link
Contributor

tharvik commented Aug 5, 2016

Seems that I'm unable to reproduce 'Incompatible types in assignment' on master with or without --strict-optional.

@gnprice in ctypes, when declaring a foreign function, one can redefine errcheck, which should be a callable. It allows processing of the return value (and raising on failure).

Example from the documentation

def errcheck(result, func, args):
    if not result:
        raise WinError()
     return args

GetWindowRect.errcheck = errcheck

This way, we can load a library, in which, every attribute is a foreign function, then assign to some an errcheck for freeing some pointer or other.

For another example, here goes libLAS, where the library is loaded then it is assigned some checks.

@gvanrossum
Copy link
Member

gvanrossum commented Aug 5, 2016 via email

@Stiivi
Copy link

Stiivi commented Feb 3, 2017

Just for the completeness of cases: same error with Python 3.6 variable annotations:

MyFunction = Callable[[int], int]                     
                                                      
class Thing:                                          
    function: MyFunction                              
                                                      
    def __init__(self, function: MyFunction) -> None: 
        self.function = function                      

@hendi
Copy link

hendi commented Feb 24, 2017

This affects basically every Django app that uses django.contrib.admin. The following is an often-used pattern there:

class FooAdmin(admin.ModelAdmin):
    list_display = ("id", "custom")

    def custom(self, obj):
        return obj.bar
    custom.short_description = "My Custom Field"  # error in mypy

@gvanrossum
Copy link
Member

gvanrossum commented Feb 24, 2017 via email

@gvanrossum gvanrossum removed this from the 0.5 milestone Mar 29, 2017
@pkch
Copy link
Contributor

pkch commented Apr 7, 2017

Just an example I ran into today:

from collections import defaultdict
d: DefaultDict[int, int] = defaultdict(int)

d.default_factory = None

fails with --strict-optional with 3 errors, and fails without --strict-optional with 2 errors.

When I changed typeshed to make default_factory an Optional[Callable...], --strict-optional type check passed without errors, and --no-strict-optional was unaffected.

@gvanrossum
Copy link
Member

To reiterate, assigning a callable to a class variable is fraught with problems in Python itself too. The reason is descriptors (things with a __get__ method) -- if the object is a descriptor then its __get__ method will be called when you use e.g. C.f so what you get back is a bound method. (Regular function objects defined with def or lambda are descriptors that work this way, but builtin functions and other callables typucally don't).

We should really teach mypy about all this (after all it now does know about descriptors!) but that would be a complicated change.

@vegolasse
Copy link

Being a bit of a Python beginner and also ran into this which might or might not have more implications than I realise.

My workaround (since all I wanted was a "selfless" function pointer as a member variable) was to put the Callable in a List like: _get_time_ms: List[Callable[[], int]]. It means an extra list lookup every use, but is in my case negligible and the type check is worth more.

@gvanrossum
Copy link
Member

gvanrossum commented Jul 11, 2017 via email

miketheman added a commit to miketheman/warehouse that referenced this issue Mar 7, 2022
Callables are receiving dynamic attributes, something that isn't "usual".

See python/mypy#708

Signed-off-by: Mike Fiedler <[email protected]>
miketheman added a commit to miketheman/warehouse that referenced this issue Mar 7, 2022
Callables are receiving dynamic attributes, something that isn't "usual".

See python/mypy#708

Signed-off-by: Mike Fiedler <[email protected]>
di pushed a commit to pypi/warehouse that referenced this issue Mar 9, 2022
* chore(deps): install mypy in lint.in

Signed-off-by: Mike Fiedler <[email protected]>

* chore(deps): include more types-* packages for mypy

These were suggested from running mypy on the codebase.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: configure mypy

Set configuration for mypy.

Exclude some of the subdirectories we are not interested in testing to
speed up mypy execution.

Ignore any 3rd party modules that we do not have types for yet.
Added links that I could find to help track completion.

Does **not** set `strict` mode yet, since that's a bigger lift.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: add mypy runner script

Eventually this command should fold into `bin/lint` and be removed.

For now, it's a convenient execution wrapper.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore dynamic properties

Callables are receiving dynamic attributes, something that isn't "usual".

See python/mypy#708

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore lambdas

See python/mypy#4226

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore hybrid_property repeat definitions

Part of the SQLAlchemy extensions, which do not yet have reliable
stubs/plugins.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore sqlalchemy declarative

Should come along with sqlalchemy stubs.

See: https://docs.sqlalchemy.org/en/14/orm/extensions/mypy.html#using-declared-attr-and-declarative-mixins

Signed-off-by: Mike Fiedler <[email protected]>

* lint: a few more ignores

Signed-off-by: Mike Fiedler <[email protected]>

* lint: interface methods shouldn't use self

Surfaced via mypy, corrected!

Signed-off-by: Mike Fiedler <[email protected]>

* lint: correct subclass path

Surfaced via mypy, corrected.

Unclear why this wouldn't have been caught by other tools.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: rename internal variable

mypy detected this as a type mismatch, as the internal variable name was
shadowing the externally-supplied one, and changing the type.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore flake8 line too long for ignored types

Adding a `# type: ignore` comment to a couple of places triggered
flake8's line too long check.

Running `make reformat` did nothing for these - black has outstanding
design issues with line length and comments.

See psf/black#1713 for one example.

Instead of changing the line structure to accommodate, ignore these two
cases, at least until the types can be fixed and the comments removed.

Signed-off-by: Mike Fiedler <[email protected]>

* Revert "chore: add mypy runner script"

This reverts commit fffeadb.

* test: include mypy in lint execution

Signed-off-by: Mike Fiedler <[email protected]>

* chore(deps): include itsdangerous type stubs

Until itsdangerous 2.0 is included, this types package is needed.

Signed-off-by: Mike Fiedler <[email protected]>
@JelleZijlstra JelleZijlstra added the topic-descriptors Properties, class vs. instance attributes label Mar 19, 2022
@AlexWaygood AlexWaygood added the affects-typeshed Anything that blocks a typeshed change label Mar 31, 2022
@AlexWaygood
Copy link
Member

This issue was recently discussed in-depth in python/typing#1113, where it was noted that mypy's behaviour differs from most other type-checkers

@altendky
Copy link

altendky commented May 2, 2022

It seems that hinting with a callable protocol instead of Callable manages to evade mypy's method detection.

https://mypy-play.net/?mypy=latest&python=3.10&gist=389004dcf62cd710bec35920fdce4b68

import typing


class CallableProtocol(typing.Protocol):
    def __call__(self) -> None:
        pass


def a() -> None:
    return

TypingCallable = typing.Callable[[], None]

class C:
    f: TypingCallable
    g: CallableProtocol
    
    def __init__(self, f: TypingCallable, g: CallableProtocol) -> None:
        self.f = f
        self.g = g

    def call(self) -> None:
        self.f()
        self.g()
main.py:19: error: Cannot assign to a method
main.py:19: error: Attribute function "f" with type "Callable[[], None]" does not accept self argument
main.py:23: error: Attribute function "f" with type "Callable[[], None]" does not accept self argument
Found 3 errors in 1 file (checked 1 source file)

youtux pushed a commit to youtux/types-factory-boy that referenced this issue May 28, 2022
domdfcoding pushed a commit to domdfcoding/warehouse that referenced this issue Jun 7, 2022
* chore(deps): install mypy in lint.in

Signed-off-by: Mike Fiedler <[email protected]>

* chore(deps): include more types-* packages for mypy

These were suggested from running mypy on the codebase.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: configure mypy

Set configuration for mypy.

Exclude some of the subdirectories we are not interested in testing to
speed up mypy execution.

Ignore any 3rd party modules that we do not have types for yet.
Added links that I could find to help track completion.

Does **not** set `strict` mode yet, since that's a bigger lift.

Signed-off-by: Mike Fiedler <[email protected]>

* chore: add mypy runner script

Eventually this command should fold into `bin/lint` and be removed.

For now, it's a convenient execution wrapper.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore dynamic properties

Callables are receiving dynamic attributes, something that isn't "usual".

See python/mypy#708

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore lambdas

See python/mypy#4226

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore hybrid_property repeat definitions

Part of the SQLAlchemy extensions, which do not yet have reliable
stubs/plugins.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore sqlalchemy declarative

Should come along with sqlalchemy stubs.

See: https://docs.sqlalchemy.org/en/14/orm/extensions/mypy.html#using-declared-attr-and-declarative-mixins

Signed-off-by: Mike Fiedler <[email protected]>

* lint: a few more ignores

Signed-off-by: Mike Fiedler <[email protected]>

* lint: interface methods shouldn't use self

Surfaced via mypy, corrected!

Signed-off-by: Mike Fiedler <[email protected]>

* lint: correct subclass path

Surfaced via mypy, corrected.

Unclear why this wouldn't have been caught by other tools.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: rename internal variable

mypy detected this as a type mismatch, as the internal variable name was
shadowing the externally-supplied one, and changing the type.

Signed-off-by: Mike Fiedler <[email protected]>

* lint: ignore flake8 line too long for ignored types

Adding a `# type: ignore` comment to a couple of places triggered
flake8's line too long check.

Running `make reformat` did nothing for these - black has outstanding
design issues with line length and comments.

See psf/black#1713 for one example.

Instead of changing the line structure to accommodate, ignore these two
cases, at least until the types can be fixed and the comments removed.

Signed-off-by: Mike Fiedler <[email protected]>

* Revert "chore: add mypy runner script"

This reverts commit fffeadb.

* test: include mypy in lint execution

Signed-off-by: Mike Fiedler <[email protected]>

* chore(deps): include itsdangerous type stubs

Until itsdangerous 2.0 is included, this types package is needed.

Signed-off-by: Mike Fiedler <[email protected]>
deeplow added a commit to freedomofpress/dangerzone that referenced this issue Aug 4, 2022
ignore method assignment. Currently mypy cannot check this.
Related upstream issues:
  - python/mypy#2427
  - python/mypy#708
ilevkivskyi pushed a commit to ilevkivskyi/mypy that referenced this issue Aug 12, 2022
Fixes python#708 and Fixes python#5485

Prevent handling as bounded method of callable members declared as instance variables.
ilevkivskyi added a commit that referenced this issue Aug 13, 2022
Fixes #708
Fixes #5485

This builds on the original proposal, but handles three important issues/edge cases:
* This PR fixes serialization of `is_inferred` so that the distinction works correctly in incremental mode (I added a test)
* Dunder operator methods are always considered class variables (this is a relatively common pattern and matches Python semantics; there is an existing tests that previously needed `ClassVar[...]`)
* If we detect a `Too few arguments` error for a variable with callable type we give a note suggesting to try `ClassVar[...]`

I also add a short doc paragraph on this.

Co-authored-by: wyfo <[email protected]>
deeplow added a commit to freedomofpress/dangerzone that referenced this issue Aug 16, 2022
ignore method assignment. Currently mypy cannot check this.
Related upstream issues:
  - python/mypy#2427
  - python/mypy#708
deeplow added a commit to freedomofpress/dangerzone that referenced this issue Aug 16, 2022
ignore method assignment. Currently mypy cannot check this.
Related upstream issues:
  - python/mypy#2427
  - python/mypy#708
deeplow added a commit to freedomofpress/dangerzone that referenced this issue Aug 16, 2022
ignore method assignment. Currently mypy cannot check this.
Related upstream issues:
  - python/mypy#2427
  - python/mypy#708
deeplow added a commit to freedomofpress/dangerzone that referenced this issue Aug 17, 2022
ignore method assignment. Currently mypy cannot check this.
Related upstream issues:
  - python/mypy#2427
  - python/mypy#708
deeplow added a commit to freedomofpress/dangerzone that referenced this issue Aug 22, 2022
ignore method assignment. Currently mypy cannot check this.
Related upstream issues:
  - python/mypy#2427
  - python/mypy#708
zanieb added a commit to zanieb/poetry-relax that referenced this issue Mar 24, 2023
zanieb added a commit to zanieb/poetry-relax that referenced this issue Mar 24, 2023
@kerimovkhikmet
Copy link

There was issue #5062, which was closed as a duplicate in favor of this issue, which is shown as fixed. However, I am currently using Python 3.11.4 with mypy 1.4.1 and still getting the errors with ConfigParser, described in this comment - #5062 (comment). Is this expected, or duplicate issue was closed by mistake, and the error with ConfigParser is not fixed?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-typeshed Anything that blocks a typeshed change bug mypy got something wrong false-positive mypy gave an error on correct code priority-0-high size-large topic-descriptors Properties, class vs. instance attributes
Projects
None yet