Question about using zope with PyCharm and typehints?

[kevinmu]

Consider the following python 3.6 script that utilizes both zope and the builtin property decorator:

from zope.cachedescriptors import property as zproperty
import math

class Point:
    def __init__(self, x, y):
        self.x, self.y = x, y

    @property
    def builtin_radius(self) -> int:
        print('computing builtin radius')
        return math.sqrt(self.x**2 + self.y**2)

    @zproperty.readproperty
    def zope_radius(self) -> int:
        print('computing zope radius')
        return math.sqrt(self.x**2 + self.y**2)

if __name__ == "__main__":
    point = Point(1, 2);
    print(point.builtin_radius)
    print(point.zope_radius)

It seems that PyCharm is able to detect that point.buildin_radius has type int, but when writing point.zope_radius, PyCharm stll thinks that the type is Callable[[], int]. The same is true for property.cachedIn and property.CachedProperty.

I'm wondering if there is a way to work around this and get the correct typehints in Pycharm? I'm asking here because I don't /think/ there are any settings in Pycharm that can be changed to get the desired behavior; I believe it the change would have to come from the code. Thanks, really appreciate your help!

[d-maurer]

Kevin Mu wrote at 2020-5-13 21:28 -0700:

Consider the following python 3.6 script that utilizes both zope and the builtin property decorator: ``` from zope.cachedescriptors import property as zproperty import math class Point: def __init__(self, x, y): self.x, self.y = x, y @Property def builtin_radius(self) -> int: print('computing builtin radius') return math.sqrt(self.x**2 + self.y**2) @zproperty.readproperty def zope_radius(self) -> int: print('computing zope radius') return math.sqrt(self.x**2 + self.y**2) if __name__ == "__main__": point = Point(1, 2); print(point.builtin_radius) print(point.zope_radius) ``` It seems that PyCharm is able to detect that `point.buildin_radius` has type int, but when writing `point.zope_radius`, PyCharm stll thinks that the type is `Callable[[], int]`. The same is true for `property.cachedIn` and `property.CachedProperty`. I'm wondering if there is a way to work around this? Would something like the following snippet work, or would that change the behavior of zope? ``` @Property @zproperty.readproperty def zope_radius(self) -> int: print('computing zope radius') return math.sqrt(self.x**2 + self.y**2) ``` Appreciate your help!

Most decorators wrap a function and return typically another function with the same signature; this means, that usually the type hint for a function can be reused for the wrapped function. `readproperty` is an exception: it turns a function into a descriptor with a different signature. Apparently, `zope.cachedescriptors` does not yet take care of this. I think the best way would be a new version of `zope.cachedescriptor`s which handles type hints correctly - emulating logic from `property`. I do not think that `@property @readproperty` will work: it may get the type hint right but it will likely fail when executed: `@property` expects a function but `@readproperty` will result in an atom. Of course, you can define a (new) decorator which applies the type hint related logic of `property` to fix up the wrong type hint produced by `readproperty`.

[kevinmu]

I see, thanks @d-maurer ; appreciate the quick response! Yeah, I kind of quickly realized the @property @readproperty wouldn't work correctly :( I'm currently trying to streamline my project's use of zope cachedescriptors, so I had a few other tangential questions that I'd love your input on if you have the context!

1. assuming we did not need the ability to set the attributes directly, is @readproperty otherwise equivalent to the Python builtin @property decorator?

2. If a @property.CachedProperty has no dependencies, is it roughly equivalent to a @property.Lazy? i.e., it will never get re-computed unless cleared manually?

3. is @property.cachedIn(<exact_field_name>) equivalent to @property.Lazy?

4. I'm also investigating the @cached_property decorator from https://docs.python.org/dev/library/functools.html?highlight=s because my project is python 3.8-compatible. I'm wondering if this is roughly equivalent to @property.Lazy? It seems like it can be cleared manually via delattr as well.

Thanks again for your help! :)

[jamadden]

I think the best way would be a new version of zope.cachedescriptors which handles type hints correctly - emulating logic from property….you can define a (new) decorator which applies
the type hint related logic of property

What logic would that be? I was under the impression that the builtin @property decorator was simply special cased by IDEs. There's nothing in the standard descriptor protocol that has anything to do with type hints.

All of the property-like descriptors defined here already use functools.update_wrapper to copy as much information from the underlying function as the standard library allows.

[jamadden]

assuming we did not need the ability to set the attributes directly, is @readproperty otherwise equivalent to the Python builtin @property decorator?

Functionally, more-or-less. That's the distinction between the two.

If a @property.CachedProperty has no dependencies, is it roughly equivalent to a @property.Lazy? i.e., it will never get re-computed unless cleared manually?

Close. The distinction is that the code in Lazy will only run once; after that first time, the value will be read directly from the instance __dict__ by the internals of Python without ever invoking Lazy. However, even if there are no dependencies, the code in CachedProperty will have to run each and every time to determine that before returning the answer. (Both function like @readproperty in that values can be assigned directly, too, which will bypass the decorator code.)

is @property.cachedIn(<exact_field_name>) equivalent to @property.Lazy?

No, not even close. cachedIn gets turned into a regular @property, meaning it can't be directly set (although it can be written at <exact_field_name>).

Some of the above should probably more clearly be spelled out in the documentation.

I'm also investigating the @cached_property decorator...because my project is python 3.8-compatible. I'm wondering if this is roughly equivalent to @property.Lazy…

Maybe? I haven't looked into how it works. But it's unlikely to use _v attributes and thus unlikely to do the right thing with Persistent objects, if you're using those.

[d-maurer]

Jason Madden wrote at 2020-5-14 03:26 -0700:

> I think the best way would be a new version of `zope.cachedescriptor`s which handles type hints correctly - emulating logic from `property`….you can define a (new) decorator which applies the type hint related logic of `property` What logic would that be? I was under the impression that the builtin ***@***.***` decorator was simply special cased by IDEs. There's nothing in [the standard descriptor protocol](https://docs.python.org/3/reference/datamodel.html#implementing-descriptors) that has anything to do with type hints.

I do not know. I have just interpreted the reported observations that "pyCharm" reports the correct type hint for `@property`. If the logic is in the IDE rather than `property`, then this logic should be emulated by the `property` like decorators of `zope.cachedecriptors`.

All of the property-like descriptors defined here already use `functools.update_wrapper` to copy as much information from the underlying function as the standard library allows.

(As I wrote), this is approriate for decorators which do not change the signature; it is not appropriate for `property` and friends (which change the signature).

[jamadden]

If the logic is in the IDE rather than property, then this logic should be emulated by the property like decorators of zope.cachedecriptors.

Based on this (very long) thread, mypy has special cases for @property, exactly. There's a lot of work to make that much happen, and apparently its still somewhat fragile. GVR wrote back in December:

The problem is that property (both get and set functionality) is special-cased in mypy, and currently doesn't support additional decorators at all. Sorry!

I continue to imagine that other type-checkers and static analysis tools are the same.

I don't think there's anything that this package can do, the individual type-checkers will have to get full-featured descriptor protocol support.

[icemac]

As said by @jamadden: This cannot be fixed in this package, so closing the issue.

[jamadden]

FWIW, this is the configuration snippet I use to get pylint to treat the cachedescriptors as properties:

[MASTER]
# Fix zope.cachedescriptors.property.Lazy; the property-classes doesn't seem to
# do anything.
# https://stackoverflow.com/questions/51160955/pylint-how-to-specify-a-self-defined-property-decorator-with-property-classes
# For releases prior to 2.14.2, this needs to be a one-line, quoted string. After that,
# a multi-line string.
# - Make zope.cachedescriptors.property.Lazy look like a property;
#   fixes pylint thinking it is a method.
# - Run in Pure Python mode (ignore C extensions that respect this);
#   fixes some issues with zope.interface, like IFoo.providedby(ob)
#   claiming not to have the right number of parameters...except no, it does not.
init-hook =
    import astroid.bases
    astroid.bases.POSSIBLE_PROPERTIES.add('Lazy')
    astroid.bases.POSSIBLE_PROPERTIES.add('readproperty')
    astroid.bases.POSSIBLE_PROPERTIES.add('non_overridable')
    import os
    os.environ['PURE_PYTHON'] = ("1")
    # Ending on a quoted string
    # breaks pylint 2.14.5 (it strips the trailing quote. This is
    # probably because it tries to handle one-line quoted strings as well as multi-blocks).
    # The parens around it fix the issue.