Support metaclasses #2475

elazarg · 2016-11-22T10:50:37Z

Reopening #2392 (itself a reopen of #2365).

Store the type of the metaclass in TypeInfo, and use it when possible, instead of builtins.type.

I assume metaclasses always inherit from type.

This PR will help fixing #2305 and #741, but the fix is not part of this PR.

elazarg · 2016-11-22T11:23:33Z

(No tests yet)

JukkaL · 2016-11-25T13:04:12Z

Looks reasonable -- thanks for working on this! One thing wasn't clear: does this follow the MRO in search of the metaclass?

Also, we may also want to check for incompatible metaclasses in the MRO, but that can be a separate issue and doesn't need to be covered here.

JukkaL · 2016-11-25T13:04:40Z

(I only did a quick pass, not a full review.)

gvanrossum · 2016-12-13T22:03:29Z

Hey @elazarg, are you still planning to work on this?

elazarg · 2016-12-13T22:54:00Z

test-data/unit/semanal-errors.test

 [out]
+main:2: error: Name 'abc.Foo' is not defined
+main:2: error: Invalid metaclass 'abc.Foo'


Which one is preferable?

Given that abc.Foo does not exist, I prefer "Name 'abc.Foo' is not defined" -- if I were to see the other I would assume that there is an abc.Foo that was unacceptable as a metaclass. (Typos and large lists of imports might obscure that it's not defined at all.)

elazarg · 2016-12-13T23:04:35Z

Yes, sorry. Todo:

Add test
Find out why would self.mro be None
Follow mro better

Also, I should look for other places where the fallback is needed, other than for iterable. (I must note that the tests for fallbackness feels awkward and error prone)

elazarg · 2016-12-14T10:15:22Z

I think this is ready for a review.

gvanrossum · 2016-12-14T17:52:33Z

test-data/unit/check-classes.test

+    def __iter__(self) -> Iterator[int]: yield 1
+
+class Good(metaclass=GoodMeta): pass
+for _ in Good: pass


If this works, shouldn't list(C) work too?

I just assumed it will work. Turns out it didn't, but after handling the fallback in the solver it does.

https://docs.python.org/3/reference/datamodel.html#determining-the-appropriate-metaclass

gvanrossum · 2017-02-06T22:45:41Z

mypy/nodes.py

+
+    def calculate_metaclass_type(self) -> 'Optional[mypy.types.Instance]':
+        if self.mro is None:
+            # XXX why does this happen?


Do you have a sample that triggers this condition?

Presumably the clue is the comment in __init__ on line 1940 above. Maybe it has something to do with the MRO being calculated twice -- the first time (in semanal.py on line 850) it may fail if there's an import cycle causing a base class being a forward reference; the second time on line 3128 in ThirdPass, called from build.py line 1440. (There's also a call from fixup.py, but I think that's not what you're seeing.)

Turns out it happens during deserialization in incremental mode: the type of a direct base class may be None. (I should have looked into it long time ago. Sorry)

I think I will remove the call from deserialize, since it shouldn't be done yet - it should be done from fixup.py. Does it sound reasonable?

gvanrossum · 2017-02-06T23:29:18Z

Ah, yes, that sounds right. Also my intuition was 180 degrees off here, sorry! :-)

gvanrossum · 2017-02-06T23:35:42Z

mypy/semanal.py

                self.fail("Invalid metaclass '%s'" % defn.metaclass, defn)
+                return
+            inst = fill_typevars(sym.node)
+            assert isinstance(inst, Instance)


I can trigger this assert as follows:

class M(Tuple[int]): pass class C(metaclass=M): pass

I think it should be an error then. Do you agree? I don't want to go silently to a fallback,

Yes, you can add the check to line 911.

gvanrossum

Could you also look into #1771? Possibly as a follow-up PR?

gvanrossum · 2017-02-06T23:37:50Z

mypy/nodes.py

+        candidates = {s.declared_metaclass for s in self.mro} - {None}
+        for c in candidates:
+            if all(other.type in c.type.mro for other in candidates):
+                return c


How sure are you that the answer does not depend on the order of candidates?

Not at all. Perhaps redundant ~~tests~~ checks are better.

I have replaced the set with a simple list

candidates = [s.declared_metaclass for s in self.mro if s.declared_metaclass is not None]

gvanrossum · 2017-02-06T23:49:29Z

mypy/semanal.py

                self.fail("Invalid metaclass '%s'" % defn.metaclass, defn)
+                return
+            inst = fill_typevars(sym.node)
+            assert isinstance(inst, Instance)


Yes, you can add the check to line 911.

gvanrossum · 2017-02-06T23:52:23Z

test-data/unit/check-classes.test

+
+class A(metaclass=M): pass
+
+reveal_type(A.x)  # E: Revealed type is 'builtins.int'


This line seems redundant, it's the same as in the previous test.

I have removed it in the last commit but it still appears here.

elazarg · 2017-02-07T00:34:15Z

I will try to look into #1771 later. But it feels like it will require a hack (or a very deep change).

gvanrossum · 2017-02-07T01:05:18Z

OK, let's put off #1771 until a separate PR. (That's why I reopened it anyways.) I will look at the rest now.

gvanrossum · 2017-02-07T01:16:28Z

Thanks. Great work!

Fix an issue in #2475 - metaclasses are not calculated for subclasses (this is needed for Enum).

Fix #2818. ABCMeta requires special treatment. Related: #2475

Fix subissue of #2413. Follow-up for #2475 * check for member in fallback * check for fallback only for type objects

elazarg changed the title ~~Support metaclasses~~ [WIP] Support metaclasses Nov 22, 2016

elazarg force-pushed the metaclasses branch from 59e1883 to 8ed204e Compare December 13, 2016 22:44

elazarg commented Dec 13, 2016

View reviewed changes

elazarg changed the title ~~[WIP] Support metaclasses~~ Support metaclasses Dec 14, 2016

gvanrossum reviewed Dec 14, 2016

View reviewed changes

elazarg mentioned this pull request Jan 7, 2017

Add support for __init_subclass__ #2654

Merged

elazarg force-pushed the metaclasses branch from 0089ec7 to a91c0a1 Compare January 9, 2017 23:53

elazarg mentioned this pull request Feb 6, 2017

Allow E[<str>] where E is an Enum type #2812

Merged

elazarg added 15 commits February 6, 2017 19:23

checker

b20cfa4

checkexpr and checkmember

7eee764

nodes

6ab7368

semanal

0d0e9ff

types

41ecc84

testcheck

6dc66b2

fix name lookup and parse errors

314a300

Follow metaclass resollution as described in docs

3a05361

https://docs.python.org/3/reference/datamodel.html#determining-the-appropriate-metaclass

Some metaclass consistency checks

c42f777

Add tests

5de0e80

serialization

cf02b0d

Turns out subtyping is enough for iterable

f90fc80

simplify diff

ac49100

simplify diff cont.

4c70367

fix inference

7876d3f

elazarg force-pushed the metaclasses branch from a91c0a1 to 7876d3f Compare February 6, 2017 17:43

gvanrossum mentioned this pull request Feb 6, 2017

Mypy doesn't recognize metaclass implementing __getitem__ #1771

Closed

gvanrossum reviewed Feb 6, 2017

View reviewed changes

elazarg added 2 commits February 7, 2017 01:45

do not initialize metaclass from deserialize

c256da5

fail for metaclass(Tuple)

d65c3b4

gvanrossum requested changes Feb 6, 2017

View reviewed changes

dont use set. fail for metaclass(Tuple)

b083a97

gvanrossum approved these changes Feb 7, 2017

View reviewed changes

gvanrossum merged commit cab1e0d into python:master Feb 7, 2017

This was referenced Feb 7, 2017

Mypy should look in metaclass for operations on class objects #2413

Closed

MyPy does not consider classes instances of their metaclass #1267

Closed

Metaclasses should inherit from type. #2818

Closed

This was referenced Feb 7, 2017

Calculate implicit metaclasses #2819

Merged

Do not support metaclasses not inheriting from type #2820

Merged

JukkaL pushed a commit that referenced this pull request Feb 7, 2017

Calculate implicit metaclasses (#2819)

b731899

Fix an issue in #2475 - metaclasses are not calculated for subclasses (this is needed for Enum).

elazarg mentioned this pull request Feb 7, 2017

Check for member in type-fallback #2822

Merged

JukkaL pushed a commit that referenced this pull request Feb 7, 2017

Do not support metaclasses not inheriting from type (#2820)

452a64f

Fix #2818. ABCMeta requires special treatment. Related: #2475

JukkaL pushed a commit that referenced this pull request Feb 7, 2017

Check for member in type-fallback (#2822)

76da328

Fix subissue of #2413. Follow-up for #2475 * check for member in fallback * check for fallback only for type objects

TRManderson mentioned this pull request Feb 8, 2017

mypy doesn't understand six.with_metaclass #1764

Closed

gvanrossum mentioned this pull request Mar 8, 2017

Typechecking attrs-generated classes #2088

Closed

raphCode mentioned this pull request Jul 17, 2023

metaclasses implementing __getitem__ work only in reveal_type(), cannot be used anywhere #15107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support metaclasses #2475

Support metaclasses #2475

elazarg commented Nov 22, 2016 •

edited

Loading

elazarg commented Nov 22, 2016

JukkaL commented Nov 25, 2016

JukkaL commented Nov 25, 2016

gvanrossum commented Dec 13, 2016

elazarg Dec 13, 2016 •

edited

Loading

gvanrossum Dec 13, 2016

elazarg commented Dec 13, 2016 •

edited

Loading

elazarg commented Dec 14, 2016

gvanrossum Dec 14, 2016

elazarg Dec 14, 2016

gvanrossum Feb 6, 2017

elazarg Feb 6, 2017

elazarg Feb 6, 2017

gvanrossum commented Feb 6, 2017 via email

gvanrossum Feb 6, 2017

elazarg Feb 6, 2017

gvanrossum Feb 6, 2017

gvanrossum left a comment

gvanrossum Feb 6, 2017

elazarg Feb 7, 2017 •

edited

Loading

elazarg Feb 7, 2017

gvanrossum Feb 6, 2017

gvanrossum Feb 6, 2017

elazarg Feb 7, 2017

elazarg commented Feb 7, 2017

gvanrossum commented Feb 7, 2017 via email

gvanrossum commented Feb 7, 2017


		class A(metaclass=M): pass

		reveal_type(A.x) # E: Revealed type is 'builtins.int'

Support metaclasses #2475

Support metaclasses #2475

Conversation

elazarg commented Nov 22, 2016 • edited Loading

elazarg commented Nov 22, 2016

JukkaL commented Nov 25, 2016

JukkaL commented Nov 25, 2016

gvanrossum commented Dec 13, 2016

elazarg Dec 13, 2016 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elazarg commented Dec 13, 2016 • edited Loading

elazarg commented Dec 14, 2016

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gvanrossum commented Feb 6, 2017 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gvanrossum left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elazarg Feb 7, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elazarg commented Feb 7, 2017

gvanrossum commented Feb 7, 2017 via email

gvanrossum commented Feb 7, 2017

elazarg commented Nov 22, 2016 •

edited

Loading

elazarg Dec 13, 2016 •

edited

Loading

elazarg commented Dec 13, 2016 •

edited

Loading

elazarg Feb 7, 2017 •

edited

Loading