Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Core dumped when using JSON.loads with a recursive type #13795

Closed
mousetail opened this issue Oct 3, 2022 · 4 comments · Fixed by #13808
Closed

Core dumped when using JSON.loads with a recursive type #13795

mousetail opened this issue Oct 3, 2022 · 4 comments · Fixed by #13808
Labels
crash topic-recursive-types

Comments

@mousetail
Copy link

Crash Report

~/censored/mypytest $ mypy . --enable-recursive-aliases --show-traceback                                                                                                                     SEGV ✘ 
zsh: segmentation fault (core dumped)  mypy . --enable-recursive-aliases --show-traceback

To Reproduce

My code:

import typing
import json

Tree = typing.Union[str, tuple['Tree', 'Tree']]

def a(b: Tree) -> dict[str, Tree]:
   return json.loads('{}')

mypy 0.981 using python 3.10.6 on linux x64

Your Environment

  • Mypy version used: 0.981
  • Mypy command-line flags: --enable-recurisve-aliases
  • Mypy configuration options from mypy.ini (and other config files): None
  • Python version used: 3.10.6
  • Operating system and version:
Operating System: Manjaro Linux
KDE Plasma Version: 5.25.5
KDE Frameworks Version: 5.97.0
Qt Version: 5.15.5
Kernel Version: 5.15.65-1-MANJARO (64-bit)
Graphics Platform: X11
Processors: 12 × Intel® Core™ i7-8700 CPU @ 3.20GHz
Graphics Processor: Mesa Intel® UHD Graphics 630
Manufacturer: Dell Inc.
@mousetail mousetail added the crash label Oct 3, 2022
@JelleZijlstra
Copy link
Member

cc @ilevkivskyi

@hauntsaninja
Copy link
Collaborator

Thanks for the report!

  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 390, in _is_subtype
    return is_proper_subtype(left, right, subtype_context=self.subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 301, in _is_subtype
    is_subtype_of_item = any(
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 302, in <genexpr>
    check_item(orig_left, item, subtype_context) for item in right.items
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 294, in check_item
    return is_proper_subtype(left, right, subtype_context=subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2169, in accept
    return visitor.visit_tuple_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 714, in visit_tuple_type
    elif self._is_subtype(mypy.typeops.tuple_fallback(left), right):
  File "/Users/shantanu/dev/mypy/mypy/typeops.py", line 108, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 664, in join_type_list
    joined = join_types(joined, t)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 239, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 257, in visit_union_type
    if is_proper_subtype(self.s, t):
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 390, in _is_subtype
    return is_proper_subtype(left, right, subtype_context=self.subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 301, in _is_subtype
    is_subtype_of_item = any(
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 302, in <genexpr>
    check_item(orig_left, item, subtype_context) for item in right.items
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 294, in check_item
    return is_proper_subtype(left, right, subtype_context=subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2169, in accept
    return visitor.visit_tuple_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 714, in visit_tuple_type
    elif self._is_subtype(mypy.typeops.tuple_fallback(left), right):
  File "/Users/shantanu/dev/mypy/mypy/typeops.py", line 108, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 664, in join_type_list
    joined = join_types(joined, t)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 239, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 257, in visit_union_type
    if is_proper_subtype(self.s, t):
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 390, in _is_subtype
    return is_proper_subtype(left, right, subtype_context=self.subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 301, in _is_subtype
    is_subtype_of_item = any(
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 302, in <genexpr>
    check_item(orig_left, item, subtype_context) for item in right.items
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 294, in check_item
    return is_proper_subtype(left, right, subtype_context=subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2169, in accept
    return visitor.visit_tuple_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 714, in visit_tuple_type
    elif self._is_subtype(mypy.typeops.tuple_fallback(left), right):
  File "/Users/shantanu/dev/mypy/mypy/typeops.py", line 108, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 664, in join_type_list
    joined = join_types(joined, t)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 239, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 257, in visit_union_type
    if is_proper_subtype(self.s, t):
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 390, in _is_subtype
    return is_proper_subtype(left, right, subtype_context=self.subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 301, in _is_subtype
    is_subtype_of_item = any(
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 302, in <genexpr>
    check_item(orig_left, item, subtype_context) for item in right.items
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 294, in check_item
    return is_proper_subtype(left, right, subtype_context=subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2169, in accept
    return visitor.visit_tuple_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 714, in visit_tuple_type
    elif self._is_subtype(mypy.typeops.tuple_fallback(left), right):
  File "/Users/shantanu/dev/mypy/mypy/typeops.py", line 108, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 664, in join_type_list
    joined = join_types(joined, t)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 239, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 257, in visit_union_type
    if is_proper_subtype(self.s, t):
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 390, in _is_subtype
    return is_proper_subtype(left, right, subtype_context=self.subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 301, in _is_subtype
    is_subtype_of_item = any(
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 302, in <genexpr>
    check_item(orig_left, item, subtype_context) for item in right.items
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 294, in check_item
    return is_proper_subtype(left, right, subtype_context=subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2169, in accept
    return visitor.visit_tuple_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 714, in visit_tuple_type
    elif self._is_subtype(mypy.typeops.tuple_fallback(left), right):
  File "/Users/shantanu/dev/mypy/mypy/typeops.py", line 108, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 664, in join_type_list
    joined = join_types(joined, t)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 239, in join_types
    return t.accept(TypeJoinVisitor(s, instance_joiner))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 257, in visit_union_type
    if is_proper_subtype(self.s, t):
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in visit_union_type
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 876, in <genexpr>
    return all(self._is_subtype(item, self.orig_right) for item in left.items)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 390, in _is_subtype
    return is_proper_subtype(left, right, subtype_context=self.subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 301, in _is_subtype
    is_subtype_of_item = any(
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 302, in <genexpr>
    check_item(orig_left, item, subtype_context) for item in right.items
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 294, in check_item
    return is_proper_subtype(left, right, subtype_context=subtype_context)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 224, in is_proper_subtype
    return _is_subtype(left, right, subtype_context, proper_subtype=True)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 329, in _is_subtype
    return left.accept(SubtypeVisitor(orig_right, subtype_context, proper_subtype))
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2169, in accept
    return visitor.visit_tuple_type(self)
  File "/Users/shantanu/dev/mypy/mypy/subtypes.py", line 714, in visit_tuple_type
    elif self._is_subtype(mypy.typeops.tuple_fallback(left), right):
  File "/Users/shantanu/dev/mypy/mypy/typeops.py", line 108, in tuple_fallback
    return Instance(info, [join_type_list(items)], extra_attrs=typ.partial_fallback.extra_attrs)
  File "/Users/shantanu/dev/mypy/mypy/join.py", line 662, in join_type_list
    joined = get_proper_type(types[0])
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2811, in get_proper_type
    typ = typ._expand_once()
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 277, in _expand_once
    return replace_alias_tvars(
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 3226, in replace_alias_tvars
    new_tp = tp.accept(replacer)
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 2589, in accept
    return visitor.visit_union_type(self)
  File "/Users/shantanu/dev/mypy/mypy/type_visitor.py", line 273, in visit_union_type
    return UnionType(self.translate_types(t.items), t.line, t.column)
  File "/Users/shantanu/dev/mypy/mypy/type_visitor.py", line 276, in translate_types
    return [t.accept(self) for t in types]
  File "/Users/shantanu/dev/mypy/mypy/type_visitor.py", line 276, in <listcomp>
    return [t.accept(self) for t in types]
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 1280, in accept
    return visitor.visit_instance(self)
  File "/Users/shantanu/dev/mypy/mypy/type_visitor.py", line 214, in visit_instance
    return Instance(
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 1218, in __init__
    super().__init__(line, column)
  File "/Users/shantanu/dev/mypy/mypy/types.py", line 208, in __init__
    super().__init__(line, column)

@ilevkivskyi
Copy link
Member

Yep, should be a few line fix (unless I am missing something).

@ilevkivskyi ilevkivskyi mentioned this issue Oct 4, 2022
Merged
@ilevkivskyi
Copy link
Member

After all my fix turned out to be more like a hack, so I tried to come with a more robust solution in #13808

hauntsaninja pushed a commit that referenced this issue Oct 8, 2022
@ilevkivskyi
Make join of recursive types more robust (#13808)
dbe9a88 
Fixes #13795

Calculating tuple fallbacks on the fly creates a cycle between joins and
subtyping. Although IMO this is conceptually not a right thing, it is
hard to get rid of (unless we want to use unions in the fallbacks, cc
@JukkaL). So instead I re-worked how `join_types()` works w.r.t.
`get_proper_type()`, essentially it now follows the golden rule "Always
pass on the original type if possible".
@eswald eswald mentioned this issue Apr 20, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
crash topic-recursive-types
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make join of recursive types more robust ilevkivskyi/mypy
4 participants
@JelleZijlstra @ilevkivskyi @mousetail @hauntsaninja