Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Narrowing unions with Literal and TypedDict #7944

Closed
porglezomp opened this issue Nov 13, 2019 · 2 comments · Fixed by #8151
Closed

Narrowing unions with Literal and TypedDict #7944

porglezomp opened this issue Nov 13, 2019 · 2 comments · Fixed by #8151
Assignees
@Michael0x2a
Labels
feature

Comments

@porglezomp
Copy link

I expected the following code to narrow the type of the argument, but it is unable to.

from mypy_extensions import TypedDict
from typing_extensions import Literal

TaggedStr = TypedDict('TaggedStr', {'tag': Literal['str'], 's': str})
TaggedInt = TypedDict('TaggedInt', {'tag': Literal['int'], 'i': int})

def do_tagged(x: Union[TaggedStr, TaggedInt]):
    if x['tag'] == 'str':
        reveal_type(x)
    else:
        reveal_type(x)
  • What is the actual behavior/output?
    No narrowing occurs:
example.py:10: note: Revealed type is 'Union[TypedDict('example.TaggedStr', {'tag': Literal['str'], 's': builtins.str}), TypedDict('example.TaggedInt', {'tag': Literal['int'], 'i': builtins.int})]'
example.py:12: note: Revealed type is 'Union[TypedDict('example.TaggedStr', {'tag': Literal['str'], 's': builtins.str}), TypedDict('example.TaggedInt', {'tag': Literal['int'], 'i': builtins.int})]'
  • What is the behavior/output you expect?
    I'd expect the comparisons against literal keys to allow narrowing the types.
example.py:10: note: Revealed type is 'TypedDict('example.TaggedStr', {'tag': Literal['str'], 's': builtins.str})'
example.py:12: note: Revealed type is 'TypedDict('example.TaggedInt', {'tag': Literal['int'], 'i': builtins.int})'
  • What are the versions of mypy and Python you are using?
    mypy 0.740, Python 3.7.4
@msullivan
Copy link
Collaborator

I feel like this issue must be a duplicate but I can't find what it is a duplicate of. I can, however, find a PR that should fix it: #7169

@msullivan
Copy link
Collaborator

Also as of mypy master, this works with enums but not yet with literals:

from typing import Union
from mypy_extensions import TypedDict
from typing_extensions import Literal
from enum import Enum

class Key(Enum):
    Str = "str"
    Int = "int"


TaggedStr = TypedDict('TaggedStr', {'tag': Literal[Key.Str], 's': str})
TaggedInt = TypedDict('TaggedInt', {'tag': Literal[Key.Int], 'i': int})

def do_tagged(x: Union[TaggedStr, TaggedInt]):
    if x['tag'] is Key.Str:
        reveal_type(x)
    else:
        reveal_type(x)

Michael0x2a added a commit to Michael0x2a/mypy that referenced this issue Dec 16, 2019
@Michael0x2a
Add support for narrowing Literals using equality
f48e890 
This pull request (finally) adds support for narrowing expressions
using Literal types by equality, instead of just identity. For example,
the following "tagged union" pattern is now supported:

```python
class Foo(TypedDict):
    key: Literal["A"]
    blah: int

class Bar(TypedDict):
    key: Literal["B"]
    something: str

x: Union[Foo, Bar]
if x.key == "A":
    reveal_type(x)  # Revealed type is 'Foo'
else:
    reveal_type(x)  # Revealed type is 'Bar'
```

Previously, this was possible to do only with Enum Literals and the
`is` operator, which is perhaps not very intuitive.

The main limitation with this pull request is that it'll perform narrowing
only if either the LHS or RHS contains an explicit Literal type
somewhere. If this limitation is not present, we end up breaking a decent
amount of real-world code -- mostly tests -- that do something like this:

```python
def some_test_case() -> None:
    worker = Worker()

    # Without the limitation, we narrow 'worker.state' to
    # Literal['ready'] in this assert...
    assert worker.state == 'ready'

    worker.start()

    # ...which subsequently causes this second assert to narrow
    # worker.state to <uninhabited>, causing the last line to be
    # unreachable.
    assert worker.state == 'running'
    worker.query()
```

I tried for several weeks to find a more intelligent way around this
problem, but everything I tried ended up being either insufficient or
super-hacky, so I gave up and went for this brute-force solution.

The other main limitation is that we perform narrowing only if both the
LHS and RHS do not define custom `__eq__` or `__ne__` methods, but this
seems like a more reasonable one to me.

Resolves python#7944.
Michael0x2a added a commit to Michael0x2a/mypy that referenced this issue Dec 16, 2019
@Michael0x2a
Add support for narrowing Literals using equality
77622f9 
This pull request (finally) adds support for narrowing expressions
using Literal types by equality, instead of just identity. For example,
the following "tagged union" pattern is now supported:

```python
class Foo(TypedDict):
    key: Literal["A"]
    blah: int

class Bar(TypedDict):
    key: Literal["B"]
    something: str

x: Union[Foo, Bar]
if x.key == "A":
    reveal_type(x)  # Revealed type is 'Foo'
else:
    reveal_type(x)  # Revealed type is 'Bar'
```

Previously, this was possible to do only with Enum Literals and the
`is` operator, which is perhaps not very intuitive.

The main limitation with this pull request is that it'll perform narrowing
only if either the LHS or RHS contains an explicit Literal type
somewhere. If this limitation is not present, we end up breaking a decent
amount of real-world code -- mostly tests -- that do something like this:

```python
def some_test_case() -> None:
    worker = Worker()

    # Without the limitation, we narrow 'worker.state' to
    # Literal['ready'] in this assert...
    assert worker.state == 'ready'

    worker.start()

    # ...which subsequently causes this second assert to narrow
    # worker.state to <uninhabited>, causing the last line to be
    # unreachable.
    assert worker.state == 'running'
    worker.query()
```

I tried for several weeks to find a more intelligent way around this
problem, but everything I tried ended up being either insufficient or
super-hacky, so I gave up and went for this brute-force solution.

The other main limitation is that we perform narrowing only if both the
LHS and RHS do not define custom `__eq__` or `__ne__` methods, but this
seems like a more reasonable one to me.

Resolves python#7944.
@Michael0x2a Michael0x2a mentioned this issue Dec 16, 2019
Merged
Michael0x2a added a commit to Michael0x2a/mypy that referenced this issue Dec 25, 2019
@Michael0x2a
Add support for narrowing Literals using equality
f82a019 
This pull request (finally) adds support for narrowing expressions
using Literal types by equality, instead of just identity. For example,
the following "tagged union" pattern is now supported:

```python
class Foo(TypedDict):
    key: Literal["A"]
    blah: int

class Bar(TypedDict):
    key: Literal["B"]
    something: str

x: Union[Foo, Bar]
if x.key == "A":
    reveal_type(x)  # Revealed type is 'Foo'
else:
    reveal_type(x)  # Revealed type is 'Bar'
```

Previously, this was possible to do only with Enum Literals and the
`is` operator, which is perhaps not very intuitive.

The main limitation with this pull request is that it'll perform narrowing
only if either the LHS or RHS contains an explicit Literal type
somewhere. If this limitation is not present, we end up breaking a decent
amount of real-world code -- mostly tests -- that do something like this:

```python
def some_test_case() -> None:
    worker = Worker()

    # Without the limitation, we narrow 'worker.state' to
    # Literal['ready'] in this assert...
    assert worker.state == 'ready'

    worker.start()

    # ...which subsequently causes this second assert to narrow
    # worker.state to <uninhabited>, causing the last line to be
    # unreachable.
    assert worker.state == 'running'
    worker.query()
```

I tried for several weeks to find a more intelligent way around this
problem, but everything I tried ended up being either insufficient or
super-hacky, so I gave up and went for this brute-force solution.

The other main limitation is that we perform narrowing only if both the
LHS and RHS do not define custom `__eq__` or `__ne__` methods, but this
seems like a more reasonable one to me.

Resolves python#7944.
Michael0x2a added a commit that referenced this issue Jan 8, 2020
@Michael0x2a
Add support for narrowing Literals using equality (#8151)
3dce3fd 
This pull request (finally) adds support for narrowing expressions
using Literal types by equality, instead of just identity. For example,
the following "tagged union" pattern is now supported:

```python
class Foo(TypedDict):
    key: Literal["A"]
    blah: int

class Bar(TypedDict):
    key: Literal["B"]
    something: str

x: Union[Foo, Bar]
if x.key == "A":
    reveal_type(x)  # Revealed type is 'Foo'
else:
    reveal_type(x)  # Revealed type is 'Bar'
```

Previously, this was possible to do only with Enum Literals and the
`is` operator, which is perhaps not very intuitive.

The main limitation with this pull request is that it'll perform narrowing
only if either the LHS or RHS contains an explicit Literal type
somewhere. If this limitation is not present, we end up breaking a decent
amount of real-world code -- mostly tests -- that do something like this:

```python
def some_test_case() -> None:
    worker = Worker()

    # Without the limitation, we narrow 'worker.state' to
    # Literal['ready'] in this assert...
    assert worker.state == 'ready'

    worker.start()

    # ...which subsequently causes this second assert to narrow
    # worker.state to <uninhabited>, causing the last line to be
    # unreachable.
    assert worker.state == 'running'
    worker.query()
```

I tried for several weeks to find a more intelligent way around this
problem, but everything I tried ended up being either insufficient or
super-hacky, so I gave up and went for this brute-force solution.

The other main limitation is that we perform narrowing only if both the
LHS and RHS do not define custom `__eq__` or `__ne__` methods, but this
seems like a more reasonable one to me.

Resolves #7944.
@devin-git devin-git mentioned this issue Sep 9, 2021
Closed
@aasiffaizal aasiffaizal mentioned this issue Aug 18, 2022
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Michael0x2a Michael0x2a

Labels
feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for narrowing Literals using equality Michael0x2a/mypy
3 participants
@msullivan @Michael0x2a @porglezomp