Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to have TypedDict generator functions? #3932

Closed
wilfreddenton opened this issue Sep 6, 2017 · 7 comments
Closed

Is it possible to have TypedDict generator functions? #3932

wilfreddenton opened this issue Sep 6, 2017 · 7 comments

Comments

@wilfreddenton
Copy link

For my use case, I am working with a schema language that is different from TypedDict's but contains similar information. I have a function that can convert the schema to the correct format. How can I then create the TypedDict for use in other files?

See example below.

# file1.py

# Custom Schema
schema = {
    'prop': {'type': 'string'}
}

def create_typed_dict(name: str, schema: Dict[str,Any]) -> ??:
   return TypedDict(name, convert_custom_schema_to_typed_dict_schema(schema))

MyType = create_typed_dict('MyType', schema)
# file2.py
from file1 import MyType

X = {'prop': 'hey'} # type: MyType
@wilfreddenton wilfreddenton changed the title Is it possible to have TypedDict generator function? Is it possible to have TypedDict generator functions? Sep 6, 2017
@wilfreddenton
Copy link
Author

wilfreddenton commented Sep 6, 2017

I'm guessing that this is not possible because when I write something like:

TypedDict(name, schema)

I get a warning TypedDict() expects a dictionary literal as the second argument. I guess this is because mypy "reads" the file to get the types for each property in the dictionary?

@gvanrossum
Copy link
Member

Yeah, you're probably going to have to write something that generates stub (.pyi) files containing the TypedDict definitions in the form mypy supports.

@wilfreddenton
Copy link
Author

Thanks for the response. Do you think you could give me an example of this? I'm not sure how the stub file would help in this scenario and I couldn't find the stub representation of a TypedDict in the documentation.

@gvanrossum
Copy link
Member

Stub files have the same syntax as regular Python files: http://mypy.readthedocs.io/en/stable/basics.html#library-stubs-and-the-typeshed-repo . For your example you would have to generate something like

from mypy_extensions import TypedDict
schema = TypedDict('schema', {'type': str})

@JukkaL
Copy link
Collaborator

JukkaL commented Sep 7, 2017

Instead of generating a stub file, you could generate .py a file with TypedDict definitions and then import the generated types from there. For example, you can generate a file like this:

# mytypes.py -- automatically generated
from typing import TypedDict

MyType = TypedDict('MyType', {'prop': str})

Then you can use the type by importing it:

from mytypes import MyType

def create() -> MyType:
    return {'prop': 'value'}

@wilfreddenton
Copy link
Author

Thanks for the help. I was able to get this to work by manually creating the stub file. So eventually I'll put in the time to create the generation script.

@Emerentius
Copy link

I had the same need and ended up writing a quick and dirty generator some time ago. I've cleaned it up a bit and uploaded it here: https://github.com/emerentius/typed_dict_generator .
So far it only takes a single Json object and generates a typed dict for it. I plan to extend it, when I have need of it again, but it may already be useful to someone in its current state.

br3ndonland added a commit to br3ndonland/inboard that referenced this issue Dec 6, 2022
This commit will enable mypy strict mode, and update code accordingly.

Type annotations are not used at runtime. The standard library `typing`
module includes a `TYPE_CHECKING` constant that is `False` at runtime,
but `True` when conducting static type checking prior to runtime. Type
imports will be included under `if TYPE_CHECKING:` conditions. These
conditions will be ignored when calculating test coverage.
https://docs.python.org/3/library/typing.html

The Python standard library `logging.config` module uses type stubs.
The typeshed types for the `logging.config` module are used solely for
type-checking usage of the `logging.config` module itself. They cannot
be imported and used to type annotate other modules. For this reason,
dict config types will be vendored into a module in the inboard package.
https://github.com/python/typeshed/blob/main/stdlib/logging/config.pyi

The ASGI application in `inboard.app.main_base` will be updated to ASGI3
and type-annotated with `asgiref.typing`. Note that, while Uvicorn uses
`asgiref.typing`, Starlette does not. The type signature expected by the
Starlette/FastAPI `TestClient` therefore does not match
`asgiref.typing.ASGIApplication`. A mypy `type: ignore[arg-type]`
comment will be used to resolve this difference.
https://asgi.readthedocs.io/en/stable/specs/main.html

Also note that, while the `asgiref` package was a runtime dependency of
Uvicorn 0.17.6, it was later removed from Uvicorn's runtime dependencies
in 0.18.0 (encode/uvicorn#1305, encode/uvicorn#1532). However, `asgiref`
is still used to type-annotate Uvicorn, so any downstream projects like
inboard that type-check Uvicorn objects must also install `asgiref`.
Therefore, `asgiref` will be added to inboard's development dependencies
to ensure that type checking continues to work as expected.

A Uvicorn options type will be added to a new inboard types module.
The Uvicorn options type will be a `TypedDict` with fields corresponding
to arguments to `uvicorn.run`. This type can be used to check arguments
passed to `uvicorn.run`, which is how `inboard.start` runs Uvicorn.

Uvicorn 0.17.6 is not fully type-annotated, and Uvicorn does not ship
with a `py.typed` marker file until 0.19.0.

It would be convenient to generate types dynamically with something like
`getattr(uvicorn.run, "__annotations__")` (Python 3.9 or earlier)
or `inspect.get_annotations(uvicorn.run)` (Python 3.10 or later).
https://docs.python.org/3/howto/annotations.html

It could look something like this:

```py
UvicornOptions = TypedDict(  # type: ignore[misc]
    "UvicornOptions",
    inspect.get_annotations(uvicorn.run),
    total=False,
)
```

Note the `type: ignore[misc]` comment. Mypy raises a `misc` error:
`TypedDict() expects a dictionary literal as the second argument`.
Unfortunately, `TypedDict` types are not intended to be generated
dynamically, because they exist for the benefit of static type checking
(python/mypy#3932, python/mypy#4128, python/mypy#13940).

Furthermore, prior to Uvicorn 0.18.0, `uvicorn.run()` didn't enumerate
keyword arguments, but instead accepted `kwargs` and passed them to
`uvicorn.Config.__init__()` (encode/uvicorn#1423). The annotations from
`uvicorn.Config.__init__()` would need to be used instead. Even after
Uvicorn 0.18.0, the signatures of the two functions are not exactly the
same (encode/uvicorn#1545), so it helps to have a static type defined.

There will be some other differences from `uvicorn.run()`:

- The `app` argument to `uvicorn.run()` accepts an un-parametrized
  `Callable` because Uvicorn tests use callables (encode/uvicorn#1067).
  It is not necessary for other packages to accept `Callable`, and it
  would need to be parametrized to pass mypy strict mode anyway.
  For these reasons, `Callable` will not be accepted in this type.
- The `log_config` argument will use the new inboard dict config type
  instead of `dict[str, Any]` for stricter type checking.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants