Bug: pydantic PrivateAttr fields are inspected for type hints

[mike0sv]

Description

If a private field is annotated with unresolvable ForwardRef, openapi schema build fails (see example)

URL to code causing the issue

No response

MCVE

from typing import TYPE_CHECKING

from litestar import Litestar, post
from litestar._openapi.plugin import OpenAPIPlugin
from pydantic import BaseModel

if TYPE_CHECKING:
    from typing import Any


class Model(BaseModel):
    class Config:
        underscore_attrs_are_private = True

    _value: "Any"


@post("/")
def hello_world(data: Model) -> dict[str, str]:
    """Keeping the tradition alive with hello world."""
    return {"hello": "world"}


def main():
    app = Litestar(route_handlers=[hello_world], )

    app.plugins.get(OpenAPIPlugin).provide_openapi()


if __name__ == '__main__':
    main()

Steps to reproduce

run the example

Screenshots

no screenshots

Logs

.../litestar_schema/app.py:21: LitestarWarning: Use of a synchronous callable <function hello_world at 0x102eafeb0> without setting sync_to_thread is discouraged since synchronous callables can block the main thread if they perform blocking operations. If the callable is guaranteed to be non-blocking, you can set sync_to_thread=False to skip this warning, or set the environmentvariable LITESTAR_WARN_IMPLICIT_SYNC_TO_THREAD=0 to disable warnings of this type entirely.
  def hello_world(data: Model) -> dict[str, str]:
Traceback (most recent call last):
  File ".../litestar_schema/app.py", line 33, in <module>
    main()
  File ".../litestar_schema/app.py", line 29, in main
    app.plugins.get(OpenAPIPlugin).provide_openapi()
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/plugin.py", line 46, in provide_openapi
    self._openapi_schema = self._build_openapi_schema()
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/plugin.py", line 37, in _build_openapi_schema
    openapi.paths = {
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/plugin.py", line 38, in <dictcomp>
    route.path_format or "/": create_path_item_for_route(context, route)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/path_item.py", line 137, in create_path_item_for_route
    return path_item_factory.create_path_item()
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/path_item.py", line 42, in create_path_item
    operation = self.create_operation_for_handler_method(route_handler, HttpMethod(http_method))
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/path_item.py", line 66, in create_operation_for_handler_method
    request_body = create_request_body(
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/request_body.py", line 49, in create_request_body
    schema = schema_creator.for_field_definition(data_field)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/schema_generation/schema.py", line 327, in for_field_definition
    result = self.for_plugin(field_definition, plugin_for_annotation)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/_openapi/schema_generation/schema.py", line 487, in for_plugin
    schema = plugin.to_openapi_schema(field_definition=field_definition, schema_creator=self)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/contrib/pydantic/pydantic_schema_plugin.py", line 234, in to_openapi_schema
    return self.for_pydantic_model(field_definition=field_definition, schema_creator=schema_creator)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/contrib/pydantic/pydantic_schema_plugin.py", line 250, in for_pydantic_model
    unwrapped_annotation, annotation_hints = pydantic_get_unwrapped_annotation_and_type_hints(annotation)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/litestar/contrib/pydantic/utils.py", line 162, in pydantic_get_unwrapped_annotation_and_type_hints
    return annotation, get_type_hints(annotation, include_extras=True)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/site-packages/typing_extensions.py", line 1104, in get_type_hints
    hint = typing.get_type_hints(
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/typing.py", line 1836, in get_type_hints
    value = _eval_type(value, base_globals, base_locals)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/typing.py", line 327, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "/Users/mike0sv/mambaforge/envs/evidently/lib/python3.10/typing.py", line 695, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'Any' is not defined. Did you mean: 'any'?

Litestar Version

litestar==2.5.1

Platform

• Linux
• Mac
• Windows
• Other (Please specify in the description above)

---

Note

While we are open for sponsoring on GitHub Sponsors and
OpenCollective, we also utilize Polar.sh to engage in pledge-based sponsorship.

Check out all issues funded or available for funding on our Polar.sh dashboard

• If you would like to see an issue prioritized, make a pledge towards it!
• We receive the pledge once the issue is completed & verified
• This, along with engagement in the community, helps us know which features are a priority to our users.

[provinzkraut]

Okay so I think the issue here is that we don't respect the Config.underscore_attrs_are_private. IMO we simply shouldn't touch the underscore fields at all if this is set. @litestar-org/maintainers?

We actually do. This is a different issue.

[provinzkraut]

Issue here is that while we do exclude those fields from the schema, we still try to extract their type annotations.

[provinzkraut]

@mike0sv There's a fix in #3151. Can you give it a try and confirm that it solve the issue for you?

[github-actions]

This issue has been closed in #3151. The change will be included in the upcoming patch release.

[mike0sv]

I tested it and it works for MCVE, but it still fails for evidently codebase. I narrowed it down to the fact that our "Model" class is actually a generic. Here is the new MCVE

from typing import Generic, Optional, TYPE_CHECKING, TypeVar

from litestar import Litestar, post
from litestar._openapi.plugin import OpenAPIPlugin
from pydantic import BaseModel

if TYPE_CHECKING:
    from typing import Any

T = TypeVar("T")


class Model(BaseModel, Generic[T]):
    class Config:
        underscore_attrs_are_private = True

    _value: Optional["Any"]


@post("/")
def hello_world(data: Model) -> dict[str, str]:
    """Keeping the tradition alive with hello world."""
    return {"hello": "world"}


def main():
    app = Litestar(route_handlers=[hello_world], )

    app.plugins.get(OpenAPIPlugin).provide_openapi()


if __name__ == '__main__':
    main()

[provinzkraut]

This seems to be exclusive to Pydantic V1 though; I couldn't reproduce with V2.

[provinzkraut]

@mike0sv I've created a follow-up here: #3161, which seems to fix your most recent MCVE as well. Let me know if this works for you.

[github-actions]

This issue has been closed in #3161. The change will be included in the upcoming patch release.

[mike0sv]

Yes! Thanks!

[github-actions]

A fix for this issue has been released in v2.6.3