Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve JSON typing in requests.requests.models #11465

Closed
wants to merge 1 commit into from

Conversation

AdemOdza
Copy link

@AdemOdza AdemOdza commented Feb 23, 2024

Added a dict[str, Any] type to the Response json() method and Request.

The Request class expects a serializable object (https://github.com/psf/requests/blob/main/src/requests/models.py#L510)

The Response.json() method returns a deserialized JSON string (https://github.com/psf/requests/blob/main/src/requests/models.py#L958)

This was causing issues with my type checker (MyPy) so I thought I would add it in.

def test(response: Response) -> dict[str, Any]:
  res = response.json()
  return res
  # MyPy complains because this function returns an Any object rather than a python 
  # JSON object

@Akuli
Copy link
Collaborator

Akuli commented Feb 23, 2024

The .json() method doesn't necessarily return a dict:

>>> import requests
>>> requests.get('https://httpbin.org/base64/WzEsMiwzXQ==').json()
[1, 2, 3]

Copy link
Contributor

Diff from mypy_primer, showing the effect of this PR on open source code:

openlibrary (https://github.com/internetarchive/openlibrary)
+ openlibrary/core/admin.py: note: In function "_get_loan_counts_from_graphite":
+ openlibrary/core/admin.py:81: error: Invalid index type "int" for "dict[str, Any]"; expected type "str"  [index]
+ openlibrary/core/admin.py: note: In function "_get_visitor_counts_from_graphite":
+ openlibrary/core/admin.py:126: error: Invalid index type "int" for "dict[str, Any]"; expected type "str"  [index]

cwltool (https://github.com/common-workflow-language/cwltool)
+ cwltool/resolver.py: note: In function "resolve_ga4gh_tool":
+ cwltool/resolver.py:82:20: error: "str" has no attribute "get"  [attr-defined]
+ cwltool/resolver.py:83:36: error: "str" has no attribute "get"  [attr-defined]

@AdemOdza
Copy link
Author

The .json() method doesn't necessarily return a dict:

>>> import requests
>>> requests.get('https://httpbin.org/base64/WzEsMiwzXQ==').json()
[1, 2, 3]

Oh right, of course. Would there be a better way to type this other than Any?

@Akuli
Copy link
Collaborator

Akuli commented Feb 23, 2024

Unfortunately, Any is the best we can do now. If we return dict[str, Any] | something, mypy will still complain when you try to use the return value as a list, because it thinks the return value could be a dict[str, Any] and you're not handling that. Ideally we would have something like AnyOf to work around this.

For your use case, you can return cast(dict[str, Any], response.json()).

@Akuli
Copy link
Collaborator

Akuli commented Feb 23, 2024

Maybe adding assert isinstance(res, dict) is better than a cast:

from requests import Response
from typing import Any

def test(response: Response) -> dict[str, Any]:
    res = response.json()
    assert isinstance(res, dict)
    return res

@AdemOdza
Copy link
Author

AdemOdza commented Feb 23, 2024

Thanks for the suggestions! I'll go ahead and close this PR since I was wrong

@AdemOdza AdemOdza closed this Feb 23, 2024
@AdemOdza AdemOdza deleted the requests-response-json branch February 23, 2024 19:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants