Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core typings #28114

Merged
merged 53 commits into from
Jan 30, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
7cef566
Core typings
lmazuel Dec 29, 2022
c7ea330
Restore mypy clean
lmazuel Dec 29, 2022
e7c843e
Black it
lmazuel Dec 29, 2022
44f9215
Mypy+PyLint+Black
lmazuel Dec 29, 2022
54440c1
Remove object inheritance
lmazuel Dec 29, 2022
38ceca3
dict/list are not typing before 3.9
lmazuel Dec 30, 2022
e328985
Runtime fixes
lmazuel Dec 30, 2022
d9a32b8
Forgotten List
lmazuel Dec 30, 2022
d146e18
Update _authentication.py
lmazuel Dec 30, 2022
67fe49e
Replace more dict by Dict
lmazuel Dec 30, 2022
7731deb
More typing fixes
lmazuel Dec 30, 2022
e3a0685
More Dict fixes
lmazuel Dec 30, 2022
e780393
Fix List typing
lmazuel Dec 30, 2022
8555fb4
Pyright Paging
lmazuel Dec 30, 2022
aa4097c
Simplify some ABC import
lmazuel Dec 30, 2022
71d0a63
Pylint clean-up
lmazuel Dec 30, 2022
b5afe05
Fix typing for Identity
lmazuel Dec 30, 2022
22dd4d8
Keep 'next' in paging as this used in some Py3 code...
lmazuel Dec 30, 2022
3879eef
Form can have int values as well
lmazuel Dec 30, 2022
e9745e2
Merge remote-tracking branch 'origin/main' into core_typing
lmazuel Dec 30, 2022
59d7ba0
Better annotations
lmazuel Dec 30, 2022
18d79b5
Pylint has trouble on 3.7
lmazuel Dec 30, 2022
f4ec82c
Merge remote-tracking branch 'origin/main' into core_typing
lmazuel Dec 31, 2022
6e8a0c4
Type configuration
lmazuel Dec 31, 2022
3f58e7f
MatchCondition doc
lmazuel Dec 31, 2022
f554628
PyLint happyness
lmazuel Dec 31, 2022
8bc6857
Feedback from Kashif
lmazuel Jan 3, 2023
d6c9e89
Merge branch 'main' into core_typing
lmazuel Jan 3, 2023
279a859
Redesign some of the async pipeline types
lmazuel Jan 3, 2023
922b774
Fix for 3.7
lmazuel Jan 4, 2023
941b370
Remove unecessary type
lmazuel Jan 4, 2023
0f58639
Feedback
lmazuel Jan 4, 2023
d46ccfe
Male kwarg only syntax
lmazuel Jan 4, 2023
5bf702e
Correct enum doc
lmazuel Jan 5, 2023
4fd71a8
Full typing for CloudEvent
lmazuel Jan 6, 2023
4a0a6d0
New feedback from Johan
lmazuel Jan 11, 2023
f9b8d6a
More feedback
lmazuel Jan 19, 2023
6207b92
Merge remote-tracking branch 'origin/main' into core_typing
lmazuel Jan 19, 2023
01009d6
Feedback
lmazuel Jan 19, 2023
ecc66e7
Merge remote-tracking branch 'origin/main' into core_typing
lmazuel Jan 23, 2023
30fb385
Enabling mypy and typecheck
lmazuel Jan 23, 2023
75aca88
Improve policy typing
lmazuel Jan 23, 2023
6b8e495
Improve response typing
lmazuel Jan 23, 2023
186ebb2
Update sdk/core/azure-core/azure/core/pipeline/policies/_authenticati…
lmazuel Jan 24, 2023
372513b
Update sdk/core/azure-core/azure/core/_pipeline_client_async.py
lmazuel Jan 24, 2023
65c5704
Update sdk/core/azure-core/azure/core/_pipeline_client_async.py
lmazuel Jan 24, 2023
60ca8fa
Feedback
lmazuel Jan 24, 2023
90d4a5b
Black
lmazuel Jan 24, 2023
d761f00
Enum typing
lmazuel Jan 24, 2023
e5d9aaf
Remove init for better typing
lmazuel Jan 27, 2023
7428d91
Merge remote-tracking branch 'origin/main' into core_typing
lmazuel Jan 30, 2023
571d0df
Fix typing
lmazuel Jan 30, 2023
f461802
PyLint
lmazuel Jan 30, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions sdk/core/azure-core/azure/core/_enum_meta.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@
# IN THE SOFTWARE.
#
# --------------------------------------------------------------------------

from enum import EnumMeta
from typing import Any
from enum import EnumMeta, Enum


class CaseInsensitiveEnumMeta(EnumMeta):
Expand All @@ -43,13 +43,13 @@ class MyCustomEnum(str, Enum, metaclass=CaseInsensitiveEnumMeta):

"""

def __getitem__(cls, name):
def __getitem__(cls, name: str) -> Any:
# disabling pylint bc of pylint bug https://github.com/PyCQA/astroid/issues/713
return super( # pylint: disable=no-value-for-parameter
CaseInsensitiveEnumMeta, cls
).__getitem__(name.upper())

def __getattr__(cls, name):
def __getattr__(cls, name: str) -> Enum:
"""Return the enum member matching `name`
We use __getattr__ instead of descriptors or inserting into the enum
class' __dict__ in order to support `name` and `value` being both
Expand Down
19 changes: 14 additions & 5 deletions sdk/core/azure-core/azure/core/_match_conditions.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,17 @@
class MatchConditions(Enum):
"""An enum to describe match conditions."""

Unconditionally = 1 # Matches any condition
IfNotModified = 2 # If the target object is not modified. Usually it maps to etag=<specific etag>
IfModified = 3 # Only if the target object is modified. Usually it maps to etag!=<specific etag>
IfPresent = 4 # If the target object exists. Usually it maps to etag='*'
IfMissing = 5 # If the target object does not exist. Usually it maps to etag!='*'
Unconditionally = 1
"""Matches any condition"""

IfNotModified = 2
"""If the target object is not modified. Usually it maps to etag=<specific etag>"""

IfModified = 3
"""Only if the target object is modified. Usually it maps to etag!=<specific etag>"""

IfPresent = 4
"""If the target object exists. Usually it maps to etag='*'"""

IfMissing = 5
"""If the target object does not exist. Usually it maps to etag!='*'"""
4 changes: 1 addition & 3 deletions sdk/core/azure-core/azure/core/_pipeline_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@
import logging
from collections.abc import Iterable
from typing import (
Any,
TypeVar,
TYPE_CHECKING,
)
Expand Down Expand Up @@ -169,8 +168,7 @@ def _build_pipeline(self, config, **kwargs): # pylint: disable=no-self-use

return Pipeline(transport, policies)

def send_request(self, request, **kwargs):
# type: (HTTPRequestType, Any) -> HTTPResponseType
def send_request(self, request: "HTTPRequestType", **kwargs) -> "HTTPResponseType":
"""Method that runs the network request through the client's chained policies.

>>> from azure.core.rest import HttpRequest
Expand Down
98 changes: 81 additions & 17 deletions sdk/core/azure-core/azure/core/_pipeline_client_async.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,16 @@

import logging
import collections.abc
from typing import Any, Awaitable, TypeVar
from typing import (
Any,
Awaitable,
TypeVar,
AsyncContextManager,
Generator,
cast,
TYPE_CHECKING,
)
from typing_extensions import Protocol
from .configuration import Configuration
from .pipeline import AsyncPipeline
from .pipeline.transport._base import PipelineClientBase
Expand All @@ -38,33 +47,88 @@
AsyncRetryPolicy,
)


if TYPE_CHECKING: # Protocol and non-Protocol can't mix in Python 3.7

class _AsyncContextManagerCloseable(AsyncContextManager, Protocol):
"""Defines a context manager that is closeable at the same time."""

async def close(self):
...


HTTPRequestType = TypeVar("HTTPRequestType")
AsyncHTTPResponseType = TypeVar("AsyncHTTPResponseType")
AsyncHTTPResponseType = TypeVar(
"AsyncHTTPResponseType", bound="_AsyncContextManagerCloseable"
)

_LOGGER = logging.getLogger(__name__)


class _AsyncContextManager(collections.abc.Awaitable):
def __init__(self, wrapped: collections.abc.Awaitable):
class _Coroutine(Awaitable[AsyncHTTPResponseType]):
"""Wrapper to get both context manager and awaitable in place.

Naming it "_Coroutine" because if you don't await it makes the error message easier:
>>> result = client.send_request(request)
>>> result.text()
AttributeError: '_Coroutine' object has no attribute 'text'
lmazuel marked this conversation as resolved.
Show resolved Hide resolved

Indeed, the message for calling a coroutine without waiting would be:
AttributeError: 'coroutine' object has no attribute 'text'

This allows the dev to either use the "async with" syntax, or simply the object directly.
It's also why "send_request" is not declared as async, since it couldn't be both easily.

"wrapped" must be an awaitable that returns an object that:
- has an async "close()"
- has an "__aexit__" method (IOW, is an async context manager)

This permits this code to work for both requests.

```python
from azure.core import AsyncPipelineClient
from azure.core.rest import HttpRequest

async def main():

request = HttpRequest("GET", "https://httpbin.org/user-agent")
async with AsyncPipelineClient("https://httpbin.org/") as client:
# Can be used directly
result = await client.send_request(request)
print(result.text())

# Can be used as an async context manager
async with client.send_request(request) as result:
print(result.text())
```

:param wrapped: Must be an awaitable the returns an async context manager that supports async "close()"
"""

def __init__(self, wrapped: Awaitable[AsyncHTTPResponseType]) -> None:
super().__init__()
self.wrapped = wrapped
self.response = None
self._wrapped = wrapped
# If someone tries to use the object without awaiting, they will get a
# AttributeError: '_Coroutine' object has no attribute 'text'
self._response: AsyncHTTPResponseType = cast(AsyncHTTPResponseType, None)

def __await__(self):
return self.wrapped.__await__()
def __await__(self) -> Generator[Any, None, AsyncHTTPResponseType]:
return self._wrapped.__await__()

async def __aenter__(self):
self.response = await self
return self.response
async def __aenter__(self) -> AsyncHTTPResponseType:
self._response = await self
return self._response

async def __aexit__(self, *args):
await self.response.__aexit__(*args)
async def __aexit__(self, *args) -> None:
await self._response.__aexit__(*args)

async def close(self):
await self.response.close()
async def close(self) -> None:
await self._response.close()


class AsyncPipelineClient(PipelineClientBase):
class AsyncPipelineClient(
PipelineClientBase, AsyncContextManager["AsyncPipelineClient"]
):
"""Service client core methods.

Builds an AsyncPipeline client.
Expand Down Expand Up @@ -212,4 +276,4 @@ def send_request(
:rtype: ~azure.core.rest.AsyncHttpResponse
"""
wrapped = self._make_pipeline_call(request, stream=stream, **kwargs)
return _AsyncContextManager(wrapped=wrapped)
return _Coroutine(wrapped=wrapped)
15 changes: 7 additions & 8 deletions sdk/core/azure-core/azure/core/async_paging.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
Tuple,
Optional,
Awaitable,
Any,
)

from .exceptions import AzureError
Expand Down Expand Up @@ -85,10 +86,10 @@ def __init__(
self._extract_data = extract_data
self.continuation_token = continuation_token
self._did_a_call_already = False
self._response = None
self._current_page = None
self._response: Optional[ResponseType] = None
lmazuel marked this conversation as resolved.
Show resolved Hide resolved
self._current_page: Optional[AsyncIterator[ReturnType]] = None

async def __anext__(self):
async def __anext__(self) -> AsyncIterator[ReturnType]:
if self.continuation_token is None and self._did_a_call_already:
raise StopAsyncIteration("End of paging")
try:
Expand All @@ -112,18 +113,16 @@ async def __anext__(self):


class AsyncItemPaged(AsyncIterator[ReturnType]):
def __init__(self, *args, **kwargs) -> None:
def __init__(self, *args: Any, **kwargs: Any) -> None:
"""Return an async iterator of items.

args and kwargs will be passed to the AsyncPageIterator constructor directly,
except page_iterator_class
"""
self._args = args
self._kwargs = kwargs
self._page_iterator = (
None
) # type: Optional[AsyncIterator[AsyncIterator[ReturnType]]]
self._page = None # type: Optional[AsyncIterator[ReturnType]]
self._page_iterator: Optional[AsyncIterator[AsyncIterator[ReturnType]]] = None
self._page: Optional[AsyncIterator[ReturnType]] = None
self._page_iterator_class = self._kwargs.pop(
"page_iterator_class", AsyncPageIterator
)
Expand Down
33 changes: 22 additions & 11 deletions sdk/core/azure-core/azure/core/configuration.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,10 @@
# IN THE SOFTWARE.
#
# --------------------------------------------------------------------------
from typing import Union, Optional


class Configuration(object):
class Configuration:
"""Provides the home for all of the configurable policies in the pipeline.

A new Configuration object provides no default policies and does not specify in what
Expand Down Expand Up @@ -88,16 +89,17 @@ def __init__(self, **kwargs):
self.polling_interval = kwargs.get("polling_interval", 30)


class ConnectionConfiguration(object):
class ConnectionConfiguration:
"""HTTP transport connection configuration settings.

Common properties that can be configured on all transports. Found in the
Configuration object.

:keyword int connection_timeout: A single float in seconds for the connection timeout. Defaults to 300 seconds.
:keyword int read_timeout: A single float in seconds for the read timeout. Defaults to 300 seconds.
:keyword bool connection_verify: SSL certificate verification. Enabled by default. Set to False to disable,
:keyword float connection_timeout: A single float in seconds for the connection timeout. Defaults to 300 seconds.
:keyword float read_timeout: A single float in seconds for the read timeout. Defaults to 300 seconds.
:keyword connection_verify: SSL certificate verification. Enabled by default. Set to False to disable,
alternatively can be set to the path to a CA_BUNDLE file or directory with certificates of trusted CAs.
:paramtype connection_verify: bool or str
:keyword str connection_cert: Client-side certificates. You can specify a local cert to use as client side
certificate, as a single file (containing the private key and the certificate) or as a tuple of both files' paths.
:keyword int connection_data_block_size: The block size of data sent over the connection. Defaults to 4096 bytes.
Expand All @@ -112,9 +114,18 @@ class ConnectionConfiguration(object):
:caption: Configuring transport connection settings.
"""

def __init__(self, **kwargs):
self.timeout = kwargs.pop("connection_timeout", 300)
self.read_timeout = kwargs.pop("read_timeout", 300)
self.verify = kwargs.pop("connection_verify", True)
self.cert = kwargs.pop("connection_cert", None)
self.data_block_size = kwargs.pop("connection_data_block_size", 4096)
def __init__(
self, # pylint: disable=unused-argument
*,
connection_timeout: float = 300,
read_timeout: float = 300,
connection_verify: Union[bool, str] = True,
connection_cert: Optional[str] = None,
connection_data_block_size: int = 4096,
**kwargs
) -> None:
self.timeout = connection_timeout
self.read_timeout = read_timeout
self.verify = connection_verify
self.cert = connection_cert
self.data_block_size = connection_data_block_size
Loading