Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Storage] [Blob] Fixed next pylint errors in blob storage #34945

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ def upload_blob_to_url(
- except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:keyword bool overwrite:
Whether the blob to be uploaded should overwrite the current data.
If True, upload_blob_to_url will overwrite any existing data. If set to False, the
Expand Down Expand Up @@ -156,7 +156,7 @@ def download_blob_from_url(
- except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:keyword bool overwrite:
Whether the local file should be overwritten if it already exists. The default value is
`False` - in which case a ValueError will be raised if the file already exists. If set to
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -602,6 +602,8 @@ def upload_blob_from_url(self, source_url, **kwargs):
:keyword str source_authorization:
Authenticate as a service principal using a client secret to access a source blob. Ensure "bearer " is
the prefix of the source_authorization string.
:returns: Response from creating a new block blob for a given URL.
:rtype: Dict[str, Any]
"""
options = self._upload_blob_from_url_options(
source_url=self._encode_source_url(source_url),
Expand All @@ -622,6 +624,7 @@ def upload_blob(
"""Creates a new blob from a data source with automatic chunking.

:param data: The blob data to upload.
:type data: Union[bytes, str, Iterable[AnyStr], IO[AnyStr]]
:param ~azure.storage.blob.BlobType blob_type: The type of the blob. This can be
either BlockBlob, PageBlob or AppendBlob. The default value is BlockBlob.
:param int length:
Expand Down Expand Up @@ -1532,6 +1535,7 @@ def set_blob_metadata(self, metadata=None, **kwargs):
see `here <https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob
#other-client--per-operation-configuration>`_.
:returns: Blob-updated property dict (Etag and last modified)
:rtype: Dict[str, Union[str, datetime]]
"""
options = self._set_blob_metadata_options(metadata=metadata, **kwargs)
try:
Expand Down Expand Up @@ -2515,6 +2519,7 @@ def stage_block(
The string should be less than or equal to 64 bytes in size.
For a given blob, the block_id must be the same size for each block.
:param data: The blob data.
:type data: Union[Iterable[AnyStr], IO[AnyStr]]
:param int length: Size of the block.
:keyword bool validate_content:
If true, calculates an MD5 hash for each chunk of the blob. The storage
Expand Down Expand Up @@ -3284,7 +3289,7 @@ def get_page_range_diff_for_managed_disk(
.. versionadded:: 12.2.0
This operation was introduced in API version '2019-07-07'.

:param previous_snapshot_url:
:param str previous_snapshot_url:
Specifies the URL of a previous snapshot of the managed disk.
The response will only contain pages that were changed between the target blob and
its previous snapshot.
Expand Down Expand Up @@ -3812,6 +3817,8 @@ def upload_pages_from_url(self, source_url, # type: str
:keyword str source_authorization:
Authenticate as a service principal using a client secret to access a source blob. Ensure "bearer " is
the prefix of the source_authorization string.
:returns: Response after uploading pages from specified URL.
:rtype: Dict[str, Any]
"""
options = self._upload_pages_from_url_options(
source_url=self._encode_source_url(source_url),
Expand Down Expand Up @@ -4240,6 +4247,8 @@ def append_block_from_url(self, copy_source_url, # type: str
:keyword str source_authorization:
Authenticate as a service principal using a client secret to access a source blob. Ensure "bearer " is
the prefix of the source_authorization string.
:returns: Result after appending a new block.
:rtype: Dict[str, Union[str, datetime, int]]
"""
options = self._append_block_from_url_options(
copy_source_url=self._encode_source_url(copy_source_url),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ def from_connection_string(
Credentials provided here will take precedence over those in the connection string.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], "AzureNamedKeyCredential", "AzureSasCredential", "TokenCredential"]] # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], "AzureNamedKeyCredential", "AzureSasCredential", "TokenCredential"]] # pylint: disable=line-too-long
:keyword str audience: The audience to use when requesting tokens for Azure Active Directory
authentication. Only has an effect when credential is of type TokenCredential. The value could be
https://storage.azure.com/ (default) or https://<account>.blob.core.windows.net.
Expand Down Expand Up @@ -551,6 +551,7 @@ def create_container(
This value is not tracked or validated on the client. To configure client-side network timesouts
see `here <https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob
#other-client--per-operation-configuration>`_.
:returns: A container client to interact with the newly created container.
:rtype: ~azure.storage.blob.ContainerClient

.. admonition:: Example:
Expand Down Expand Up @@ -588,7 +589,7 @@ def delete_container(
If specified, delete_container only succeeds if the
container's lease is active and matches this ID.
Required if the container has an active lease.
:paramtype lease: ~azure.storage.blob.BlobLeaseClient or str
:type lease: ~azure.storage.blob.BlobLeaseClient or str
:keyword ~datetime.datetime if_modified_since:
A DateTime value. Azure expects the date value passed in to be UTC.
If timezone is included, any non-UTC datetimes will be converted to UTC.
Expand All @@ -612,7 +613,6 @@ def delete_container(
This value is not tracked or validated on the client. To configure client-side network timesouts
see `here <https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob
#other-client--per-operation-configuration>`_.
:rtype: None

.. admonition:: Example:

Expand Down Expand Up @@ -652,6 +652,7 @@ def _rename_container(self, name, new_name, **kwargs):
This value is not tracked or validated on the client. To configure client-side network timesouts
see `here <https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob
#other-client--per-operation-configuration>`_.
:returns: A container client for the renamed container.
:rtype: ~azure.storage.blob.ContainerClient
"""
renamed_container = self.get_container_client(new_name)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ def _get_blob_name(blob):
"""Return the blob name.

:param blob: A blob string or BlobProperties
:paramtype blob: str or BlobProperties
:type blob: str or BlobProperties
:returns: The name of the blob.
:rtype: str
"""
Expand Down Expand Up @@ -202,7 +202,7 @@ def from_container_url(
- except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], "AzureNamedKeyCredential", "AzureSasCredential", "TokenCredential"]] = None, # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], "AzureNamedKeyCredential", "AzureSasCredential", "TokenCredential"]] = None, # pylint: disable=line-too-long
:keyword str audience: The audience to use when requesting tokens for Azure Active Directory
authentication. Only has an effect when credential is of type TokenCredential. The value could be
https://storage.azure.com/ (default) or https://<account>.blob.core.windows.net.
Expand Down Expand Up @@ -251,7 +251,7 @@ def from_connection_string(
Credentials provided here will take precedence over those in the connection string.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], "AzureNamedKeyCredential", "AzureSasCredential", "TokenCredential"]] = None, # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], "AzureNamedKeyCredential", "AzureSasCredential", "TokenCredential"]] = None, # pylint: disable=line-too-long
:keyword str audience: The audience to use when requesting tokens for Azure Active Directory
authentication. Only has an effect when credential is of type TokenCredential. The value could be
https://storage.azure.com/ (default) or https://<account>.blob.core.windows.net.
Expand Down Expand Up @@ -338,13 +338,14 @@ def _rename_container(self, new_name, **kwargs):
:keyword lease:
Specify this to perform only if the lease ID given
matches the active lease ID of the source container.
:paramtype lease: ~azure.storage.blob.BlobLeaseClient or str
:type lease: ~azure.storage.blob.BlobLeaseClient or str
:keyword int timeout:
Sets the server-side timeout for the operation in seconds. For more details see
https://learn.microsoft.com/rest/api/storageservices/setting-timeouts-for-blob-service-operations.
This value is not tracked or validated on the client. To configure client-side network timesouts
see `here <https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob
#other-client--per-operation-configuration>`_.
:returns: The renamed container client.
:rtype: ~azure.storage.blob.ContainerClient
"""
lease = kwargs.pop('lease', None)
Expand Down Expand Up @@ -994,6 +995,7 @@ def upload_blob(

:param str name: The blob with which to interact.
:param data: The blob data to upload.
:type data: Union[bytes, str, Iterable[AnyStr], IO[AnyStr]]
:param ~azure.storage.blob.BlobType blob_type: The type of the blob. This can be
either BlockBlob, PageBlob or AppendBlob. The default value is BlockBlob.
:param int length:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ def __init__(self, size, content, downloader, chunk_size):
self._current_content = content
self._iter_downloader = downloader
self._iter_chunks = None
self._complete = (size == 0)
self._complete = size == 0

def __len__(self):
return self.size
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -857,7 +857,7 @@ def decrypt_blob( # pylint: disable=too-many-locals,too-many-statements
:param key_resolver:
The user-provided key resolver. Uses the kid string to return a key-encryption-key
implementing the interface defined above.
:paramtype key_resolver: Optional[Callable[[str], KeyEncryptionKey]]
:type key_resolver: Optional[Callable[[str], KeyEncryptionKey]]
:param bytes content:
The encrypted blob content.
:param int start_offset:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -433,9 +433,3 @@ def read_record(self, writer_schema, decoder):
def skip_record(self, writer_schema, decoder):
for field in writer_schema.fields:
self.skip_data(field.type, decoder)


# ------------------------------------------------------------------------------

if __name__ == '__main__':
raise Exception('Not a standalone module')
Original file line number Diff line number Diff line change
Expand Up @@ -417,9 +417,3 @@ async def read_record(self, writer_schema, decoder):
async def skip_record(self, writer_schema, decoder):
for field in writer_schema.fields:
await self.skip_data(field.type, decoder)


# ------------------------------------------------------------------------------

if __name__ == '__main__':
raise Exception('Not a standalone module')
Original file line number Diff line number Diff line change
Expand Up @@ -255,7 +255,3 @@ def __next__(self):
def close(self):
"""Close this reader."""
self.reader.close()


if __name__ == '__main__':
raise Exception('Not a standalone module')
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,3 @@ async def __anext__(self):
def close(self):
"""Close this reader."""
self.reader.close()


if __name__ == '__main__':
raise Exception('Not a standalone module')
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ def __str__(self):
# re-defining schemas that are already listed in the parameter names.
@abc.abstractmethod
def to_json(self, names):
raise Exception('Cannot run abstract method.')
...


# ------------------------------------------------------------------------------
Expand Down Expand Up @@ -433,7 +433,7 @@ def name_ref(self, names):
# of not re-defining schemas that are already listed in the parameter names.
@abc.abstractmethod
def to_json(self, names):
raise Exception('Cannot run abstract method.')
...

# ------------------------------------------------------------------------------

Expand Down Expand Up @@ -1086,7 +1086,7 @@ def MakeFields(names):
other_props=other_props,
)
else:
raise Exception(f'Internal error: unknown type {data_type!r}.')
raise ValueError(f'Internal error: unknown type {data_type!r}.')

elif data_type in VALID_TYPES:
# Unnamed, non-primitive Avro type:
Expand Down Expand Up @@ -1118,7 +1118,7 @@ def MakeFields(names):
result = ErrorUnionSchema(schemas=error_schemas)

else:
raise Exception(f'Internal error: unknown type {data_type!r}.')
raise ValueError(f'Internal error: unknown type {data_type!r}.')
else:
raise SchemaParseException(f'Invalid JSON descriptor for an Avro schema: {json_object!r}')
return result
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
import uuid
from io import SEEK_SET, UnsupportedOperation
from time import time
from typing import Any, Dict, Optional, TYPE_CHECKING, Union
from typing import Any, Dict, Optional, TYPE_CHECKING
from urllib.parse import (
parse_qsl,
urlencode,
Expand Down Expand Up @@ -471,12 +471,12 @@ def increment(
) -> bool:
"""Increment the retry counters.

Dict[str, Any]] settings: The configurable values pertaining to the increment operation.
:param Dict[str, Any]] settings: The configurable values pertaining to the increment operation.
:param PipelineRequest request: A pipeline request object.
:param Optional[PipelineResponse] response: A pipeline response object.
:param error: An error encountered during the request, or
None if the response was received successfully.
:paramtype error: Optional[AzureError]
:type error: Optional[AzureError]
:returns: Whether the retry attempts are exhausted.
:rtype: bool
"""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ async def upload_blob_to_url(
- except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:keyword bool overwrite:
Whether the blob to be uploaded should overwrite the current data.
If True, upload_blob_to_url will overwrite any existing data. If set to False, the
Expand Down Expand Up @@ -99,7 +99,7 @@ async def download_blob_from_url(
- except in the case of AzureSasCredential, where the conflicting SAS tokens will raise a ValueError.
If using an instance of AzureNamedKeyCredential, "name" should be the storage account name, and "key"
should be the storage account key.
:paramtype credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:type credential: Optional[Union[str, Dict[str, str], AzureNamedKeyCredential, AzureSasCredential, "TokenCredential"]] # pylint: disable=line-too-long
:keyword bool overwrite:
Whether the local file should be overwritten if it already exists. The default value is
`False` - in which case a ValueError will be raised if the file already exists. If set to
Expand Down
Loading