Skip to content

Commit

Permalink
minor doc update
Browse files Browse the repository at this point in the history
  • Loading branch information
mjishnu committed Jun 5, 2024
1 parent 13281a8 commit efed9e0
Show file tree
Hide file tree
Showing 4 changed files with 38 additions and 16 deletions.
31 changes: 29 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ dl.start(
Each option is explained below:
- `allow_reuse`: Whether to allow reuse of existing Pypdl object for next download. The default value is `False`.
- `url`: The URL of the file to download.
- `file_path`: An optional path to save the downloaded file. By default, it uses the present working directory. If `file_path` is a directory, then the file is downloaded into it otherwise, the file is downloaded into the given path.\
- `file_path`: An optional path to save the downloaded file. By default, it uses the present working directory. If `file_path` is a directory, then the file is downloaded into it otherwise, the file is downloaded into the given path.
- `segments`: The number of segments the file should be divided in multi-segmented download. The default value is 10.
- `display`: Whether to display download progress and other optional messages. The default value is `True`.
- `multisegment`: Whether to use multi-segmented download. The default value is `True`.
Expand Down Expand Up @@ -274,7 +274,20 @@ The `Pypdl` class represents a file downloader that can download a file from a g

#### Arguments
- `allow_reuse`: (bool, Optional) Whether to allow reuse of existing `Pypdl` object for next download. The default value is `False`.It's essential to use `shutdown()` method when `allow_reuse` is enabled to ensure efficient resource management.
- Additional keyword arguments supported by [`aiohttp.ClientSession`](https://docs.aiohttp.org/en/stable/client_advanced.html#client-session).
- Additional keyword arguments
- `params` (default `None`): Parameters to be sent in the query string of the new request.
- `data` (default `None`): The data to send in the body of the request.
- `json` (default `None`): A JSON-compatible Python object to send in the body of the request.
- `cookies` (default `None`): HTTP Cookies to send with the request.
- `headers` (default `None`): HTTP Headers to send with the request.
- `auth` (default `None`): An object that represents HTTP Basic Authorization.
- `allow_redirects` (default `True`): If set to False, do not follow redirects.
- `max_redirects` (default `10`): Maximum number of redirects to follow.
- `proxy` (default `None`): Proxy URL.
- `proxy_auth` (default `None`): An object that represents proxy HTTP Basic Authorization.
- `timeout` (default `aiohttp.ClientTimeout(sock_read=60)`): Override the session’s timeout.
- `ssl` (default `None`): SSL validation mode.
- `proxy_headers` (default `None`): HTTP headers to send to the proxy if the `proxy` parameter has been provided.

#### Attributes

Expand Down Expand Up @@ -323,6 +336,20 @@ The `PypdlFactory` class manages multiple instances of the `Pypdl` downloader. I

- `instances`: (int, Optional) The number of `Pypdl` instances to create. The default value is 5.
- `allow_reuse`: (bool, Optional) Whether to allow reuse of existing `PypdlFactory` objects for next download. The default value is `False`. It's essential to use `shutdown()` method when `allow_reuse` is enabled to ensure efficient resource management.
- Additional keyword arguments
- `params` (default `None`): Parameters to be sent in the query string of the new request.
- `data` (default `None`): The data to send in the body of the request.
- `json` (default `None`): A JSON-compatible Python object to send in the body of the request.
- `cookies` (default `None`): HTTP Cookies to send with the request.
- `headers` (default `None`): HTTP Headers to send with the request.
- `auth` (default `None`): An object that represents HTTP Basic Authorization.
- `allow_redirects` (default `True`): If set to False, do not follow redirects.
- `max_redirects` (default `10`): Maximum number of redirects to follow.
- `proxy` (default `None`): Proxy URL.
- `proxy_auth` (default `None`): An object that represents proxy HTTP Basic Authorization.
- `timeout` (default `aiohttp.ClientTimeout(sock_read=60)`): Override the session’s timeout.
- `ssl` (default `None`): SSL validation mode.
- `proxy_headers` (default `None`): HTTP headers to send to the proxy if the `proxy` parameter has been provided.


#### Attributes
Expand Down
2 changes: 1 addition & 1 deletion pypdl/factory.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@


class Factory:
def __init__(self, instances: int = 2, allow_reuse=False, **kwargs):
def __init__(self, instances: int = 2, allow_reuse: bool = False, **kwargs):
self._instances = [Pypdl(True, **kwargs) for _ in range(instances)]
self._allow_reuse = allow_reuse
self._pool = ThreadPoolExecutor(max_workers=2)
Expand Down
4 changes: 2 additions & 2 deletions pypdl/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ class Pypdl(DownloadManager):
"""
A multi-segment file downloader that supports progress tracking, retries, pause/resume functionality etc.
This class accepts keyword arguments that are valid for aiohttp.ClientSession.
This class also supports additional keyword arguments specified in the documentation.
"""


class PypdlFactory(Factory):
"""
A factory class for managing multiple instances of the Pypdl downloader.
This class accepts keyword arguments that are valid for Pypdl.
This class also supports additional keyword arguments specified in the documentation.
"""
17 changes: 6 additions & 11 deletions pypdl/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,11 @@ def __init__(self, allow_reuse: bool = False, **kwargs):
self._status = 0
self._interrupt = Event()
self._stop = False
self._kwargs = {"timeout": aiohttp.ClientTimeout(sock_read=60)}
self._kwargs.update({**kwargs, "raise_for_status": True})
self._kwargs = {
"timeout": aiohttp.ClientTimeout(sock_read=60),
"raise_for_status": True,
}
self._kwargs.update(kwargs)
self._allow_reuse = allow_reuse

self.size = None
Expand Down Expand Up @@ -215,23 +218,15 @@ def _get_info(self, url, file_path, multisegment, etag):
return file_path, multisegment, etag

async def _get_header(self, url):
kwargs = self._kwargs.copy()
kwargs.update({"raise_for_status": False})

async with aiohttp.ClientSession() as session:
async with session.head(url, **kwargs) as response:
async with session.head(url, **self._kwargs) as response:
if response.status == 200:
return response.headers

async with session.get(url, **self._kwargs) as response:
if response.status == 200:
return response.headers

self._interrupt.set()
raise ConnectionError(
f"Server Returned: {response.reason}({response.status}), Invalid URL"
)

async def _multi_segment(self, segments, segment_table):
tasks = []
async with aiohttp.ClientSession() as session:
Expand Down

0 comments on commit efed9e0

Please sign in to comment.