From 139069af539b5687edfd6ad2790dc7373c0c515b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 9 Oct 2023 10:48:13 +0000 Subject: [PATCH 01/25] Bump aiodns from 3.0.0 to 3.1.0 (#7681) Bumps [aiodns](https://github.com/saghul/aiodns) from 3.0.0 to 3.1.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aiodns&package-manager=pip&previous-version=3.0.0&new-version=3.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/base.txt | 2 +- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/runtime-deps.txt | 2 +- requirements/test.txt | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/requirements/base.txt b/requirements/base.txt index bd2e392ca3a..a98ebfea1fd 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/base.txt --strip-extras requirements/base.in # -aiodns==3.0.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 9d39a8e0e19..b14855e9b81 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/constraints.txt --resolver=backtracking --strip-extras requirements/constraints.in # -aiodns==3.0.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiohttp-theme==0.1.6 # via -r requirements/doc.in diff --git a/requirements/dev.txt b/requirements/dev.txt index 422bc9ac049..e16f5aeeecd 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/dev.txt --resolver=backtracking --strip-extras requirements/dev.in # -aiodns==3.0.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiohttp-theme==0.1.6 # via -r requirements/doc.in diff --git a/requirements/runtime-deps.txt b/requirements/runtime-deps.txt index 7dcce778e89..f0d44599437 100644 --- a/requirements/runtime-deps.txt +++ b/requirements/runtime-deps.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/runtime-deps.txt --strip-extras requirements/runtime-deps.in # -aiodns==3.0.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in diff --git a/requirements/test.txt b/requirements/test.txt index 9219ae1f28a..a5b99294a0a 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/test.txt --resolver=backtracking --strip-extras requirements/test.in # -aiodns==3.0.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in From c0ba7e537e6a959f553b3c299ae1c02066d4d194 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Mon, 9 Oct 2023 18:25:59 +0100 Subject: [PATCH 02/25] Make setup.cfg packages explicit (#7684) --- setup.cfg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/setup.cfg b/setup.cfg index 8fa44a86565..4b63cf039e4 100644 --- a/setup.cfg +++ b/setup.cfg @@ -41,7 +41,7 @@ classifiers = [options] python_requires = >=3.8 -packages = find: +packages = aiohttp # https://setuptools.readthedocs.io/en/latest/setuptools.html#setting-the-zip-safe-flag zip_safe = False include_package_data = True From 30850babb43a8e28dd2df036776c62fd613d3d89 Mon Sep 17 00:00:00 2001 From: Tymofii Tsiapa <31003183+ttsia@users.noreply.github.com> Date: Mon, 9 Oct 2023 21:31:23 +0300 Subject: [PATCH 03/25] Fix #7306 - Set ClientWebSocketResponse.close_code correctly in concurrent closing scenario (#7680) --- CHANGES/7306.bugfix | 1 + aiohttp/client_ws.py | 5 +++-- tests/test_client_ws_functional.py | 27 +++++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 2 deletions(-) create mode 100644 CHANGES/7306.bugfix diff --git a/CHANGES/7306.bugfix b/CHANGES/7306.bugfix new file mode 100644 index 00000000000..173236d2fd2 --- /dev/null +++ b/CHANGES/7306.bugfix @@ -0,0 +1 @@ +Fixed ``ClientWebSocketResponse.close_code`` being erroneously set to ``None`` when there are concurrent async tasks receiving data and closing the connection. diff --git a/aiohttp/client_ws.py b/aiohttp/client_ws.py index 0a010fa7920..02d9f6b6de7 100644 --- a/aiohttp/client_ws.py +++ b/aiohttp/client_ws.py @@ -191,7 +191,8 @@ async def send_json( async def close(self, *, code: int = WSCloseCode.OK, message: bytes = b"") -> bool: # we need to break `receive()` cycle first, # `close()` may be called from different task - if self._waiting is not None and not self._closed: + if self._waiting is not None and not self._closing: + self._closing = True self._reader.feed_data(WS_CLOSING_MESSAGE, 0) await self._waiting @@ -210,7 +211,7 @@ async def close(self, *, code: int = WSCloseCode.OK, message: bytes = b"") -> bo self._response.close() return True - if self._closing: + if self._close_code: self._response.close() return True diff --git a/tests/test_client_ws_functional.py b/tests/test_client_ws_functional.py index 205f4d50a6e..5a4b6edbbfe 100644 --- a/tests/test_client_ws_functional.py +++ b/tests/test_client_ws_functional.py @@ -233,6 +233,33 @@ async def handler(request): assert msg.type == aiohttp.WSMsgType.CLOSED +async def test_concurrent_task_close(aiohttp_client: Any) -> None: + async def handler(request): + ws = web.WebSocketResponse() + await ws.prepare(request) + await ws.receive() + return ws + + app = web.Application() + app.router.add_route("GET", "/", handler) + + client = await aiohttp_client(app) + async with client.ws_connect("/") as resp: + # wait for the message in a separate task + task = asyncio.create_task(resp.receive()) + + # Make sure we start to wait on receiving message before closing the connection + await asyncio.sleep(0.1) + + closed = await resp.close() + + await task + + assert closed + assert resp.closed + assert resp.close_code == 1000 + + async def test_concurrent_close(aiohttp_client: Any) -> None: client_ws = None From 2590d8df3b1675d7b8a0873f03ba0e04ea268775 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 12 Oct 2023 00:18:22 +0000 Subject: [PATCH 04/25] Bump mypy from 1.5.1 to 1.6.0 (#7692) Bumps [mypy](https://github.com/python/mypy) from 1.5.1 to 1.6.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mypy&package-manager=pip&previous-version=1.5.1&new-version=1.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sam Bull --- aiohttp/http_parser.py | 2 +- aiohttp/http_websocket.py | 2 +- aiohttp/http_writer.py | 2 +- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- requirements/test.txt | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 24be6a28bdd..192ecfd22bf 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -966,7 +966,7 @@ def end_http_chunk_receiving(self) -> None: try: if not NO_EXTENSIONS: - from ._http_parser import ( # type: ignore[import,no-redef] + from ._http_parser import ( # type: ignore[import-not-found,no-redef] HttpRequestParser, HttpResponseParser, RawRequestMessage, diff --git a/aiohttp/http_websocket.py b/aiohttp/http_websocket.py index deb8ab9dcc5..ffd882a3128 100644 --- a/aiohttp/http_websocket.py +++ b/aiohttp/http_websocket.py @@ -160,7 +160,7 @@ def _websocket_mask_python(mask: bytes, data: bytearray) -> None: _websocket_mask = _websocket_mask_python else: try: - from ._websocket import _websocket_mask_cython # type: ignore[import] + from ._websocket import _websocket_mask_cython # type: ignore[import-not-found] _websocket_mask = _websocket_mask_cython except ImportError: # pragma: no cover diff --git a/aiohttp/http_writer.py b/aiohttp/http_writer.py index 8f2d9086b92..d6b02e6f566 100644 --- a/aiohttp/http_writer.py +++ b/aiohttp/http_writer.py @@ -189,7 +189,7 @@ def _py_serialize_headers(status_line: str, headers: "CIMultiDict[str]") -> byte _serialize_headers = _py_serialize_headers try: - import aiohttp._http_writer as _http_writer # type: ignore[import] + import aiohttp._http_writer as _http_writer # type: ignore[import-not-found] _c_serialize_headers = _http_writer._serialize_headers if not NO_EXTENSIONS: diff --git a/requirements/constraints.txt b/requirements/constraints.txt index b14855e9b81..cfe3a28a552 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -103,7 +103,7 @@ multidict==6.0.4 # -r requirements/multidict.in # -r requirements/runtime-deps.in # yarl -mypy==1.5.1 ; implementation_name == "cpython" +mypy==1.6.0 ; implementation_name == "cpython" # via # -r requirements/lint.in # -r requirements/test.in diff --git a/requirements/dev.txt b/requirements/dev.txt index e16f5aeeecd..4c95faf775e 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -100,7 +100,7 @@ multidict==6.0.4 # via # -r requirements/runtime-deps.in # yarl -mypy==1.5.1 ; implementation_name == "cpython" +mypy==1.6.0 ; implementation_name == "cpython" # via # -r requirements/lint.in # -r requirements/test.in diff --git a/requirements/lint.txt b/requirements/lint.txt index 1c8dfcbf2db..3cb01305940 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -22,7 +22,7 @@ identify==2.5.26 # via pre-commit iniconfig==2.0.0 # via pytest -mypy==1.5.1 ; implementation_name == "cpython" +mypy==1.6.0 ; implementation_name == "cpython" # via -r requirements/lint.in mypy-extensions==1.0.0 # via mypy diff --git a/requirements/test.txt b/requirements/test.txt index a5b99294a0a..b2caa7bf02c 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -51,7 +51,7 @@ multidict==6.0.4 # via # -r requirements/runtime-deps.in # yarl -mypy==1.5.1 ; implementation_name == "cpython" +mypy==1.6.0 ; implementation_name == "cpython" # via -r requirements/test.in mypy-extensions==1.0.0 # via mypy From b7fe1758cd6e2c11c494b151393672eba275cab8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 12 Oct 2023 10:54:42 +0000 Subject: [PATCH 05/25] Bump cherry-picker from 2.1.0 to 2.2.0 (#7694) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [cherry-picker](https://github.com/python/cherry-picker) from 2.1.0 to 2.2.0.
Release notes

Sourced from cherry-picker's releases.

cherry-picker v.2.2.0

What's Changed

Full Changelog: https://github.com/python/cherry-picker/compare/cherry-picker-v2.1.0...cherry-picker-v2.2.0

Commits
  • 64b3ffd Bump to v 2.2.0
  • f8d10c2 Update changelog to mention dropping Python 3.7
  • 5143f8a Merge branch 'main' into prepare-release
  • f59f7b7 Drop support for EOL Python 3.7 (#90)
  • aead5f8 Merge branch 'main' into prepare-release
  • b471da1 Remove initial_state as it gets out of sync with what's in .git/config (#88)
  • 6440b34 Merge branch 'main' into prepare-release
  • e1a8edc Update the version
  • 69cc5f3 Prepare Changelog for 2.1.1 release
  • e4f927d When raising error, show the current state vs expected state (#87)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cherry-picker&package-manager=pip&previous-version=2.1.0&new-version=2.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 5 ++--- requirements/dev.txt | 5 ++--- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index cfe3a28a552..69231c4f59c 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -36,7 +36,7 @@ cfgv==3.3.1 # via pre-commit charset-normalizer==3.2.0 # via requests -cherry-picker==2.1.0 +cherry-picker==2.2.0 # via -r requirements/dev.in click==8.1.6 # via @@ -202,11 +202,10 @@ sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" # via -r requirements/doc-spelling.in sphinxcontrib-towncrier==0.3.2a0 # via -r requirements/doc.in -toml==0.10.2 - # via cherry-picker tomli==2.0.1 # via # build + # cherry-picker # coverage # mypy # pip-tools diff --git a/requirements/dev.txt b/requirements/dev.txt index 4c95faf775e..3ea44deafa5 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -36,7 +36,7 @@ cfgv==3.3.1 # via pre-commit charset-normalizer==3.2.0 # via requests -cherry-picker==2.1.0 +cherry-picker==2.2.0 # via -r requirements/dev.in click==8.1.6 # via @@ -194,11 +194,10 @@ sphinxcontrib-serializinghtml==1.1.5 # via sphinx sphinxcontrib-towncrier==0.3.2a0 # via -r requirements/doc.in -toml==0.10.2 - # via cherry-picker tomli==2.0.1 # via # build + # cherry-picker # coverage # mypy # pip-tools From 39f8ff225a3eb93bae3b9fd54fa377e31e0a9f6d Mon Sep 17 00:00:00 2001 From: Tymofii Tsiapa <31003183+ttsia@users.noreply.github.com> Date: Thu, 12 Oct 2023 19:18:50 +0300 Subject: [PATCH 06/25] Implement WebSocketResponse.get_extra_info method to avoid using private member (#7658) --- CHANGES/7078.feature | 1 + CONTRIBUTORS.txt | 1 + aiohttp/web_ws.py | 13 +++++++++++++ docs/client_reference.rst | 10 +++++++++- docs/web_reference.rst | 14 ++++++++++++++ tests/test_web_websocket.py | 32 ++++++++++++++++++++++++++++++++ 6 files changed, 70 insertions(+), 1 deletion(-) create mode 100644 CHANGES/7078.feature diff --git a/CHANGES/7078.feature b/CHANGES/7078.feature new file mode 100644 index 00000000000..9a58141e200 --- /dev/null +++ b/CHANGES/7078.feature @@ -0,0 +1 @@ +Added ``WebSocketResponse.get_extra_info()`` to access a protocol transport's extra info. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 25ca8cf6249..5f4da7a35ab 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -317,6 +317,7 @@ Tolga Tezel Tomasz Trebski Toshiaki Tanaka Trinh Hoang Nhu +Tymofii Tsiapa Vadim Suharnikov Vaibhav Sagar Vamsi Krishna Avula diff --git a/aiohttp/web_ws.py b/aiohttp/web_ws.py index 9fcdc4bdd53..a21443dff1b 100644 --- a/aiohttp/web_ws.py +++ b/aiohttp/web_ws.py @@ -323,6 +323,19 @@ def ws_protocol(self) -> Optional[str]: def compress(self) -> bool: return self._compress + def get_extra_info(self, name: str, default: Any = None) -> Any: + """Get optional transport information. + + If no value associated with ``name`` is found, ``default`` is returned. + """ + writer = self._writer + if writer is None: + return default + transport = writer.transport + if transport is None: + return default + return transport.get_extra_info(name, default) + def exception(self) -> Optional[BaseException]: return self._exception diff --git a/docs/client_reference.rst b/docs/client_reference.rst index 8b1fad334a2..df584b23d03 100644 --- a/docs/client_reference.rst +++ b/docs/client_reference.rst @@ -1508,7 +1508,15 @@ manually. .. method:: get_extra_info(name, default=None) - Reads extra info from connection's transport + Reads optional extra information from the connection's transport. + If no value associated with ``name`` is found, ``default`` is returned. + + See :meth:`asyncio.BaseTransport.get_extra_info` + + :param str name: The key to look up in the transport extra information. + + :param default: Default value to be used when no value for ``name`` is + found (default is ``None``). .. method:: exception() diff --git a/docs/web_reference.rst b/docs/web_reference.rst index 6af022c7f84..6652edb8490 100644 --- a/docs/web_reference.rst +++ b/docs/web_reference.rst @@ -379,6 +379,8 @@ and :ref:`aiohttp-web-signals` handlers. Reads extra information from the protocol's transport. If no value associated with ``name`` is found, ``default`` is returned. + See :meth:`asyncio.BaseTransport.get_extra_info` + :param str name: The key to look up in the transport extra information. :param default: Default value to be used when no value for ``name`` is @@ -1049,6 +1051,18 @@ and :ref:`aiohttp-web-signals` handlers:: May be ``None`` if server and client protocols are not overlapping. + .. method:: get_extra_info(name, default=None) + + Reads optional extra information from the writer's transport. + If no value associated with ``name`` is found, ``default`` is returned. + + See :meth:`asyncio.BaseTransport.get_extra_info` + + :param str name: The key to look up in the transport extra information. + + :param default: Default value to be used when no value for ``name`` is + found (default is ``None``). + .. method:: exception() Returns last occurred exception or None. diff --git a/tests/test_web_websocket.py b/tests/test_web_websocket.py index 093cf549cf6..90e798813f4 100644 --- a/tests/test_web_websocket.py +++ b/tests/test_web_websocket.py @@ -412,3 +412,35 @@ async def test_no_transfer_encoding_header(make_request: Any, mocker: Any) -> No await ws._start(req) assert "Transfer-Encoding" not in ws.headers + + +@pytest.mark.parametrize( + "ws_transport, expected_result", + [ + ( + mock.MagicMock( + transport=mock.MagicMock( + get_extra_info=lambda name, default=None: {"test": "existent"}.get( + name, default + ) + ) + ), + "existent", + ), + (None, "default"), + (mock.MagicMock(transport=None), "default"), + ], +) +async def test_get_extra_info( + make_request: Any, mocker: Any, ws_transport: Any, expected_result: Any +) -> None: + valid_key = "test" + default_value = "default" + + req = make_request("GET", "/") + ws = WebSocketResponse() + + await ws.prepare(req) + ws._writer = ws_transport + + assert ws.get_extra_info(valid_key, default_value) == expected_result From 312f747de91f20fa33af03fd368f857fbd32f36a Mon Sep 17 00:00:00 2001 From: Ben Kallus <49924171+kenballus@users.noreply.github.com> Date: Sun, 15 Oct 2023 10:20:50 -0400 Subject: [PATCH 07/25] Require full version and method regex matches (#7701) ## What do these changes do? These changes ensure that HTTP versions and methods fully match the regular expressions for those constructs. AIOHTTP currently only applies prefix-matching, which I assume was unintentional. ## Are there changes in behavior for the user? There should be no observable changes to the user, unless they use HTTP servers/clients that generate very malformed request lines. Such clients/servers are unlikely to exist because most other web servers reject these malformed messages. ## Related issue number Fixes #7700 --- CHANGES/7700.bugfix | 1 + CONTRIBUTORS.txt | 1 + aiohttp/http_parser.py | 6 +++--- tests/test_http_parser.py | 4 ++-- 4 files changed, 7 insertions(+), 5 deletions(-) create mode 100644 CHANGES/7700.bugfix diff --git a/CHANGES/7700.bugfix b/CHANGES/7700.bugfix new file mode 100644 index 00000000000..26fdfa9076b --- /dev/null +++ b/CHANGES/7700.bugfix @@ -0,0 +1 @@ +Fix issue with insufficient HTTP method and version validation. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 5f4da7a35ab..f327a205cd3 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -53,6 +53,7 @@ Arthur Darcet Austin Scola Ben Bader Ben Greiner +Ben Kallus Ben Timby Benedikt Reinartz Bob Haddleton diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 192ecfd22bf..4dd615ad389 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -549,11 +549,11 @@ def parse_message(self, lines: List[bytes]) -> RawRequestMessage: ) # method - if not METHRE.match(method): + if not METHRE.fullmatch(method): raise BadStatusLine(method) # version - match = VERSRE.match(version) + match = VERSRE.fullmatch(version) if match is None: raise BadStatusLine(line) version_o = HttpVersion(int(match.group(1)), int(match.group(2))) @@ -652,7 +652,7 @@ def parse_message(self, lines: List[bytes]) -> RawResponseMessage: ) # version - match = VERSRE.match(version) + match = VERSRE.fullmatch(version) if match is None: raise BadStatusLine(line) version_o = HttpVersion(int(match.group(1)), int(match.group(2))) diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 9bbdf255276..4ed27e67ea4 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -727,7 +727,7 @@ def test_http_request_parser_two_slashes(parser: Any) -> None: def test_http_request_parser_bad_method(parser: Any) -> None: with pytest.raises(http_exceptions.BadStatusLine): - parser.feed_data(b'=":(e),[T];?" /get HTTP/1.1\r\n\r\n') + parser.feed_data(b'G=":<>(e),[T];?" /get HTTP/1.1\r\n\r\n') def test_http_request_parser_bad_version(parser: Any) -> None: @@ -737,7 +737,7 @@ def test_http_request_parser_bad_version(parser: Any) -> None: def test_http_request_parser_bad_version_number(parser: Any) -> None: with pytest.raises(http_exceptions.BadHttpMessage): - parser.feed_data(b"GET /test HTTP/12.3\r\n\r\n") + parser.feed_data(b"GET /test HTTP/1.32\r\n\r\n") @pytest.mark.parametrize("size", [40965, 8191]) From d7802fe2e48bd7b1f70c8d66786ad125305eaa24 Mon Sep 17 00:00:00 2001 From: Hugo Hromic Date: Sun, 15 Oct 2023 17:29:57 +0100 Subject: [PATCH 08/25] feature: allow link argument to be set to None/empty in HTTP 451 exception (#7689) ## What do these changes do? This PR allows for the link argument to be set to None/empty (so it can be optional) as per RFC in the HTTP 451 exception (`HTTPUnavailableForLegalReasons`). It also adds suitable unit tests for the `link` argument and aligns variable naming in the test class of this exception. I also updated the documentation for this exception and adjusted the documented signatures of the other exceptions in the same section (the `body` argument does not exist anymore). ## Are there changes in behavior for the user? The exception constructor no longer requires the `link` argument nor accepts empty or invalid values. ## Checklist - [X] I think the code is well written - [X] Unit tests for the changes exist - [X] Documentation reflects the changes - [X] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES` folder * name it `.` for example (588.bugfix) * if you don't have an `issue_id` change it to the pr id after creating the pr * ensure type is one of the following: * `.feature`: Signifying a new feature. * `.bugfix`: Signifying a bug fix. * `.doc`: Signifying a documentation improvement. * `.removal`: Signifying a deprecation or removal of public API. * `.misc`: A ticket has been closed, but it is not of interest to users. * Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files." --------- Co-authored-by: Sam Bull --- CHANGES/7689.feature | 1 + CONTRIBUTORS.txt | 1 + aiohttp/web_exceptions.py | 10 +++++---- docs/web_exceptions.rst | 23 ++++++++++++++------- tests/test_web_exceptions.py | 40 ++++++++++++++++++++++++++++++------ 5 files changed, 58 insertions(+), 17 deletions(-) create mode 100644 CHANGES/7689.feature diff --git a/CHANGES/7689.feature b/CHANGES/7689.feature new file mode 100644 index 00000000000..086f33c2388 --- /dev/null +++ b/CHANGES/7689.feature @@ -0,0 +1 @@ +Allow ``link`` argument to be set to None/empty in HTTP 451 exception. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index f327a205cd3..22756eb9d89 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -145,6 +145,7 @@ Hrishikesh Paranjape Hu Bo Hugh Young Hugo Herter +Hugo Hromic Hugo van Kemenade Hynek Schlawack Igor Alexandrov diff --git a/aiohttp/web_exceptions.py b/aiohttp/web_exceptions.py index 332ca9fa565..9ae61b43c9f 100644 --- a/aiohttp/web_exceptions.py +++ b/aiohttp/web_exceptions.py @@ -420,7 +420,7 @@ class HTTPUnavailableForLegalReasons(HTTPClientError): def __init__( self, - link: StrOrURL, + link: Optional[StrOrURL], *, headers: Optional[LooseHeaders] = None, reason: Optional[str] = None, @@ -430,11 +430,13 @@ def __init__( super().__init__( headers=headers, reason=reason, text=text, content_type=content_type ) - self.headers["Link"] = f'<{str(link)}>; rel="blocked-by"' - self._link = URL(link) + self._link = None + if link: + self._link = URL(link) + self.headers["Link"] = f'<{str(self._link)}>; rel="blocked-by"' @property - def link(self) -> URL: + def link(self) -> Optional[URL]: return self._link diff --git a/docs/web_exceptions.rst b/docs/web_exceptions.rst index 989f1d90f52..fd15632fb6a 100644 --- a/docs/web_exceptions.rst +++ b/docs/web_exceptions.rst @@ -85,7 +85,7 @@ HTTP Exception hierarchy chart:: All HTTP exceptions have the same constructor signature:: HTTPNotFound(*, headers=None, reason=None, - body=None, text=None, content_type=None) + text=None, content_type=None) If not directly specified, *headers* will be added to the *default response headers*. @@ -94,8 +94,8 @@ Classes :exc:`HTTPMultipleChoices`, :exc:`HTTPMovedPermanently`, :exc:`HTTPFound`, :exc:`HTTPSeeOther`, :exc:`HTTPUseProxy`, :exc:`HTTPTemporaryRedirect` have the following constructor signature:: - HTTPFound(location, *, headers=None, reason=None, - body=None, text=None, content_type=None) + HTTPFound(location, *,headers=None, reason=None, + text=None, content_type=None) where *location* is value for *Location HTTP header*. @@ -104,7 +104,15 @@ unsupported method and list of allowed methods:: HTTPMethodNotAllowed(method, allowed_methods, *, headers=None, reason=None, - body=None, text=None, content_type=None) + text=None, content_type=None) + +:exc:`HTTPUnavailableForLegalReasons` should be constructed with a ``link`` +to yourself (as the entity implementing the blockage), and an explanation for +the block included in ``text``.:: + + HTTPUnavailableForLegalReasons(link, *, + headers=None, reason=None, + text=None, content_type=None) Base HTTP Exception ------------------- @@ -478,14 +486,15 @@ HTTP exceptions for status code in range 400-499, e.g. ``raise web.HTTPNotFound( An exception for *451 Unavailable For Legal Reasons*, a subclass of :exc:`HTTPClientError`. - :param link: A link to a resource with information for blocking reason, - :class:`str` or :class:`~yarl.URL` + :param link: A link to yourself (as the entity implementing the blockage), + :class:`str`, :class:`~yarl.URL` or ``None``. For other parameters see :exc:`HTTPException` constructor. + A reason for the block should be included in ``text``. .. attribute:: link - A :class:`~yarl.URL` link to a resource with information for blocking reason, + A :class:`~yarl.URL` link to the entity implementing the blockage or ``None``, read-only property. diff --git a/tests/test_web_exceptions.py b/tests/test_web_exceptions.py index 2c9e2d32d2e..1d3262abab6 100644 --- a/tests/test_web_exceptions.py +++ b/tests/test_web_exceptions.py @@ -312,23 +312,51 @@ def test_pickle(self) -> None: class TestHTTPUnavailableForLegalReasons: def test_ctor(self) -> None: - resp = web.HTTPUnavailableForLegalReasons( + exc = web.HTTPUnavailableForLegalReasons( link="http://warning.or.kr/", headers={"X-Custom": "value"}, reason="Zaprescheno", text="text", content_type="custom", ) - assert resp.link == URL("http://warning.or.kr/") - assert resp.text == "text" + assert exc.link == URL("http://warning.or.kr/") + assert exc.text == "text" compare: Mapping[str, str] = { "X-Custom": "value", "Content-Type": "custom", "Link": '; rel="blocked-by"', } - assert resp.headers == compare - assert resp.reason == "Zaprescheno" - assert resp.status == 451 + assert exc.headers == compare + assert exc.reason == "Zaprescheno" + assert exc.status == 451 + + def test_no_link(self) -> None: + with pytest.raises(TypeError): + web.HTTPUnavailableForLegalReasons() # type: ignore[call-arg] + + def test_none_link(self) -> None: + exc = web.HTTPUnavailableForLegalReasons(link=None) + assert exc.link is None + assert "Link" not in exc.headers + + def test_empty_link(self) -> None: + exc = web.HTTPUnavailableForLegalReasons(link="") + assert exc.link is None + assert "Link" not in exc.headers + + def test_link_str(self) -> None: + exc = web.HTTPUnavailableForLegalReasons(link="http://warning.or.kr/") + assert exc.link == URL("http://warning.or.kr/") + assert exc.headers["Link"] == '; rel="blocked-by"' + + def test_link_url(self) -> None: + exc = web.HTTPUnavailableForLegalReasons(link=URL("http://warning.or.kr/")) + assert exc.link == URL("http://warning.or.kr/") + assert exc.headers["Link"] == '; rel="blocked-by"' + + def test_link_CRLF(self) -> None: + exc = web.HTTPUnavailableForLegalReasons(link="http://warning.or.kr/\r\n") + assert "\r\n" not in exc.headers["Link"] def test_pickle(self) -> None: resp = web.HTTPUnavailableForLegalReasons( From 8cd5d1f9e1a5cf1767542e5d6dfc22c6f9d9c6e3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 16 Oct 2023 11:50:19 +0000 Subject: [PATCH 09/25] Bump aiodns from 3.1.0 to 3.1.1 (#7709) Bumps [aiodns](https://github.com/saghul/aiodns) from 3.1.0 to 3.1.1.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=aiodns&package-manager=pip&previous-version=3.1.0&new-version=3.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/base.txt | 2 +- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/runtime-deps.txt | 2 +- requirements/test.txt | 2 +- 5 files changed, 5 insertions(+), 5 deletions(-) diff --git a/requirements/base.txt b/requirements/base.txt index a98ebfea1fd..c266d1bb69d 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/base.txt --strip-extras requirements/base.in # -aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.1 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 69231c4f59c..fc5d4e45b35 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/constraints.txt --resolver=backtracking --strip-extras requirements/constraints.in # -aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.1 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiohttp-theme==0.1.6 # via -r requirements/doc.in diff --git a/requirements/dev.txt b/requirements/dev.txt index 3ea44deafa5..8607c113581 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/dev.txt --resolver=backtracking --strip-extras requirements/dev.in # -aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.1 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiohttp-theme==0.1.6 # via -r requirements/doc.in diff --git a/requirements/runtime-deps.txt b/requirements/runtime-deps.txt index f0d44599437..d478d05e720 100644 --- a/requirements/runtime-deps.txt +++ b/requirements/runtime-deps.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/runtime-deps.txt --strip-extras requirements/runtime-deps.in # -aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.1 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in diff --git a/requirements/test.txt b/requirements/test.txt index b2caa7bf02c..e4d371c1ab9 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/test.txt --resolver=backtracking --strip-extras requirements/test.in # -aiodns==3.1.0 ; sys_platform == "linux" or sys_platform == "darwin" +aiodns==3.1.1 ; sys_platform == "linux" or sys_platform == "darwin" # via -r requirements/runtime-deps.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in From 5c3adc402caa941ab162d0d25d54217a1ff82a88 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 16 Oct 2023 11:59:34 +0000 Subject: [PATCH 10/25] Bump pre-commit from 3.4.0 to 3.5.0 (#7710) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.4.0 to 3.5.0.
Release notes

Sourced from pre-commit's releases.

pre-commit v3.5.0

Features

Fixes

Migrating

Changelog

Sourced from pre-commit's changelog.

3.5.0 - 2023-10-13

Features

Fixes

Migrating

Commits
  • 61cc55a v3.5.0
  • c9945b9 Merge pull request #3029 from adamchainz/improve_duration_timing
  • d988767 Improve hook duration timing
  • 0d8b245 Merge pull request #3023 from pre-commit/pre-commit-ci-update-config
  • 155c521 [pre-commit.ci] pre-commit autoupdate
  • 676e51a Merge pull request #3024 from pre-commit/pick-shebang-path-without-spaces
  • 997ea0a use sys.executable instead of echo.exe in parse_shebang
  • 19aa121 Merge pull request #3016 from pre-commit/pre-commit-ci-update-config
  • a4ab977 [pre-commit.ci] pre-commit autoupdate
  • 3f3760b Merge pull request #3011 from hack3ric/bump-node-and-go-version
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pre-commit&package-manager=pip&previous-version=3.4.0&new-version=3.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index fc5d4e45b35..f84f8bed870 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -127,7 +127,7 @@ platformdirs==3.10.0 # via virtualenv pluggy==1.2.0 # via pytest -pre-commit==3.4.0 +pre-commit==3.5.0 # via -r requirements/lint.in proxy-py==2.4.3 # via -r requirements/test.in diff --git a/requirements/dev.txt b/requirements/dev.txt index 8607c113581..14228976a82 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -124,7 +124,7 @@ platformdirs==3.10.0 # via virtualenv pluggy==1.2.0 # via pytest -pre-commit==3.4.0 +pre-commit==3.5.0 # via -r requirements/lint.in proxy-py==2.4.3 # via -r requirements/test.in diff --git a/requirements/lint.txt b/requirements/lint.txt index 3cb01305940..5a51f94ba81 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -34,7 +34,7 @@ platformdirs==3.10.0 # via virtualenv pluggy==1.2.0 # via pytest -pre-commit==3.4.0 +pre-commit==3.5.0 # via -r requirements/lint.in pytest==7.4.2 # via -r requirements/lint.in From d697d4211b38e435fd59a0ced331989c8b258097 Mon Sep 17 00:00:00 2001 From: Ben Kallus <49924171+kenballus@users.noreply.github.com> Date: Mon, 16 Oct 2023 12:32:28 -0400 Subject: [PATCH 11/25] Add check to validate absolute URIs (#7713) --- CHANGES/7712.bugfix | 1 + aiohttp/http_parser.py | 7 +++++++ tests/test_http_parser.py | 5 +++++ 3 files changed, 13 insertions(+) create mode 100644 CHANGES/7712.bugfix diff --git a/CHANGES/7712.bugfix b/CHANGES/7712.bugfix new file mode 100644 index 00000000000..b5304c34ac2 --- /dev/null +++ b/CHANGES/7712.bugfix @@ -0,0 +1 @@ +Add check to validate that absolute URIs have schemes. diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 4dd615ad389..3eef05020f6 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -34,6 +34,7 @@ ContentEncodingError, ContentLengthError, InvalidHeader, + InvalidURLError, LineTooLong, TransferEncodingError, ) @@ -578,10 +579,16 @@ def parse_message(self, lines: List[bytes]) -> RawRequestMessage: fragment=url_fragment, encoded=True, ) + elif path == "*" and method == "OPTIONS": + # asterisk-form, + url = URL(path, encoded=True) else: # absolute-form for proxy maybe, # https://datatracker.ietf.org/doc/html/rfc7230#section-5.3.2 url = URL(path, encoded=True) + if url.scheme == "": + # not absolute-form + raise InvalidURLError(line) # read headers ( diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 4ed27e67ea4..a234a895fe2 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -740,6 +740,11 @@ def test_http_request_parser_bad_version_number(parser: Any) -> None: parser.feed_data(b"GET /test HTTP/1.32\r\n\r\n") +def test_http_request_parser_bad_uri(parser: Any) -> None: + with pytest.raises(http_exceptions.InvalidURLError): + parser.feed_data(b"GET ! HTTP/1.1\r\n\r\n") + + @pytest.mark.parametrize("size", [40965, 8191]) def test_http_request_max_status_line(parser: Any, size: Any) -> None: path = b"t" * (size - 5) From 47acf47d9a1e405e0b09f701a9957707b95af05a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 16 Oct 2023 18:26:25 +0000 Subject: [PATCH 12/25] Bump actions/cache from 3.0.4 to 3.3.2 (#7591) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/cache](https://github.com/actions/cache) from 3.0.4 to 3.3.2.
Release notes

Sourced from actions/cache's releases.

v3.3.2

What's Changed

New Contributors

Full Changelog: https://github.com/actions/cache/compare/v3...v3.3.2

v3.3.1

What's Changed

Full Changelog: https://github.com/actions/cache/compare/v3...v3.3.1

v3.3.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/cache/compare/v3...v3.3.0

v3.2.6

What's Changed

Full Changelog: https://github.com/actions/cache/compare/v3...v3.2.6

v3.2.5

What's Changed

... (truncated)

Changelog

Sourced from actions/cache's changelog.

3.0.4

  • Fixed tar creation error while trying to create tar with path as ~/ home folder on ubuntu-latest. (issue)

3.0.5

  • Removed error handling by consuming actions/cache 3.0 toolkit, Now cache server error handling will be done by toolkit. (PR)

3.0.6

  • Fixed #809 - zstd -d: no such file or directory error
  • Fixed #833 - cache doesn't work with github workspace directory

3.0.7

  • Fixed #810 - download stuck issue. A new timeout is introduced in the download process to abort the download if it gets stuck and doesn't finish within an hour.

3.0.8

  • Fix zstd not working for windows on gnu tar in issues #888 and #891.
  • Allowing users to provide a custom timeout as input for aborting download of a cache segment using an environment variable SEGMENT_DOWNLOAD_TIMEOUT_MINS. Default is 60 minutes.

3.0.9

  • Enhanced the warning message for cache unavailablity in case of GHES.

3.0.10

  • Fix a bug with sorting inputs.
  • Update definition for restore-keys in README.md

3.0.11

  • Update toolkit version to 3.0.5 to include @actions/core@^1.10.0
  • Update @actions/cache to use updated saveState and setOutput functions from @actions/core@^1.10.0

3.1.0-beta.1

  • Update @actions/cache on windows to use gnu tar and zstd by default and fallback to bsdtar and zstd if gnu tar is not available. (issue)

3.1.0-beta.2

  • Added support for fallback to gzip to restore old caches on windows.

3.1.0-beta.3

  • Bug fixes for bsdtar fallback if gnutar not available and gzip fallback if cache saved using old cache action on windows.

3.2.0-beta.1

... (truncated)

Commits
  • 704facf Merge pull request #1236 from actions/bethanyj28/bump-version
  • 17e2888 Add to RELEASES.md
  • 667d8fd bump action version to 3.3.2
  • f7ebb81 Consume latest toolkit and fix dangling promise bug (#1217)
  • 67b839e Merge pull request #1187 from jorendorff/jorendorff/rm-add-to-project
  • 57f0e3f Remove actions to add new PRs and issues to a project board
  • 04f198b Merge pull request #1132 from vorburger/bazel-example
  • bd9b49b Merge branch 'main' into bazel-example
  • ea05037 Merge pull request #1122 from actions/pdotl-patch-1
  • 6a1a45d Merge branch 'main' into pdotl-patch-1
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/cache&package-manager=github_actions&previous-version=3.0.4&new-version=3.3.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci-cd.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 876933b3dd3..5f13f951944 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -50,7 +50,7 @@ jobs: with: python-version: 3.9 - name: Cache PyPI - uses: actions/cache@v3.0.4 + uses: actions/cache@v3.3.2 with: key: pip-lint-${{ hashFiles('requirements/*.txt') }} path: ~/.cache/pip @@ -111,7 +111,7 @@ jobs: with: submodules: true - name: Cache llhttp generated files - uses: actions/cache@v3.0.4 + uses: actions/cache@v3.3.2 id: cache with: key: llhttp-${{ hashFiles('vendor/llhttp/package.json', 'vendor/llhttp/src/**/*') }} @@ -199,7 +199,7 @@ jobs: run: | echo "::set-output name=dir::$(pip cache dir)" # - name: Cache - name: Cache PyPI - uses: actions/cache@v3.0.4 + uses: actions/cache@v3.3.2 with: key: pip-ci-${{ runner.os }}-${{ matrix.pyver }}-${{ matrix.no-extensions }}-${{ hashFiles('requirements/*.txt') }} path: ${{ steps.pip-cache.outputs.dir }} From 5b4b7b84cf3fa0cec0384a3115b4d43636834c85 Mon Sep 17 00:00:00 2001 From: JJ <103335846+computerscienceiscool@users.noreply.github.com> Date: Mon, 16 Oct 2023 14:26:42 -0700 Subject: [PATCH 13/25] Update README.rst (#7702) --- README.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.rst b/README.rst index 875b8fc7196..2ea1f4e2bb3 100644 --- a/README.rst +++ b/README.rst @@ -44,7 +44,7 @@ Key Features - Supports both client and server side of HTTP protocol. - Supports both client and server Web-Sockets out-of-the-box and avoids Callback Hell. -- Provides Web-server with middlewares and plugable routing. +- Provides Web-server with middleware and pluggable routing. Getting started From 5a499d040654455eb03909ef6ab66a454316e317 Mon Sep 17 00:00:00 2001 From: Ben Kallus <49924171+kenballus@users.noreply.github.com> Date: Tue, 17 Oct 2023 14:31:10 -0400 Subject: [PATCH 14/25] Fix unhandled exception for invalid Unicode in Python HTTP parser (#7716) ## What do these changes do? Fixes an unhandled exception in the Python HTTP parser that causes servers to 500 when they should 400 upon receiving a header with an invalid Unicode sequence. ## Are there changes in behavior for the user? Nope. ## Related issue number Fixes #7715 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- CHANGES/7715.bugfix | 1 + aiohttp/http_exceptions.py | 7 +++---- aiohttp/http_parser.py | 4 +++- tests/test_http_exceptions.py | 8 ++++---- tests/test_http_parser.py | 24 +++++++++++++++++++++++- 5 files changed, 34 insertions(+), 10 deletions(-) create mode 100644 CHANGES/7715.bugfix diff --git a/CHANGES/7715.bugfix b/CHANGES/7715.bugfix new file mode 100644 index 00000000000..863ea25a693 --- /dev/null +++ b/CHANGES/7715.bugfix @@ -0,0 +1 @@ +Fix unhandled exception when Python HTTP parser encounters unpaired Unicode surrogates. diff --git a/aiohttp/http_exceptions.py b/aiohttp/http_exceptions.py index f1711321f78..610514b5523 100644 --- a/aiohttp/http_exceptions.py +++ b/aiohttp/http_exceptions.py @@ -85,10 +85,9 @@ def __init__( class InvalidHeader(BadHttpMessage): def __init__(self, hdr: Union[bytes, str]) -> None: - if isinstance(hdr, bytes): - hdr = hdr.decode("utf-8", "surrogateescape") - super().__init__(f"Invalid HTTP Header: {hdr}") - self.hdr = hdr + hdr_s = hdr.decode(errors="backslashreplace") if isinstance(hdr, bytes) else hdr + super().__init__(f"Invalid HTTP header: {hdr!r}") + self.hdr = hdr_s self.args = (hdr,) diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index 3eef05020f6..c2e04139a44 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -588,7 +588,9 @@ def parse_message(self, lines: List[bytes]) -> RawRequestMessage: url = URL(path, encoded=True) if url.scheme == "": # not absolute-form - raise InvalidURLError(line) + raise InvalidURLError( + path.encode(errors="surrogateescape").decode("latin1") + ) # read headers ( diff --git a/tests/test_http_exceptions.py b/tests/test_http_exceptions.py index 28fdcbe0c69..ace35062a0f 100644 --- a/tests/test_http_exceptions.py +++ b/tests/test_http_exceptions.py @@ -103,7 +103,7 @@ class TestInvalidHeader: def test_ctor(self) -> None: err = http_exceptions.InvalidHeader("X-Spam") assert err.code == 400 - assert err.message == "Invalid HTTP Header: X-Spam" + assert err.message == "Invalid HTTP header: 'X-Spam'" assert err.headers is None def test_pickle(self) -> None: @@ -113,17 +113,17 @@ def test_pickle(self) -> None: pickled = pickle.dumps(err, proto) err2 = pickle.loads(pickled) assert err2.code == 400 - assert err2.message == "Invalid HTTP Header: X-Spam" + assert err2.message == "Invalid HTTP header: 'X-Spam'" assert err2.headers is None assert err2.foo == "bar" def test_str(self) -> None: err = http_exceptions.InvalidHeader(hdr="X-Spam") - assert str(err) == "400, message:\n Invalid HTTP Header: X-Spam" + assert str(err) == "400, message:\n Invalid HTTP header: 'X-Spam'" def test_repr(self) -> None: err = http_exceptions.InvalidHeader(hdr="X-Spam") - expected = "" + expected = "" assert repr(err) == expected diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index a234a895fe2..4cd38836af2 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -185,6 +185,23 @@ def test_bad_headers(parser: Any, hdr: str) -> None: parser.feed_data(text) +def test_unpaired_surrogate_in_header_py(loop: Any, protocol: Any) -> None: + parser = HttpRequestParserPy( + protocol, + loop, + 2**16, + max_line_size=8190, + max_field_size=8190, + ) + text = b"POST / HTTP/1.1\r\n\xff\r\n\r\n" + message = None + try: + parser.feed_data(text) + except http_exceptions.InvalidHeader as e: + message = e.message.encode("utf-8") + assert message is not None + + def test_content_length_transfer_encoding(parser: Any) -> None: text = ( b"GET / HTTP/1.1\r\nHost: a\r\nContent-Length: 5\r\nTransfer-Encoding: a\r\n\r\n" @@ -740,11 +757,16 @@ def test_http_request_parser_bad_version_number(parser: Any) -> None: parser.feed_data(b"GET /test HTTP/1.32\r\n\r\n") -def test_http_request_parser_bad_uri(parser: Any) -> None: +def test_http_request_parser_bad_ascii_uri(parser: Any) -> None: with pytest.raises(http_exceptions.InvalidURLError): parser.feed_data(b"GET ! HTTP/1.1\r\n\r\n") +def test_http_request_parser_bad_nonascii_uri(parser: Any) -> None: + with pytest.raises(http_exceptions.InvalidURLError): + parser.feed_data(b"GET \xff HTTP/1.1\r\n\r\n") + + @pytest.mark.parametrize("size", [40965, 8191]) def test_http_request_max_status_line(parser: Any, size: Any) -> None: path = b"t" * (size - 5) From 493f06797654c383242f0e8007f6e06b818a1fbc Mon Sep 17 00:00:00 2001 From: Ben Kallus <49924171+kenballus@users.noreply.github.com> Date: Wed, 18 Oct 2023 06:22:20 -0400 Subject: [PATCH 15/25] Stop accepting `\x80-\xff` in header names; stop accepting `\n` as separating whitespace in status-lines (#7719) ## What do these changes do? This PR updates the python HTTP parser to stop accepting `\x80-\xff` in header names and stop accepting `\n` as separating whitespace in status-lines. Both of these are not allowed in the RFCs. ## Are there changes in behavior for the user? Only users of seriously misbehaving clients would notice a change in behavior for this patch. If a client sends non-ascii UTF-8 within header names, their messages will now 400. Note that (nearly) arbitrary values are still allowed within header values. Most HTTP servers (Apache, Nginx, IIS, Node) do not accept UTF-8 within header names because of the risk of control character injection. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> --- CHANGES/7719.bugfix | 1 + aiohttp/http_parser.py | 6 ++++-- tests/test_http_parser.py | 7 +++++++ 3 files changed, 12 insertions(+), 2 deletions(-) create mode 100644 CHANGES/7719.bugfix diff --git a/CHANGES/7719.bugfix b/CHANGES/7719.bugfix new file mode 100644 index 00000000000..b5474398bc4 --- /dev/null +++ b/CHANGES/7719.bugfix @@ -0,0 +1 @@ +Update parser to disallow invalid characters in header field names and stop accepting LF as a request line separator. diff --git a/aiohttp/http_parser.py b/aiohttp/http_parser.py index c2e04139a44..34f4d040c03 100644 --- a/aiohttp/http_parser.py +++ b/aiohttp/http_parser.py @@ -65,7 +65,9 @@ # token = 1*tchar METHRE: Final[Pattern[str]] = re.compile(r"[!#$%&'*+\-.^_`|~0-9A-Za-z]+") VERSRE: Final[Pattern[str]] = re.compile(r"HTTP/(\d).(\d)") -HDRRE: Final[Pattern[bytes]] = re.compile(rb"[\x00-\x1F\x7F()<>@,;:\[\]={} \t\"\\]") +HDRRE: Final[Pattern[bytes]] = re.compile( + rb"[\x00-\x1F\x7F-\xFF()<>@,;:\[\]={} \t\"\\]" +) HEXDIGIT = re.compile(rb"[0-9a-fA-F]+") @@ -540,7 +542,7 @@ def parse_message(self, lines: List[bytes]) -> RawRequestMessage: # request line line = lines[0].decode("utf-8", "surrogateescape") try: - method, path, version = line.split(maxsplit=2) + method, path, version = line.split(" ", maxsplit=2) except ValueError: raise BadStatusLine(line) from None diff --git a/tests/test_http_parser.py b/tests/test_http_parser.py index 4cd38836af2..8b4121be87d 100644 --- a/tests/test_http_parser.py +++ b/tests/test_http_parser.py @@ -177,6 +177,7 @@ def test_cve_2023_37276(parser: Any) -> None: "Baz: abc\x00def", "Foo : bar", # https://www.rfc-editor.org/rfc/rfc9112.html#section-5.1-2 "Foo\t: bar", + "\xffoo: bar", ), ) def test_bad_headers(parser: Any, hdr: str) -> None: @@ -678,6 +679,12 @@ def test_http_request_bad_status_line(parser: Any) -> None: assert r"\n" not in exc_info.value.message +def test_http_request_bad_status_line_whitespace(parser: Any) -> None: + text = b"GET\n/path\fHTTP/1.1\r\n\r\n" + with pytest.raises(http_exceptions.BadStatusLine): + parser.feed_data(text) + + def test_http_request_upgrade(parser: Any) -> None: text = ( b"GET /test HTTP/1.1\r\n" From 2689657d76ada2ccdafbc2aff555addc335b45d7 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 18 Oct 2023 10:34:23 +0000 Subject: [PATCH 16/25] Bump cython from 3.0.3 to 3.0.4 (#7725) Bumps [cython](https://github.com/cython/cython) from 3.0.3 to 3.0.4.
Changelog

Sourced from cython's changelog.

3.0.4 (2023-10-17)

Features added

  • A new compiler directive show_performance_hints was added to disable the newly added performance hint output. (Github issue :issue:5748)

Bugs fixed

  • ```cythonize required ``distutils`` even for operations that did not build binaries. (Github issue :issue:5751`)

  • A regression in 3.0.3 was fixed that prevented calling inline functions from another inline function in .pxd files. (Github issue :issue:5748)

  • Some C compiler warnings were resolved. Patch by Pierre Jolivet. (Github issue :issue:5756)

Commits
  • bd5a82f Disable a test that crashes on i686.
  • f472900 Prepare release of Cython 3.0.4.
  • cc0d167 Update changelog.
  • 6940b34 Use relative import instead of a global one to make sure we end up in the sam...
  • 0000fb4 Avoid importing distutils/setuptools from 'cythonize' unless we really need i...
  • 63aa4ef Allow disabling performance hints via a directive rather than an Option (GH-5...
  • d508f8e Fix regression in 3.0.3 with inline functions in pxd files calling other func...
  • 5e4c7ac docs: Remove usage of outdated "system_packages" option.
  • f4b10ff Fix an incorrect C declaration and an unguarded macro reference (found by gcc...
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cython&package-manager=pip&previous-version=3.0.3&new-version=3.0.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/cython.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index f84f8bed870..2e4e831f90e 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -57,7 +57,7 @@ cryptography==41.0.3 # via # pyjwt # trustme -cython==3.0.3 +cython==3.0.4 # via -r requirements/cython.in distlib==0.3.7 # via virtualenv diff --git a/requirements/cython.txt b/requirements/cython.txt index 3182b6251ed..3b920b22401 100644 --- a/requirements/cython.txt +++ b/requirements/cython.txt @@ -4,7 +4,7 @@ # # pip-compile --allow-unsafe --output-file=requirements/cython.txt --resolver=backtracking --strip-extras requirements/cython.in # -cython==3.0.3 +cython==3.0.4 # via -r requirements/cython.in multidict==6.0.4 # via -r requirements/multidict.in From e2b059fbc91d01c71e81569767ed97b601c253d1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 18 Oct 2023 10:54:21 +0000 Subject: [PATCH 17/25] Bump mypy from 1.6.0 to 1.6.1 (#7726) Bumps [mypy](https://github.com/python/mypy) from 1.6.0 to 1.6.1.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=mypy&package-manager=pip&previous-version=1.6.0&new-version=1.6.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- requirements/test.txt | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 2e4e831f90e..097ba6d236c 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -103,7 +103,7 @@ multidict==6.0.4 # -r requirements/multidict.in # -r requirements/runtime-deps.in # yarl -mypy==1.6.0 ; implementation_name == "cpython" +mypy==1.6.1 ; implementation_name == "cpython" # via # -r requirements/lint.in # -r requirements/test.in diff --git a/requirements/dev.txt b/requirements/dev.txt index 14228976a82..4275f6f7afe 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -100,7 +100,7 @@ multidict==6.0.4 # via # -r requirements/runtime-deps.in # yarl -mypy==1.6.0 ; implementation_name == "cpython" +mypy==1.6.1 ; implementation_name == "cpython" # via # -r requirements/lint.in # -r requirements/test.in diff --git a/requirements/lint.txt b/requirements/lint.txt index 5a51f94ba81..d0034f8bd0f 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -22,7 +22,7 @@ identify==2.5.26 # via pre-commit iniconfig==2.0.0 # via pytest -mypy==1.6.0 ; implementation_name == "cpython" +mypy==1.6.1 ; implementation_name == "cpython" # via -r requirements/lint.in mypy-extensions==1.0.0 # via mypy diff --git a/requirements/test.txt b/requirements/test.txt index e4d371c1ab9..b76e408c661 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -51,7 +51,7 @@ multidict==6.0.4 # via # -r requirements/runtime-deps.in # yarl -mypy==1.6.0 ; implementation_name == "cpython" +mypy==1.6.1 ; implementation_name == "cpython" # via -r requirements/test.in mypy-extensions==1.0.0 # via mypy From a7bc5e9eeae7c5c90898411962e9a74bf10a9cef Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Anton=20Gr=C3=BCbel?= Date: Wed, 18 Oct 2023 16:47:23 +0200 Subject: [PATCH 18/25] replace freezegun with time_machine (#7656) --- requirements/constraints.txt | 16 +++++++++++++--- requirements/dev.txt | 16 +++++++++++++--- requirements/test.in | 2 +- requirements/test.txt | 6 +++--- tests/test_cookiejar.py | 27 ++++++++++++++++++++++----- 5 files changed, 52 insertions(+), 15 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 097ba6d236c..30d0a44a5fa 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -67,8 +67,6 @@ exceptiongroup==1.1.2 # via pytest filelock==3.12.2 # via virtualenv -freezegun==1.2.2 - # via -r requirements/test.in frozenlist==1.4.0 # via # -r requirements/runtime-deps.in @@ -88,6 +86,10 @@ idna==3.4 # yarl imagesize==1.4.1 # via sphinx +importlib-metadata==6.8.0 + # via sphinx +importlib-resources==6.1.0 + # via towncrier incremental==22.10.0 # via towncrier iniconfig==2.0.0 @@ -156,9 +158,11 @@ pytest-cov==4.1.0 pytest-mock==3.11.1 # via -r requirements/test.in python-dateutil==2.8.2 - # via freezegun + # via time-machine python-on-whales==0.65.0 # via -r requirements/test.in +pytz==2023.3.post1 + # via babel pyyaml==6.0.1 # via pre-commit re-assert==1.1.0 @@ -202,6 +206,8 @@ sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" # via -r requirements/doc-spelling.in sphinxcontrib-towncrier==0.3.2a0 # via -r requirements/doc.in +time-machine==2.13.0 ; implementation_name == "cpython" + # via -r requirements/test.in tomli==2.0.1 # via # build @@ -249,6 +255,10 @@ wheel==0.41.0 # via pip-tools yarl==1.9.2 # via -r requirements/runtime-deps.in +zipp==3.17.0 + # via + # importlib-metadata + # importlib-resources # The following packages are considered to be unsafe in a requirements file: pip==23.2.1 diff --git a/requirements/dev.txt b/requirements/dev.txt index 4275f6f7afe..eeef9bb6155 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -65,8 +65,6 @@ exceptiongroup==1.1.2 # via pytest filelock==3.12.2 # via virtualenv -freezegun==1.2.2 - # via -r requirements/test.in frozenlist==1.4.0 # via # -r requirements/runtime-deps.in @@ -86,6 +84,10 @@ idna==3.4 # yarl imagesize==1.4.1 # via sphinx +importlib-metadata==6.8.0 + # via sphinx +importlib-resources==6.1.0 + # via towncrier incremental==22.10.0 # via towncrier iniconfig==2.0.0 @@ -151,9 +153,11 @@ pytest-cov==4.1.0 pytest-mock==3.11.1 # via -r requirements/test.in python-dateutil==2.8.2 - # via freezegun + # via time-machine python-on-whales==0.65.0 # via -r requirements/test.in +pytz==2023.3.post1 + # via babel pyyaml==6.0.1 # via pre-commit re-assert==1.1.0 @@ -194,6 +198,8 @@ sphinxcontrib-serializinghtml==1.1.5 # via sphinx sphinxcontrib-towncrier==0.3.2a0 # via -r requirements/doc.in +time-machine==2.13.0 ; implementation_name == "cpython" + # via -r requirements/test.in tomli==2.0.1 # via # build @@ -241,6 +247,10 @@ wheel==0.41.0 # via pip-tools yarl==1.9.2 # via -r requirements/runtime-deps.in +zipp==3.17.0 + # via + # importlib-metadata + # importlib-resources # The following packages are considered to be unsafe in a requirements file: pip==23.2.1 diff --git a/requirements/test.in b/requirements/test.in index 9e9161272bf..417d45959be 100644 --- a/requirements/test.in +++ b/requirements/test.in @@ -2,7 +2,6 @@ -c broken-projects.in coverage -freezegun mypy; implementation_name == "cpython" proxy.py pytest @@ -11,5 +10,6 @@ pytest-mock python-on-whales re-assert setuptools-git +time-machine; implementation_name == "cpython" trustme; platform_machine != "i686" # no 32-bit wheels wait-for-it diff --git a/requirements/test.txt b/requirements/test.txt index b76e408c661..329e5e6f968 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -32,8 +32,6 @@ cryptography==41.0.3 # via trustme exceptiongroup==1.1.2 # via pytest -freezegun==1.2.2 - # via -r requirements/test.in frozenlist==1.4.0 # via # -r requirements/runtime-deps.in @@ -79,7 +77,7 @@ pytest-cov==4.1.0 pytest-mock==3.11.1 # via -r requirements/test.in python-dateutil==2.8.2 - # via freezegun + # via time-machine python-on-whales==0.65.0 # via -r requirements/test.in re-assert==1.1.0 @@ -92,6 +90,8 @@ setuptools-git==1.2 # via -r requirements/test.in six==1.16.0 # via python-dateutil +time-machine==2.13.0 ; implementation_name == "cpython" + # via -r requirements/test.in tomli==2.0.1 # via # coverage diff --git a/tests/test_cookiejar.py b/tests/test_cookiejar.py index 755838fbeac..00a32708756 100644 --- a/tests/test_cookiejar.py +++ b/tests/test_cookiejar.py @@ -4,17 +4,22 @@ import itertools import pathlib import pickle +import sys import unittest from http.cookies import BaseCookie, Morsel, SimpleCookie from typing import Any from unittest import mock import pytest -from freezegun import freeze_time from yarl import URL from aiohttp import CookieJar, DummyCookieJar +try: + from time_machine import travel +except ImportError: + travel = None # type: ignore[assignment] + def dump_cookiejar() -> bytes: # pragma: no cover """Create pickled data for test_pickle_format().""" @@ -418,10 +423,10 @@ def timed_request(self, url: Any, update_time: Any, send_time: Any): elif isinstance(send_time, float): send_time = datetime.datetime.fromtimestamp(send_time) - with freeze_time(update_time): + with travel(update_time, tick=False): self.jar.update_cookies(self.cookies_to_send) - with freeze_time(send_time): + with travel(send_time, tick=False): cookies_sent = self.jar.filter_cookies(URL(url)) self.jar.clear() @@ -608,6 +613,10 @@ def test_path_value(self) -> None: self.assertEqual(cookies_received["path-cookie"]["path"], "/somepath") self.assertEqual(cookies_received["wrong-path-cookie"]["path"], "/") + @unittest.skipIf( + sys.implementation.name != "cpython", + reason="time_machine leverages CPython specific pointers https://github.com/adamchainz/time-machine/issues/305", + ) def test_expires(self) -> None: ts_before = datetime.datetime( 1975, 1, 1, tzinfo=datetime.timezone.utc @@ -629,6 +638,10 @@ def test_expires(self) -> None: self.assertEqual(set(cookies_sent.keys()), {"shared-cookie"}) + @unittest.skipIf( + sys.implementation.name != "cpython", + reason="time_machine leverages CPython specific pointers https://github.com/adamchainz/time-machine/issues/305", + ) def test_max_age(self) -> None: cookies_sent = self.timed_request("http://maxagetest.com/", 1000, 1000) @@ -776,6 +789,10 @@ async def test_cookie_jar_clear_all() -> None: assert len(sut) == 0 +@pytest.mark.skipif( + sys.implementation.name != "cpython", + reason="time_machine leverages CPython specific pointers https://github.com/adamchainz/time-machine/issues/305", +) async def test_cookie_jar_clear_expired(): sut = CookieJar() @@ -784,11 +801,11 @@ async def test_cookie_jar_clear_expired(): cookie["foo"] = "bar" cookie["foo"]["expires"] = "Tue, 1 Jan 1990 12:00:00 GMT" - with freeze_time("1980-01-01"): + with travel("1980-01-01", tick=False): sut.update_cookies(cookie) sut.clear(lambda x: False) - with freeze_time("1980-01-01"): + with travel("1980-01-01", tick=False): assert len(sut) == 0 From 3b68b2f3abe8a9e554e4bbd8bcabb9847f364806 Mon Sep 17 00:00:00 2001 From: Harmon Date: Fri, 20 Oct 2023 12:47:31 -0500 Subject: [PATCH 19/25] Fix, update, and improve client exceptions documentation (#7733) ## What do these changes do? In the *Hierarchy of exceptions* section of the *Client Reference* documentation: - Fix `ServerConnectionError` indentation, denoting that it subclasses `ClientConnectionError`, not `ClientOSError` - Fix `ServerFingerprintMismatch` indentation, denoting that it subclasses `ServerConnectionError`, not `ClientOSError` - Normalize the formatting so as to consistently use two, rather than three, spaces for indentation - Normalize the formatting so as to consistently include a blank newline between exceptions - Order exceptions alphabetically - Add `TooManyRedirects` - Add `UnixClientConnectorError` Also order the documentation of client exceptions alphabetically by hierarchy --- CHANGES/7733.doc | 1 + docs/client_reference.rst | 75 +++++++++++++++++++++------------------ 2 files changed, 42 insertions(+), 34 deletions(-) create mode 100644 CHANGES/7733.doc diff --git a/CHANGES/7733.doc b/CHANGES/7733.doc new file mode 100644 index 00000000000..12d682a86df --- /dev/null +++ b/CHANGES/7733.doc @@ -0,0 +1 @@ +Fix, update, and improve client exceptions documentation. diff --git a/docs/client_reference.rst b/docs/client_reference.rst index df584b23d03..4c89351e841 100644 --- a/docs/client_reference.rst +++ b/docs/client_reference.rst @@ -2138,13 +2138,6 @@ Response errors .. deprecated:: 3.1 -.. class:: WSServerHandshakeError - - Web socket server response error. - - Derived from :exc:`ClientResponseError` - - .. class:: ContentTypeError Invalid content type. @@ -2165,6 +2158,13 @@ Response errors .. versionadded:: 3.2 + +.. class:: WSServerHandshakeError + + Web socket server response error. + + Derived from :exc:`ClientResponseError` + Connection errors ^^^^^^^^^^^^^^^^^ @@ -2191,14 +2191,6 @@ Connection errors Derived from :exc:`ClientConnectorError` -.. class:: UnixClientConnectorError - - Derived from :exc:`ClientConnectorError` - -.. class:: ServerConnectionError - - Derived from :exc:`ClientConnectionError` - .. class:: ClientSSLError Derived from :exc:`ClientConnectorError` @@ -2215,6 +2207,14 @@ Connection errors Derived from :exc:`ClientSSLError` and :exc:`ssl.CertificateError` +.. class:: UnixClientConnectorError + + Derived from :exc:`ClientConnectorError` + +.. class:: ServerConnectionError + + Derived from :exc:`ClientConnectionError` + .. class:: ServerDisconnectedError Server disconnected. @@ -2226,51 +2226,58 @@ Connection errors Partially parsed HTTP message (optional). -.. class:: ServerTimeoutError - - Server operation timeout: read timeout, etc. - - Derived from :exc:`ServerConnectionError` and :exc:`asyncio.TimeoutError` - .. class:: ServerFingerprintMismatch Server fingerprint mismatch. Derived from :exc:`ServerConnectionError` +.. class:: ServerTimeoutError + + Server operation timeout: read timeout, etc. + + Derived from :exc:`ServerConnectionError` and :exc:`asyncio.TimeoutError` + Hierarchy of exceptions ^^^^^^^^^^^^^^^^^^^^^^^ * :exc:`ClientError` - * :exc:`ClientResponseError` - - * :exc:`ContentTypeError` - * :exc:`WSServerHandshakeError` - * :exc:`~aiohttp.ClientHttpProxyError` - * :exc:`ClientConnectionError` * :exc:`ClientOSError` * :exc:`ClientConnectorError` - * :exc:`ClientSSLError` + * :exc:`ClientProxyConnectionError` + + * :exc:`ClientSSLError` - * :exc:`ClientConnectorCertificateError` + * :exc:`ClientConnectorCertificateError` - * :exc:`ClientConnectorSSLError` + * :exc:`ClientConnectorSSLError` - * :exc:`ClientProxyConnectionError` + * :exc:`UnixClientConnectorError` - * :exc:`ServerConnectionError` + * :exc:`ServerConnectionError` - * :exc:`ServerDisconnectedError` - * :exc:`ServerTimeoutError` + * :exc:`ServerDisconnectedError` * :exc:`ServerFingerprintMismatch` + * :exc:`ServerTimeoutError` + * :exc:`ClientPayloadError` + * :exc:`ClientResponseError` + + * :exc:`~aiohttp.ClientHttpProxyError` + + * :exc:`ContentTypeError` + + * :exc:`TooManyRedirects` + + * :exc:`WSServerHandshakeError` + * :exc:`InvalidURL` From 93c9abbd6d6837e451f687a3245fbbd406b490a3 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Mon, 23 Oct 2023 19:47:05 +0100 Subject: [PATCH 20/25] Reenable uvloop on Python 3.12 (#7729) --- requirements/base.in | 2 +- requirements/base.txt | 2 +- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.in | 2 +- requirements/lint.txt | 2 +- requirements/test.txt | 2 +- 7 files changed, 7 insertions(+), 7 deletions(-) diff --git a/requirements/base.in b/requirements/base.in index 5404c474f5c..df67f78afde 100644 --- a/requirements/base.in +++ b/requirements/base.in @@ -2,4 +2,4 @@ -r runtime-deps.in gunicorn -uvloop; platform_system != "Windows" and implementation_name == "cpython" and python_version < "3.12" # MagicStack/uvloop#14 # MagicStack/uvloop#547 +uvloop; platform_system != "Windows" and implementation_name == "cpython" # MagicStack/uvloop#14 diff --git a/requirements/base.txt b/requirements/base.txt index c266d1bb69d..26dd2799b37 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -34,7 +34,7 @@ pycparser==2.21 # via cffi typing-extensions==4.7.1 # via -r requirements/typing-extensions.in -uvloop==0.17.0 ; platform_system != "Windows" and implementation_name == "cpython" and python_version < "3.12" +uvloop==0.19.0 ; platform_system != "Windows" and implementation_name == "cpython" # via -r requirements/base.in yarl==1.9.2 # via -r requirements/runtime-deps.in diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 30d0a44a5fa..3e0ead26225 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -241,7 +241,7 @@ uritemplate==4.1.1 # via gidgethub urllib3==2.0.4 # via requests -uvloop==0.17.0 ; platform_system != "Windows" and python_version < "3.12" +uvloop==0.19.0 ; platform_system != "Windows" # via # -r requirements/base.in # -r requirements/lint.in diff --git a/requirements/dev.txt b/requirements/dev.txt index eeef9bb6155..5d74609a127 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -233,7 +233,7 @@ uritemplate==4.1.1 # via gidgethub urllib3==2.0.4 # via requests -uvloop==0.17.0 ; platform_system != "Windows" and implementation_name == "cpython" and python_version < "3.12" +uvloop==0.19.0 ; platform_system != "Windows" and implementation_name == "cpython" # via # -r requirements/base.in # -r requirements/lint.in diff --git a/requirements/lint.in b/requirements/lint.in index 14c0fd84638..34616155912 100644 --- a/requirements/lint.in +++ b/requirements/lint.in @@ -5,4 +5,4 @@ mypy; implementation_name == "cpython" pre-commit pytest slotscheck -uvloop; platform_system != "Windows" and python_version < "3.12" +uvloop; platform_system != "Windows" diff --git a/requirements/lint.txt b/requirements/lint.txt index d0034f8bd0f..40ec9520c35 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -52,7 +52,7 @@ typing-extensions==4.7.1 # -r requirements/typing-extensions.in # aioredis # mypy -uvloop==0.17.0 ; platform_system != "Windows" and python_version < "3.12" +uvloop==0.19.0 ; platform_system != "Windows" # via -r requirements/lint.in virtualenv==20.24.2 # via pre-commit diff --git a/requirements/test.txt b/requirements/test.txt index 329e5e6f968..77a1a4c288c 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -112,7 +112,7 @@ typing-extensions==4.7.1 # typer urllib3==2.0.4 # via requests -uvloop==0.17.0 ; platform_system != "Windows" and implementation_name == "cpython" and python_version < "3.12" +uvloop==0.19.0 ; platform_system != "Windows" and implementation_name == "cpython" # via -r requirements/base.in wait-for-it==2.2.2 # via -r requirements/test.in From d8936aacf7d602a58228d95fd469bbf01a43216d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 23 Oct 2023 19:39:03 +0000 Subject: [PATCH 21/25] Bump urllib3 from 2.0.6 to 2.0.7 in /requirements (#7721) Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.0.6 to 2.0.7.
Release notes

Sourced from urllib3's releases.

2.0.7

  • Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other" redirect responses. (GHSA-g4mx-q9vg-27p4)
Changelog

Sourced from urllib3's changelog.

2.0.7 (2023-10-17)

  • Made body stripped from HTTP requests changing the request method to GET after HTTP 303 "See Other" redirect responses.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.0.6&new-version=2.0.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/aio-libs/aiohttp/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/doc-spelling.txt | 2 +- requirements/doc.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/requirements/doc-spelling.txt b/requirements/doc-spelling.txt index f9fdecdebd8..ee5a37094c9 100644 --- a/requirements/doc-spelling.txt +++ b/requirements/doc-spelling.txt @@ -80,7 +80,7 @@ towncrier==23.6.0 # via # -r doc.in # sphinxcontrib-towncrier -urllib3==2.0.6 +urllib3==2.0.7 # via requests webcolors==1.13 # via blockdiag diff --git a/requirements/doc.txt b/requirements/doc.txt index afb156ae5ca..74fa3ac5be2 100644 --- a/requirements/doc.txt +++ b/requirements/doc.txt @@ -75,7 +75,7 @@ towncrier==23.6.0 # via # -r doc.in # sphinxcontrib-towncrier -urllib3==2.0.6 +urllib3==2.0.7 # via requests webcolors==1.13 # via blockdiag From c4234090a5ce155dac57f2f2f51986e75650f65f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 24 Oct 2023 11:34:08 +0000 Subject: [PATCH 22/25] Bump towncrier from 23.6.0 to 23.10.0 (#7744) Bumps [towncrier](https://github.com/twisted/towncrier) from 23.6.0 to 23.10.0.
Release notes

Sourced from towncrier's releases.

Towncrier 23.10.0

towncrier 23.10.0 (2023-10-23)

No significant changes since the previous release candidate.

Features

  • Python 3.12 is now officially supported. ([#541](https://github.com/twisted/towncrier/issues/541) <https://github.com/twisted/towncrier/issues/541>_)

  • Initial support was added for monorepo-style setup. One project with multiple independent news files stored in separate sub-directories, that share the same towncrier config. ([#548](https://github.com/twisted/towncrier/issues/548) <https://github.com/twisted/towncrier/issues/548>_)

  • Two newlines are no longer always added between the current release notes and the previous content. The newlines are now defined only inside the template.

    Important! If you're using a custom template and want to keep the same whitespace between releases, you may have to modify your template. ([#552](https://github.com/twisted/towncrier/issues/552) <https://github.com/twisted/towncrier/issues/552>_)

Bugfixes

  • Towncrier now vendors the click-default-group package that prevented installations on modern Pips. ([#540](https://github.com/twisted/towncrier/issues/540) <https://github.com/twisted/towncrier/issues/540>_)

Improved Documentation

  • The markdown docs now use the default markdown template rather than a simpler custom one. ([#545](https://github.com/twisted/towncrier/issues/545) <https://github.com/twisted/towncrier/issues/545>_)
  • Cleanup a duplicate backtick in the tutorial. ([#551](https://github.com/twisted/towncrier/issues/551) <https://github.com/twisted/towncrier/issues/551>_)

Deprecations and Removals

  • The support for Python 3.7 has been dropped. ([#521](https://github.com/twisted/towncrier/issues/521) <https://github.com/twisted/towncrier/issues/521>_)

Misc

  • [#481](https://github.com/twisted/towncrier/issues/481) <https://github.com/twisted/towncrier/issues/481>, [#520](https://github.com/twisted/towncrier/issues/520) <https://github.com/twisted/towncrier/issues/520>, [#522](https://github.com/twisted/towncrier/issues/522) <https://github.com/twisted/towncrier/issues/522>, [#523](https://github.com/twisted/towncrier/issues/523) <https://github.com/twisted/towncrier/issues/523>, [#529](https://github.com/twisted/towncrier/issues/529) <https://github.com/twisted/towncrier/issues/529>, [#536](https://github.com/twisted/towncrier/issues/536) <https://github.com/twisted/towncrier/issues/536>

Towncrier 23.10.0rc1

towncrier 23.10.0rc1 (2023-10-23)

Features

... (truncated)

Changelog

Sourced from towncrier's changelog.

towncrier 23.10.0 (2023-10-24)

No significant changes since the previous release candidate.

Features

  • Python 3.12 is now officially supported. ([#541](https://github.com/twisted/towncrier/issues/541) <https://github.com/twisted/towncrier/issues/541>_)

  • Initial support was added for monorepo-style setup. One project with multiple independent news files stored in separate sub-directories, that share the same towncrier config. ([#548](https://github.com/twisted/towncrier/issues/548) <https://github.com/twisted/towncrier/issues/548>_)

  • Two newlines are no longer always added between the current release notes and the previous content. The newlines are now defined only inside the template.

    Important! If you're using a custom template and want to keep the same whitespace between releases, you may have to modify your template. ([#552](https://github.com/twisted/towncrier/issues/552) <https://github.com/twisted/towncrier/issues/552>_)

Bugfixes

  • Towncrier now vendors the click-default-group package that prevented installations on modern Pips. ([#540](https://github.com/twisted/towncrier/issues/540) <https://github.com/twisted/towncrier/issues/540>_)

Improved Documentation

  • The markdown docs now use the default markdown template rather than a simpler custom one. ([#545](https://github.com/twisted/towncrier/issues/545) <https://github.com/twisted/towncrier/issues/545>_)
  • Cleanup a duplicate backtick in the tutorial. ([#551](https://github.com/twisted/towncrier/issues/551) <https://github.com/twisted/towncrier/issues/551>_)

Deprecations and Removals

  • The support for Python 3.7 has been dropped. ([#521](https://github.com/twisted/towncrier/issues/521) <https://github.com/twisted/towncrier/issues/521>_)

Misc

  • [#481](https://github.com/twisted/towncrier/issues/481) <https://github.com/twisted/towncrier/issues/481>, [#520](https://github.com/twisted/towncrier/issues/520) <https://github.com/twisted/towncrier/issues/520>, [#522](https://github.com/twisted/towncrier/issues/522) <https://github.com/twisted/towncrier/issues/522>, [#523](https://github.com/twisted/towncrier/issues/523) <https://github.com/twisted/towncrier/issues/523>, [#529](https://github.com/twisted/towncrier/issues/529) <https://github.com/twisted/towncrier/issues/529>, [#536](https://github.com/twisted/towncrier/issues/536) <https://github.com/twisted/towncrier/issues/536>
Commits
  • 347e944 Update for final release.
  • ee85740 Do RC1 instead of RC0.
  • 0c3b82c venv/bin/towncrier build --yes
  • 610247f Update version.
  • 0b023fa Config location separate from directory containing news file and fragments (#...
  • 3f24b6e Whitespace between releases can be configured in the template. (#553)
  • 7f37ab5 docs(tutorial): Cleanup duplicate backtick typo (#551)
  • 5b732be [pre-commit.ci] pre-commit autoupdate (#549)
  • 239f841 use markdown default template in markdown docs (#546)
  • 806bec1 Add 3.12 (#541)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=towncrier&package-manager=pip&previous-version=23.6.0&new-version=23.10.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 15 +-------------- requirements/dev.txt | 15 +-------------- requirements/doc-spelling.txt | 22 ++++++++++------------ requirements/doc.txt | 20 +++++++++----------- 4 files changed, 21 insertions(+), 51 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 3e0ead26225..044cd5e678d 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -41,14 +41,11 @@ cherry-picker==2.2.0 click==8.1.6 # via # cherry-picker - # click-default-group # pip-tools # slotscheck # towncrier # typer # wait-for-it -click-default-group==1.2.2 - # via towncrier coverage==7.3.2 # via # -r requirements/test.in @@ -86,10 +83,6 @@ idna==3.4 # yarl imagesize==1.4.1 # via sphinx -importlib-metadata==6.8.0 - # via sphinx -importlib-resources==6.1.0 - # via towncrier incremental==22.10.0 # via towncrier iniconfig==2.0.0 @@ -161,8 +154,6 @@ python-dateutil==2.8.2 # via time-machine python-on-whales==0.65.0 # via -r requirements/test.in -pytz==2023.3.post1 - # via babel pyyaml==6.0.1 # via pre-commit re-assert==1.1.0 @@ -219,7 +210,7 @@ tomli==2.0.1 # pytest # slotscheck # towncrier -towncrier==23.6.0 +towncrier==23.10.0 # via # -r requirements/doc.in # sphinxcontrib-towncrier @@ -255,10 +246,6 @@ wheel==0.41.0 # via pip-tools yarl==1.9.2 # via -r requirements/runtime-deps.in -zipp==3.17.0 - # via - # importlib-metadata - # importlib-resources # The following packages are considered to be unsafe in a requirements file: pip==23.2.1 diff --git a/requirements/dev.txt b/requirements/dev.txt index 5d74609a127..ae41b6d33df 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -41,14 +41,11 @@ cherry-picker==2.2.0 click==8.1.6 # via # cherry-picker - # click-default-group # pip-tools # slotscheck # towncrier # typer # wait-for-it -click-default-group==1.2.2 - # via towncrier coverage==7.3.2 # via # -r requirements/test.in @@ -84,10 +81,6 @@ idna==3.4 # yarl imagesize==1.4.1 # via sphinx -importlib-metadata==6.8.0 - # via sphinx -importlib-resources==6.1.0 - # via towncrier incremental==22.10.0 # via towncrier iniconfig==2.0.0 @@ -156,8 +149,6 @@ python-dateutil==2.8.2 # via time-machine python-on-whales==0.65.0 # via -r requirements/test.in -pytz==2023.3.post1 - # via babel pyyaml==6.0.1 # via pre-commit re-assert==1.1.0 @@ -211,7 +202,7 @@ tomli==2.0.1 # pytest # slotscheck # towncrier -towncrier==23.6.0 +towncrier==23.10.0 # via # -r requirements/doc.in # sphinxcontrib-towncrier @@ -247,10 +238,6 @@ wheel==0.41.0 # via pip-tools yarl==1.9.2 # via -r requirements/runtime-deps.in -zipp==3.17.0 - # via - # importlib-metadata - # importlib-resources # The following packages are considered to be unsafe in a requirements file: pip==23.2.1 diff --git a/requirements/doc-spelling.txt b/requirements/doc-spelling.txt index ee5a37094c9..e8afc882862 100644 --- a/requirements/doc-spelling.txt +++ b/requirements/doc-spelling.txt @@ -5,7 +5,7 @@ # pip-compile --allow-unsafe --output-file=requirements/doc-spelling.txt --resolver=backtracking --strip-extras requirements/doc-spelling.in # aiohttp-theme==0.1.6 - # via -r doc.in + # via -r requirements/doc.in alabaster==0.7.13 # via sphinx babel==2.12.1 @@ -17,10 +17,6 @@ certifi==2023.7.22 charset-normalizer==3.3.0 # via requests click==8.1.6 - # via - # click-default-group - # towncrier -click-default-group==1.2.2 # via towncrier docutils==0.20.1 # via sphinx @@ -42,7 +38,7 @@ packaging==23.1 # via sphinx pillow==9.5.0 # via - # -c broken-projects.in + # -c requirements/broken-projects.in # blockdiag pyenchant==3.2.2 # via sphinxcontrib-spelling @@ -54,14 +50,14 @@ snowballstemmer==2.2.0 # via sphinx sphinx==7.1.2 # via - # -r doc.in + # -r requirements/doc.in # sphinxcontrib-blockdiag # sphinxcontrib-spelling # sphinxcontrib-towncrier sphinxcontrib-applehelp==1.0.4 # via sphinx sphinxcontrib-blockdiag==3.0.0 - # via -r doc.in + # via -r requirements/doc.in sphinxcontrib-devhelp==1.0.2 # via sphinx sphinxcontrib-htmlhelp==2.0.1 @@ -73,12 +69,14 @@ sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 # via sphinx sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" - # via -r doc-spelling.in + # via -r requirements/doc-spelling.in sphinxcontrib-towncrier==0.3.2a0 - # via -r doc.in -towncrier==23.6.0 + # via -r requirements/doc.in +tomli==2.0.1 + # via towncrier +towncrier==23.10.0 # via - # -r doc.in + # -r requirements/doc.in # sphinxcontrib-towncrier urllib3==2.0.7 # via requests diff --git a/requirements/doc.txt b/requirements/doc.txt index 74fa3ac5be2..b80760fdd47 100644 --- a/requirements/doc.txt +++ b/requirements/doc.txt @@ -5,7 +5,7 @@ # pip-compile --allow-unsafe --output-file=requirements/doc.txt --resolver=backtracking --strip-extras requirements/doc.in # aiohttp-theme==0.1.6 - # via -r doc.in + # via -r requirements/doc.in alabaster==0.7.13 # via sphinx babel==2.12.1 @@ -17,10 +17,6 @@ certifi==2023.7.22 charset-normalizer==3.3.0 # via requests click==8.1.6 - # via - # click-default-group - # towncrier -click-default-group==1.2.2 # via towncrier docutils==0.20.1 # via sphinx @@ -42,7 +38,7 @@ packaging==23.1 # via sphinx pillow==9.5.0 # via - # -c broken-projects.in + # -c requirements/broken-projects.in # blockdiag pygments==2.15.1 # via sphinx @@ -52,13 +48,13 @@ snowballstemmer==2.2.0 # via sphinx sphinx==7.1.2 # via - # -r doc.in + # -r requirements/doc.in # sphinxcontrib-blockdiag # sphinxcontrib-towncrier sphinxcontrib-applehelp==1.0.4 # via sphinx sphinxcontrib-blockdiag==3.0.0 - # via -r doc.in + # via -r requirements/doc.in sphinxcontrib-devhelp==1.0.2 # via sphinx sphinxcontrib-htmlhelp==2.0.1 @@ -70,10 +66,12 @@ sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 # via sphinx sphinxcontrib-towncrier==0.3.2a0 - # via -r doc.in -towncrier==23.6.0 + # via -r requirements/doc.in +tomli==2.0.1 + # via towncrier +towncrier==23.10.0 # via - # -r doc.in + # -r requirements/doc.in # sphinxcontrib-towncrier urllib3==2.0.7 # via requests From 0566987181392437af7fc0eec829775cf6b3523f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 24 Oct 2023 11:43:07 +0000 Subject: [PATCH 23/25] Bump actions/setup-node from 3 to 4 (#7745) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/setup-node](https://github.com/actions/setup-node) from 3 to 4.
Release notes

Sourced from actions/setup-node's releases.

v4.0.0

What's Changed

In scope of this release we changed version of node runtime for action from node16 to node20 and updated dependencies in actions/setup-node#866

Besides, release contains such changes as:

New Contributors

Full Changelog: https://github.com/actions/setup-node/compare/v3...v4.0.0

v3.8.2

What's Changed

Full Changelog: https://github.com/actions/setup-node/compare/v3...v3.8.2

v3.8.1

What's Changed

In scope of this release, the filter was removed within the cache-save step by @​dmitry-shibanov in actions/setup-node#831. It is filtered and checked in the toolkit/cache library.

Full Changelog: https://github.com/actions/setup-node/compare/v3...v3.8.1

v3.8.0

What's Changed

Bug fixes:

Feature implementations:

Documentation changes:

Update dependencies:

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/setup-node&package-manager=github_actions&previous-version=3&new-version=4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci-cd.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 5f13f951944..3f1ef7a5cfa 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -118,7 +118,7 @@ jobs: path: vendor/llhttp/build - name: Setup NodeJS if: steps.cache.outputs.cache-hit != 'true' - uses: actions/setup-node@v3 + uses: actions/setup-node@v4 with: node-version: '14' - name: Generate llhttp sources From 85f7b98976bd7808ea1497af1a7e5299968da982 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 25 Oct 2023 11:21:26 +0000 Subject: [PATCH 24/25] Bump pytest from 7.4.2 to 7.4.3 (#7748) Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.2 to 7.4.3.
Release notes

Sourced from pytest's releases.

pytest 7.4.3 (2023-10-24)

Bug Fixes

  • #10447: Markers are now considered in the reverse mro order to ensure base class markers are considered first -- this resolves a regression.

  • #11239: Fixed := in asserts impacting unrelated test cases.

  • #11439: Handled an edge case where :data:sys.stderr might already be closed when :ref:faulthandler is tearing down.

Commits
  • 2390610 Tweak changelog.rst
  • a0714aa Prepare release version 7.4.3
  • 44ad1c9 [7.4.x] fix #10447 - consider marks in reverse mro order to give base classes...
  • 5dc7725 [7.4.x] Ensure logging tests always cleanup after themselves (#11541)
  • a517827 [7.4.x] Configure ReadTheDocs to fail on warnings (#11540)
  • 21fe071 [7.4.x] fix for ValueError raised in faulthandler teardown code (#11455)
  • f8bb857 Force terminal width when running tests (#11425) (#11432)
  • 1944dc0 [7.4.x] Fix --import-mode=importlib when root contains __init__.py file (#1...
  • 946634c Merge pull request #11419 from nicoddemus/backport-11414-to-7.4.x
  • d849a3e [7.4.x] fix: closes #11343's [attr-defined] type errors (#11421)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pytest&package-manager=pip&previous-version=7.4.2&new-version=7.4.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- requirements/constraints.txt | 2 +- requirements/dev.txt | 2 +- requirements/lint.txt | 2 +- requirements/test.txt | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index 044cd5e678d..042977416b7 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -140,7 +140,7 @@ pyjwt==2.8.0 # via gidgethub pyproject-hooks==1.0.0 # via build -pytest==7.4.2 +pytest==7.4.3 # via # -r requirements/lint.in # -r requirements/test.in diff --git a/requirements/dev.txt b/requirements/dev.txt index ae41b6d33df..a1b0ee2df1d 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -135,7 +135,7 @@ pyjwt==2.8.0 # via gidgethub pyproject-hooks==1.0.0 # via build -pytest==7.4.2 +pytest==7.4.3 # via # -r requirements/lint.in # -r requirements/test.in diff --git a/requirements/lint.txt b/requirements/lint.txt index 40ec9520c35..b10574614ae 100644 --- a/requirements/lint.txt +++ b/requirements/lint.txt @@ -36,7 +36,7 @@ pluggy==1.2.0 # via pytest pre-commit==3.5.0 # via -r requirements/lint.in -pytest==7.4.2 +pytest==7.4.3 # via -r requirements/lint.in pyyaml==6.0.1 # via pre-commit diff --git a/requirements/test.txt b/requirements/test.txt index 77a1a4c288c..344b3df47b3 100644 --- a/requirements/test.txt +++ b/requirements/test.txt @@ -67,7 +67,7 @@ pycparser==2.21 # via cffi pydantic==1.10.12 # via python-on-whales -pytest==7.4.2 +pytest==7.4.3 # via # -r requirements/test.in # pytest-cov From 0dc39e46c459a876faa936871879a1a1089ccdbf Mon Sep 17 00:00:00 2001 From: Sviatoslav Sydorenko Date: Thu, 26 Oct 2023 11:48:08 +0200 Subject: [PATCH 25/25] Enable testing merge queues @ GitHub Actions CI/CD (#7749) --- .github/workflows/ci-cd.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 3f1ef7a5cfa..bfd106104ac 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -1,6 +1,7 @@ name: CI on: + merge_group: push: branches: - 'master'