Support passing a custom server name parameter on HTTPS connection #7541

multani · 2023-08-19T17:14:25Z

What do these changes do?

This adds the missing support to set the server_hostname setting when creating TCP connection, when the underlying connection is authenticated using TLS.

See the documentation for the 2 stdlib functions:

This would be needed to support features in clients using aiohttp, such as tomplus/kubernetes_asyncio#267

Are there changes in behavior for the user?

The default behavior should not change, but this would allow on a per-connection basis to specify a custom server name to check the certificate name against.

Related issue number

Closes: #7114

(for reference, similar implementation in urllib3: urllib3/urllib3#1397)

Checklist

I think the code is well written
Unit tests for the changes exist
Documentation reflects the changes
If you provide code modification, please add yourself to CONTRIBUTORS.txt
- The format is <Name> <Surname>.
- Please keep alphabetical order, the file is sorted by names.
Add a new news fragment into the CHANGES folder
- name it <issue_id>.<type> for example (588.bugfix)
- if you don't have an issue_id change it to the pr id after creating the pr
- ensure type is one of the following:
  - .feature: Signifying a new feature.
  - .bugfix: Signifying a bug fix.
  - .doc: Signifying a documentation improvement.
  - .removal: Signifying a deprecation or removal of public API.
  - .misc: A ticket has been closed, but it is not of interest to users.
- Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

aiohttp/connector.py

tests/test_connector.py

Dreamsorcerer · 2023-08-19T20:06:19Z

Also, if the tests still don't work after those updates on 3.8 due to missing AsyncMock, feel free to add a pytest.skipif for 3.8.

aiohttp/connector.py

docs/client_reference.rst

multani · 2023-08-19T20:56:02Z

@Dreamsorcerer thanks for all the good comments! 🙇

I made all the changes requested, it's better now! 👍 I'll keep an eye on the CI.

Dreamsorcerer · 2023-08-19T21:36:33Z

Oh, I just realised you're pointing to the wrong branch. You need to target a PR to master.

This add the missing support to set the `server_hostname` setting when creating TCP connection, when the underlying connection is authenticated using TLS. See the documentation for the 2 stdlib functions: * https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_connection * https://docs.python.org/3/library/asyncio-eventloop.html#opening-network-connections The implemention is similar to what was done in urllib3 in urllib3/urllib3#1397 This would be needed to support features in clients using aiohttp, such as tomplus/kubernetes_asyncio#267 Closes: aio-libs#7114

multani · 2023-08-20T07:23:38Z

Oh, I just realised you're pointing to the wrong branch. You need to target a PR to master.

Arghhh, sorry about that, I did my tests through another library that was using 3.8 and I used that branch for my personal PR before publishing it here 🤦

Sorry about the inconvenience, I just rebased/squashed all my changes on top of the master branch and changed the PR to point on that branch instead 🙇

for more information, see https://pre-commit.ci

codecov · 2023-08-20T11:35:13Z

Codecov Report

Merging #7541 (f3bdd2d) into master (0a9bc32) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #7541   +/-   ##
=======================================
  Coverage   97.34%   97.35%           
=======================================
  Files         106      106           
  Lines       31433    31490   +57     
  Branches     3571     3577    +6     
=======================================
+ Hits        30600    30657   +57     
  Misses        630      630           
  Partials      203      203

Flag	Coverage Δ
CI-GHA	`97.30% <100.00%> (+<0.01%)`	⬆️
OS-Linux	`96.97% <100.00%> (+<0.01%)`	⬆️
OS-Windows	`95.43% <100.00%> (+<0.01%)`	⬆️
OS-macOS	`96.65% <100.00%> (+<0.01%)`	⬆️
Py-3.10.11	`95.35% <100.00%> (+<0.01%)`	⬆️
Py-3.10.12	`96.86% <100.00%> (+<0.01%)`	⬆️
Py-3.11.4	`96.56% <100.00%> (+<0.01%)`	⬆️
Py-3.8.10	`95.32% <100.00%> (+<0.01%)`	⬆️
Py-3.8.17	`96.79% <100.00%> (+<0.01%)`	⬆️
Py-3.9.13	`95.32% <100.00%> (+<0.01%)`	⬆️
Py-3.9.17	`96.82% <100.00%> (+<0.01%)`	⬆️
Py-pypy7.3.11	`96.36% <100.00%> (+<0.01%)`	⬆️
VM-macos	`96.65% <100.00%> (+<0.01%)`	⬆️
VM-ubuntu	`96.97% <100.00%> (+<0.01%)`	⬆️
VM-windows	`95.43% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
aiohttp/client.py	`94.43% <ø> (ø)`
aiohttp/client_reqrep.py	`97.74% <100.00%> (+<0.01%)`	⬆️
aiohttp/connector.py	`94.21% <100.00%> (+<0.01%)`	⬆️
tests/test_connector.py	`97.90% <100.00%> (+0.02%)`	⬆️
tests/test_proxy.py	`100.00% <100.00%> (ø)`

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

patchback · 2023-08-20T20:03:01Z

Backport to 3.9: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply ac29dea on top of patchback/backports/3.9/ac29dea2c6a01d718677bd26fcd09e847785d89f/pr-7541

Backporting merged PR #7541 into master

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/aio-libs/aiohttp.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/3.9/ac29dea2c6a01d718677bd26fcd09e847785d89f/pr-7541 upstream/3.9

Now, cherry-pick PR Support passing a custom server name parameter on HTTPS connection #7541 contents into that branch:
```
$ git cherry-pick -x ac29dea2c6a01d718677bd26fcd09e847785d89f
```
If it'll yell at you with something like fatal: Commit ac29dea2c6a01d718677bd26fcd09e847785d89f is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x ac29dea2c6a01d718677bd26fcd09e847785d89f
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Support passing a custom server name parameter on HTTPS connection #7541 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/3.9/ac29dea2c6a01d718677bd26fcd09e847785d89f/pr-7541

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

multani · 2023-08-20T20:28:56Z

@Dreamsorcerer thanks a lot for the merge! 🎉

I can take care of the failing backport and create the related pull request if you want 👍

Dreamsorcerer · 2023-08-20T20:30:05Z

Thanks, that would be great. Just follow the instructions and create a PR to 3.9 branch.

…io-libs#7541) This adds the missing support to set the `server_hostname` setting when creating TCP connection, when the underlying connection is authenticated using TLS. See the documentation for the 2 stdlib functions: * https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_connection * https://docs.python.org/3/library/asyncio-eventloop.html#opening-network-connections This would be needed to support features in clients using aiohttp, such as tomplus/kubernetes_asyncio#267 The default behavior should not change, but this would allow on a per-connection basis to specify a custom server name to check the certificate name against. Closes: aio-libs#7114 (for reference, similar implementation in urllib3: urllib3/urllib3#1397) - [x] I think the code is well written - [x] Unit tests for the changes exist - [x] Documentation reflects the changes - [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES` folder * name it `<issue_id>.<type>` for example (588.bugfix) * if you don't have an `issue_id` change it to the pr id after creating the pr * ensure type is one of the following: * `.feature`: Signifying a new feature. * `.bugfix`: Signifying a bug fix. * `.doc`: Signifying a documentation improvement. * `.removal`: Signifying a deprecation or removal of public API. * `.misc`: A ticket has been closed, but it is not of interest to users. * Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files." --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sam Bull <[email protected]> (cherry picked from commit ac29dea)

multani · 2023-08-20T21:05:13Z

The backport is in #7543!

…7543) This adds the missing support to set the `server_hostname` setting when creating TCP connection, when the underlying connection is authenticated using TLS. See the documentation for the 2 stdlib functions: * https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.create_connection * https://docs.python.org/3/library/asyncio-eventloop.html#opening-network-connections This would be needed to support features in clients using aiohttp, such as tomplus/kubernetes_asyncio#267 The default behavior should not change, but this would allow on a per-connection basis to specify a custom server name to check the certificate name against. Closes: #7114 Backport of #7541 to 3.9 - [x] I think the code is well written - [x] Unit tests for the changes exist - [x] Documentation reflects the changes - [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES` folder * name it `<issue_id>.<type>` for example (588.bugfix) * if you don't have an `issue_id` change it to the pr id after creating the pr * ensure type is one of the following: * `.feature`: Signifying a new feature. * `.bugfix`: Signifying a bug fix. * `.doc`: Signifying a documentation improvement. * `.removal`: Signifying a deprecation or removal of public API. * `.misc`: A ticket has been closed, but it is not of interest to users. * Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files." --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sam Bull <[email protected]> (cherry picked from commit ac29dea)

multani requested a review from asvetlov as a code owner August 19, 2023 17:14

multani marked this pull request as draft August 19, 2023 17:14

multani force-pushed the override-server_hostname branch from a0b52a3 to a6bd740 Compare August 19, 2023 17:17

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Aug 19, 2023

multani marked this pull request as ready for review August 19, 2023 17:23

multani requested a review from webknjaz as a code owner August 19, 2023 17:23

This was referenced Aug 19, 2023

Client doesn't honor tls-server-name setting in kubeconfig tomplus/kubernetes_asyncio#267

Closed

Support loading a custom TLS server name from kubeconfig tomplus/kubernetes_asyncio#270

Merged