Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dns lookup error #2305

Merged
merged 11 commits into from
Jun 28, 2021
Merged

Conversation

euri10
Copy link
Contributor

@euri10 euri10 commented Jun 25, 2021

continuation of #1008 and #1827
fixes #1003

WIP, looking mostly for some CI love at the moment

@codecov
Copy link

codecov bot commented Jun 25, 2021

Codecov Report

Merging #2305 (d004d1c) into main (6622092) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff            @@
##              main     #2305   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           25        25           
  Lines         2439      2446    +7     
=========================================
+ Hits          2439      2446    +7     
Impacted Files Coverage Δ
src/urllib3/connection.py 100.00% <100.00%> (ø)
src/urllib3/connectionpool.py 100.00% <100.00%> (ø)
src/urllib3/exceptions.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6622092...d004d1c. Read the comment docs.

@pquentin
Copy link
Member

Thank you for tackling this! Please let us know when you'd like a review

@euri10
Copy link
Contributor Author

euri10 commented Jun 25, 2021

it seems raising socket.gaierror does not yield the same message on various platforms, so I had to remove the specific match in with pytest.raises(socket.gaierror), match=......): in the test.test_util.TestUtil.test_dnsresolver_error because the CI failed on this test

windows: [Errno 11001] getaddrinfo failed in windows
linux: [Errno -2] Name or service not known
macos: [Errno 8] nodename nor servname provided, or not known

The docs say about the error that:

The numeric error value will match one of the EAI_* constants defined in this module

so given the source here I'm not sure how to interpret the 11001 win error, I'm not familiar enough with how windows errors are "translated", would like some pointers here

#define EAI_ADDRFAMILY   1      /* address family for hostname not supported */
#define EAI_AGAIN        2      /* temporary failure in name resolution */
#define EAI_BADFLAGS     3      /* invalid value for ai_flags */
#define EAI_FAIL         4      /* non-recoverable failure in name resolution */
#define EAI_FAMILY       5      /* ai_family not supported */
#define EAI_MEMORY       6      /* memory allocation failure */
#define EAI_NODATA       7      /* no address associated with hostname */
#define EAI_NONAME       8      /* hostname nor servname provided, or not known */
#define EAI_SERVICE      9      /* servname not supported for ai_socktype */
#define EAI_SOCKTYPE    10      /* ai_socktype not supported */
#define EAI_SYSTEM      11      /* system error returned in errno */
#define EAI_BADHINTS    12
#define EAI_PROTOCOL    13
#define EAI_MAX         14

@pquentin
Copy link
Member

I'm pretty sure EAI_* is Unix or Linux-only, Windows uses completely different errors, as you've seen. See https://docs.microsoft.com/en-us/windows/win32/winsock/windows-sockets-error-codes-2 which tells us that 11001 is WSAHOST_NOT_FOUND

@euri10
Copy link
Contributor Author

euri10 commented Jun 25, 2021

I think it's OK for a review

I would have liked to add a more fine-grained test (with a match in the test.test_util.TestUtil.test_dnsresolver_error) which checks for the gaierror reason, but it's too OS specific, and I'm sure it adds lots of value.

I also tried to take into account this comment from a previous attempt, but I found that if you do that, you'll get the NameResolutionError switched to a ProtocolError in here because gaierror is a sublass of OSError (not sure I get this correctly though, I begin so I have the exception layout not quite well in mind !)

@euri10 euri10 marked this pull request as ready for review June 25, 2021 14:34
pquentin
pquentin previously approved these changes Jun 28, 2021
Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me! But I would like to know what bothered @sigmavirus24 in the previous PR

try:
res = socket.getaddrinfo(host, port, family, socket.SOCK_STREAM)
except socket.gaierror as e:
raise e
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sigmavirus24 Do you remember what you concern was in #1827 (review)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm confused as to why we're re-raising an exception with no other handling. Is it to shorten the traceback stack? What is the purpose of catching and then raising? (I'm on mobile right now so correct me if I'm missing something)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, this change itself seems useless. We could reraise as a NameResolutionError here instead or drop this change

Copy link
Member

@pquentin pquentin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@pquentin pquentin merged commit 1831327 into urllib3:main Jun 28, 2021
@sethmlarson
Copy link
Member

I think we also need to add NameResolutionError to Retry._is_connection_error()?

@pquentin
Copy link
Member

@euri10 Can you please look into this? We indeed want to retry name resolution errors

@euri10
Copy link
Contributor Author

euri10 commented Jun 29, 2021

Sure, I'll try come up with a test for this and resolution later today !

@pquentin pquentin mentioned this pull request Aug 3, 2021
27 tasks
netbsd-srcmastr referenced this pull request in NetBSD/pkgsrc May 5, 2023
2.0.0 (2023-04-26)
==================

Read the `v2.0 migration guide <https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html>`__ for help upgrading to the latest version of urllib3.

Removed
-------

* Removed support for Python 2.7, 3.5, and 3.6 (`#883 <https://github.com/urllib3/urllib3/issues/883>`__, `#2336 <https://github.com/urllib3/urllib3/issues/2336>`__).
* Removed fallback on certificate ``commonName`` in ``match_hostname()`` function.
  This behavior was deprecated in May 2000 in RFC 2818. Instead only ``subjectAltName``
  is used to verify the hostname by default. To enable verifying the hostname against
  ``commonName`` use ``SSLContext.hostname_checks_common_name = True`` (`#2113 <https://github.com/urllib3/urllib3/issues/2113>`__).
* Removed support for Python with an ``ssl`` module compiled with LibreSSL, CiscoSSL,
  wolfSSL, and all other OpenSSL alternatives. Python is moving to require OpenSSL with PEP 644 (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__).
* Removed support for OpenSSL versions earlier than 1.1.1 or that don't have SNI support.
  When an incompatible OpenSSL version is detected an ``ImportError`` is raised (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__).
* Removed the list of default ciphers for OpenSSL 1.1.1+ and SecureTransport as their own defaults are already secure (`#2082 <https://github.com/urllib3/urllib3/issues/2082>`__).
* Removed ``urllib3.contrib.appengine.AppEngineManager`` and support for Google App Engine Standard Environment (`#2044 <https://github.com/urllib3/urllib3/issues/2044>`__).
* Removed deprecated ``Retry`` options ``method_whitelist``, ``DEFAULT_REDIRECT_HEADERS_BLACKLIST`` (`#2086 <https://github.com/urllib3/urllib3/issues/2086>`__).
* Removed ``urllib3.HTTPResponse.from_httplib`` (`#2648 <https://github.com/urllib3/urllib3/issues/2648>`__).
* Removed default value of ``None`` for the ``request_context`` parameter of ``urllib3.PoolManager.connection_from_pool_key``. This change should have no effect on users as the default value of ``None`` was an invalid option and was never used (`#1897 <https://github.com/urllib3/urllib3/issues/1897>`__).
* Removed the ``urllib3.request`` module. ``urllib3.request.RequestMethods`` has been made a private API.
  This change was made to ensure that ``from urllib3 import request`` imported the top-level ``request()``
  function instead of the ``urllib3.request`` module (`#2269 <https://github.com/urllib3/urllib3/issues/2269>`__).
* Removed support for SSLv3.0 from the ``urllib3.contrib.pyopenssl`` even when support is available from the compiled OpenSSL library (`#2233 <https://github.com/urllib3/urllib3/issues/2233>`__).
* Removed the deprecated ``urllib3.contrib.ntlmpool`` module (`#2339 <https://github.com/urllib3/urllib3/issues/2339>`__).
* Removed ``DEFAULT_CIPHERS``, ``HAS_SNI``, ``USE_DEFAULT_SSLCONTEXT_CIPHERS``, from the private module ``urllib3.util.ssl_`` (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__).
* Removed ``urllib3.exceptions.SNIMissingWarning`` (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__).
* Removed the ``_prepare_conn`` method from ``HTTPConnectionPool``. Previously this was only used to call ``HTTPSConnection.set_cert()`` by ``HTTPSConnectionPool`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Removed ``tls_in_tls_required`` property from ``HTTPSConnection``. This is now determined from the ``scheme`` parameter in ``HTTPConnection.set_tunnel()`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).

Deprecated
----------

* Deprecated ``HTTPResponse.getheaders()`` and ``HTTPResponse.getheader()`` which will be removed in urllib3 v2.1.0. Instead use ``HTTPResponse.headers`` and ``HTTPResponse.headers.get(name, default)``. (`#1543 <https://github.com/urllib3/urllib3/issues/1543>`__, `#2814 <https://github.com/urllib3/urllib3/issues/2814>`__).
* Deprecated ``urllib3.contrib.pyopenssl`` module which will be removed in urllib3 v2.1.0 (`#2691 <https://github.com/urllib3/urllib3/issues/2691>`__).
* Deprecated ``urllib3.contrib.securetransport`` module which will be removed in urllib3 v2.1.0 (`#2692 <https://github.com/urllib3/urllib3/issues/2692>`__).
* Deprecated ``ssl_version`` option in favor of ``ssl_minimum_version``. ``ssl_version`` will be removed in urllib3 v2.1.0 (`#2110 <https://github.com/urllib3/urllib3/issues/2110>`__).
* Deprecated the ``strict`` parameter as it's not longer needed in Python 3.x. It will be removed in urllib3 v2.1.0 (`#2267 <https://github.com/urllib3/urllib3/issues/2267>`__)
* Deprecated the ``NewConnectionError.pool`` attribute which will be removed in urllib3 v2.1.0 (`#2271 <https://github.com/urllib3/urllib3/issues/2271>`__).
* Deprecated ``format_header_param_html5`` and ``format_header_param`` in favor of ``format_multipart_header_param`` (`#2257 <https://github.com/urllib3/urllib3/issues/2257>`__).
* Deprecated ``RequestField.header_formatter`` parameter which will be removed in urllib3 v2.1.0 (`#2257 <https://github.com/urllib3/urllib3/issues/2257>`__).
* Deprecated ``HTTPSConnection.set_cert()`` method. Instead pass parameters to the ``HTTPSConnection`` constructor (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Deprecated ``HTTPConnection.request_chunked()`` method which will be removed in urllib3 v2.1.0. Instead pass ``chunked=True`` to ``HTTPConnection.request()`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).

Added
-----

* Added top-level ``urllib3.request`` function which uses a preconfigured module-global ``PoolManager`` instance (`#2150 <https://github.com/urllib3/urllib3/issues/2150>`__).
* Added the ``json`` parameter to ``urllib3.request()``, ``PoolManager.request()``, and ``ConnectionPool.request()`` methods to send JSON bodies in requests. Using this parameter will set the header ``Content-Type: application/json`` if ``Content-Type`` isn't already defined.
  Added support for parsing JSON response bodies with ``HTTPResponse.json()`` method (`#2243 <https://github.com/urllib3/urllib3/issues/2243>`__).
* Added type hints to the ``urllib3`` module (`#1897 <https://github.com/urllib3/urllib3/issues/1897>`__).
* Added ``ssl_minimum_version`` and ``ssl_maximum_version`` options which set
  ``SSLContext.minimum_version`` and ``SSLContext.maximum_version`` (`#2110 <https://github.com/urllib3/urllib3/issues/2110>`__).
* Added support for Zstandard (RFC 8878) when ``zstandard`` 1.18.0 or later is installed.
  Added the ``zstd`` extra which installs the ``zstandard`` package (`#1992 <https://github.com/urllib3/urllib3/issues/1992>`__).
* Added ``urllib3.response.BaseHTTPResponse`` class. All future response classes will be subclasses of ``BaseHTTPResponse`` (`#2083 <https://github.com/urllib3/urllib3/issues/2083>`__).
* Added ``FullPoolError`` which is raised when ``PoolManager(block=True)`` and a connection is returned to a full pool (`#2197 <https://github.com/urllib3/urllib3/issues/2197>`__).
* Added ``HTTPHeaderDict`` to the top-level ``urllib3`` namespace (`#2216 <https://github.com/urllib3/urllib3/issues/2216>`__).
* Added support for configuring header merging behavior with HTTPHeaderDict
  When using a ``HTTPHeaderDict`` to provide headers for a request, by default duplicate
  header values will be repeated. But if ``combine=True`` is passed into a call to
  ``HTTPHeaderDict.add``, then the added header value will be merged in with an existing
  value into a comma-separated list (``X-My-Header: foo, bar``) (`#2242 <https://github.com/urllib3/urllib3/issues/2242>`__).
* Added ``NameResolutionError`` exception when a DNS error occurs (`#2305 <https://github.com/urllib3/urllib3/issues/2305>`__).
* Added ``proxy_assert_hostname`` and ``proxy_assert_fingerprint`` kwargs to ``ProxyManager`` (`#2409 <https://github.com/urllib3/urllib3/issues/2409>`__).
* Added a configurable ``backoff_max`` parameter to the ``Retry`` class.
  If a custom ``backoff_max`` is provided to the ``Retry`` class, it
  will replace the ``Retry.DEFAULT_BACKOFF_MAX`` (`#2494 <https://github.com/urllib3/urllib3/issues/2494>`__).
* Added the ``authority`` property to the Url class as per RFC 3986 3.2. This property should be used in place of ``netloc`` for users who want to include the userinfo (auth) component of the URI (`#2520 <https://github.com/urllib3/urllib3/issues/2520>`__).
* Added the ``scheme`` parameter to ``HTTPConnection.set_tunnel`` to configure the scheme of the origin being tunnelled to (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Added the ``is_closed``, ``is_connected`` and ``has_connected_to_proxy`` properties to ``HTTPConnection`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Added optional ``backoff_jitter`` parameter to ``Retry``. (`#2952 <https://github.com/urllib3/urllib3/issues/2952>`__)

Changed
-------

* Changed ``urllib3.response.HTTPResponse.read`` to respect the semantics of ``io.BufferedIOBase`` regardless of compression. Specifically, this method:

  * Only returns an empty bytes object to indicate EOF (that is, the response has been fully consumed).
  * Never returns more bytes than requested.
  * Can issue any number of system calls: zero, one or multiple.

  If you want each ``urllib3.response.HTTPResponse.read`` call to issue a single system call, you need to disable decompression by setting ``decode_content=False`` (`#2128 <https://github.com/urllib3/urllib3/issues/2128>`__).
* Changed ``urllib3.HTTPConnection.getresponse`` to return an instance of ``urllib3.HTTPResponse`` instead of ``http.client.HTTPResponse`` (`#2648 <https://github.com/urllib3/urllib3/issues/2648>`__).
* Changed ``ssl_version`` to instead set the corresponding ``SSLContext.minimum_version``
  and ``SSLContext.maximum_version`` values.  Regardless of ``ssl_version`` passed
  ``SSLContext`` objects are now constructed using ``ssl.PROTOCOL_TLS_CLIENT`` (`#2110 <https://github.com/urllib3/urllib3/issues/2110>`__).
* Changed default ``SSLContext.minimum_version`` to be ``TLSVersion.TLSv1_2`` in line with Python 3.10 (`#2373 <https://github.com/urllib3/urllib3/issues/2373>`__).
* Changed ``ProxyError`` to wrap any connection error (timeout, TLS, DNS) that occurs when connecting to the proxy (`#2482 <https://github.com/urllib3/urllib3/pull/2482>`__).
* Changed ``urllib3.util.create_urllib3_context`` to not override the system cipher suites with
  a default value. The new default will be cipher suites configured by the operating system (`#2168 <https://github.com/urllib3/urllib3/issues/2168>`__).
* Changed ``multipart/form-data`` header parameter formatting matches the WHATWG HTML Standard as of 2021-06-10. Control characters in filenames are no longer percent encoded (`#2257 <https://github.com/urllib3/urllib3/issues/2257>`__).
* Changed the error raised when connecting via HTTPS when the ``ssl`` module isn't available from ``SSLError`` to ``ImportError`` (`#2589 <https://github.com/urllib3/urllib3/issues/2589>`__).
* Changed ``HTTPConnection.request()`` to always use lowercase chunk boundaries when sending requests with ``Transfer-Encoding: chunked`` (`#2515 <https://github.com/urllib3/urllib3/issues/2515>`__).
* Changed ``enforce_content_length`` default to True, preventing silent data loss when reading streamed responses (`#2514 <https://github.com/urllib3/urllib3/issues/2514>`__).
* Changed internal implementation of ``HTTPHeaderDict`` to use ``dict`` instead of ``collections.OrderedDict`` for better performance (`#2080 <https://github.com/urllib3/urllib3/issues/2080>`__).
* Changed the ``urllib3.contrib.pyopenssl`` module to wrap ``OpenSSL.SSL.Error`` with ``ssl.SSLError`` in ``PyOpenSSLContext.load_cert_chain`` (`#2628 <https://github.com/urllib3/urllib3/issues/2628>`__).
* Changed usage of the deprecated ``socket.error`` to ``OSError`` (`#2120 <https://github.com/urllib3/urllib3/issues/2120>`__).
* Changed all parameters in the ``HTTPConnection`` and ``HTTPSConnection`` constructors to be keyword-only except ``host`` and ``port`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Changed ``HTTPConnection.getresponse()`` to set the socket timeout from ``HTTPConnection.timeout`` value before reading
  data from the socket. This previously was done manually by the ``HTTPConnectionPool`` calling ``HTTPConnection.sock.settimeout(...)`` (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Changed the ``_proxy_host`` property to ``_tunnel_host`` in ``HTTPConnectionPool`` to more closely match how the property is used (value in ``HTTPConnection.set_tunnel()``) (`#1985 <https://github.com/urllib3/urllib3/issues/1985>`__).
* Changed name of ``Retry.BACK0FF_MAX`` to be ``Retry.DEFAULT_BACKOFF_MAX``.
* Changed TLS handshakes to use ``SSLContext.check_hostname`` when possible (`#2452 <https://github.com/urllib3/urllib3/pull/2452>`__).
* Changed ``server_hostname`` to behave like other parameters only used by ``HTTPSConnectionPool`` (`#2537 <https://github.com/urllib3/urllib3/pull/2537>`__).
* Changed the default ``blocksize`` to 16KB to match OpenSSL's default read amounts (`#2348 <https://github.com/urllib3/urllib3/pull/2348>`__).
* Changed ``HTTPResponse.read()`` to raise an error when calling with ``decode_content=False`` after using ``decode_content=True`` to prevent data loss (`#2800 <https://github.com/urllib3/urllib3/issues/2800>`__).

Fixed
-----

* Fixed thread-safety issue where accessing a ``PoolManager`` with many distinct origins would cause connection pools to be closed while requests are in progress (`#1252 <https://github.com/urllib3/urllib3/issues/1252>`__).
* Fixed an issue where an ``HTTPConnection`` instance would erroneously reuse the socket read timeout value from reading the previous response instead of a newly configured connect timeout.
  Instead now if ``HTTPConnection.timeout`` is updated before sending the next request the new timeout value will be used (`#2645 <https://github.com/urllib3/urllib3/issues/2645>`__).
* Fixed ``socket.error.errno`` when raised from pyOpenSSL's ``OpenSSL.SSL.SysCallError`` (`#2118 <https://github.com/urllib3/urllib3/issues/2118>`__).
* Fixed the default value of ``HTTPSConnection.socket_options`` to match ``HTTPConnection`` (`#2213 <https://github.com/urllib3/urllib3/issues/2213>`__).
* Fixed a bug where ``headers`` would be modified by the ``remove_headers_on_redirect`` feature (`#2272 <https://github.com/urllib3/urllib3/issues/2272>`__).
* Fixed a reference cycle bug in ``urllib3.util.connection.create_connection()`` (`#2277 <https://github.com/urllib3/urllib3/issues/2277>`__).
* Fixed a socket leak if ``HTTPConnection.connect()`` fails (`#2571 <https://github.com/urllib3/urllib3/pull/2571>`__).
* Fixed ``urllib3.contrib.pyopenssl.WrappedSocket`` and ``urllib3.contrib.securetransport.WrappedSocket`` close methods (`#2970 <https://github.com/urllib3/urllib3/issues/2970>`__)
github-merge-queue bot referenced this pull request in owntracks/android Aug 29, 2023
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [urllib3](https://togithub.com/urllib3/urllib3)
([changelog](https://togithub.com/urllib3/urllib3/blob/main/CHANGES.rst))
| `==1.26.16` -> `==2.0.4` |
[![age](https://developer.mend.io/api/mc/badges/age/pypi/urllib3/2.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/urllib3/2.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/urllib3/1.26.16/2.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|
[![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/urllib3/1.26.16/2.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/)
|

---

### ⚠ Dependency Lookup Warnings ⚠

Warnings were logged while processing this repo. Please check the logs
for more information.

---

### Release Notes

<details>
<summary>urllib3/urllib3 (urllib3)</summary>

###
[`v2.0.4`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#204-2023-07-19)

[Compare
Source](https://togithub.com/urllib3/urllib3/compare/2.0.3...2.0.4)

\==================

- Added support for union operators to `HTTPHeaderDict` (`#&#8203;2254
<https://github.com/urllib3/urllib3/issues/2254>`\__)
- Added `BaseHTTPResponse` to `urllib3.__all__` (`#&#8203;3078
<https://github.com/urllib3/urllib3/issues/3078>`\__)
- Fixed `urllib3.connection.HTTPConnection` to raise the
`http.client.connect` audit event to have the same behavior as the
standard library HTTP client (`#&#8203;2757
<https://github.com/urllib3/urllib3/issues/2757>`\__)
- Relied on the standard library for checking hostnames in supported
PyPy releases (`#&#8203;3087
<https://github.com/urllib3/urllib3/issues/3087>`\__)

###
[`v2.0.3`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#203-2023-06-07)

[Compare
Source](https://togithub.com/urllib3/urllib3/compare/2.0.2...2.0.3)

\==================

- Allowed alternative SSL libraries such as LibreSSL, while still
issuing a warning as we cannot help users facing issues with
implementations other than OpenSSL. (`#&#8203;3020
<https://github.com/urllib3/urllib3/issues/3020>`\__)
- Deprecated URLs which don't have an explicit scheme (`#&#8203;2950
<https://github.com/urllib3/urllib3/pull/2950>`\_)
- Fixed response decoding with Zstandard when compressed data is made of
several frames. (`#&#8203;3008
<https://github.com/urllib3/urllib3/issues/3008>`\__)
- Fixed `assert_hostname=False` to correctly skip hostname check.
(`#&#8203;3051 <https://github.com/urllib3/urllib3/issues/3051>`\__)

###
[`v2.0.2`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#202-2023-05-03)

[Compare
Source](https://togithub.com/urllib3/urllib3/compare/2.0.1...2.0.2)

\==================

- Fixed `HTTPResponse.stream()` to continue yielding bytes if buffered
decompressed data
was still available to be read even if the underlying socket is closed.
This prevents
a compressed response from being truncated. (`#&#8203;3009
<https://github.com/urllib3/urllib3/issues/3009>`\__)

###
[`v2.0.1`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#201-2023-04-30)

[Compare
Source](https://togithub.com/urllib3/urllib3/compare/2.0.0...2.0.1)

\==================

- Fixed a socket leak when fingerprint or hostname verifications fail.
(`#&#8203;2991 <https://github.com/urllib3/urllib3/issues/2991>`\__)
- Fixed an error when `HTTPResponse.read(0)` was the first `read` call
or when the internal response body buffer was otherwise empty.
(`#&#8203;2998 <https://github.com/urllib3/urllib3/issues/2998>`\__)

###
[`v2.0.0`](https://togithub.com/urllib3/urllib3/blob/HEAD/CHANGES.rst#200-2023-04-26)

[Compare
Source](https://togithub.com/urllib3/urllib3/compare/1.26.16...2.0.0)

\==================

Read the `v2.0 migration guide
<https://urllib3.readthedocs.io/en/latest/v2-migration-guide.html>`\__
for help upgrading to the latest version of urllib3.

## Removed

- Removed support for Python 2.7, 3.5, and 3.6 (`#&#8203;883
<https://github.com/urllib3/urllib3/issues/883>`**, `#&#8203;2336
<https://github.com/urllib3/urllib3/issues/2336>`**).
- Removed fallback on certificate `commonName` in `match_hostname()`
function.
This behavior was deprecated in May 2000 in RFC 2818. Instead only
`subjectAltName`
is used to verify the hostname by default. To enable verifying the
hostname against
`commonName` use `SSLContext.hostname_checks_common_name = True`
(`#&#8203;2113 <https://github.com/urllib3/urllib3/issues/2113>`\__).
- Removed support for Python with an `ssl` module compiled with
LibreSSL, CiscoSSL,
wolfSSL, and all other OpenSSL alternatives. Python is moving to require
OpenSSL with PEP 644 (`#&#8203;2168
<https://github.com/urllib3/urllib3/issues/2168>`\__).
- Removed support for OpenSSL versions earlier than 1.1.1 or that don't
have SNI support.
When an incompatible OpenSSL version is detected an `ImportError` is
raised (`#&#8203;2168
<https://github.com/urllib3/urllib3/issues/2168>`\__).
- Removed the list of default ciphers for OpenSSL 1.1.1+ and
SecureTransport as their own defaults are already secure (`#&#8203;2082
<https://github.com/urllib3/urllib3/issues/2082>`\__).
- Removed `urllib3.contrib.appengine.AppEngineManager` and support for
Google App Engine Standard Environment (`#&#8203;2044
<https://github.com/urllib3/urllib3/issues/2044>`\__).
- Removed deprecated `Retry` options `method_whitelist`,
`DEFAULT_REDIRECT_HEADERS_BLACKLIST` (`#&#8203;2086
<https://github.com/urllib3/urllib3/issues/2086>`\__).
- Removed `urllib3.HTTPResponse.from_httplib` (`#&#8203;2648
<https://github.com/urllib3/urllib3/issues/2648>`\__).
- Removed default value of `None` for the `request_context` parameter of
`urllib3.PoolManager.connection_from_pool_key`. This change should have
no effect on users as the default value of `None` was an invalid option
and was never used (`#&#8203;1897
<https://github.com/urllib3/urllib3/issues/1897>`\__).
- Removed the `urllib3.request` module. `urllib3.request.RequestMethods`
has been made a private API.
This change was made to ensure that `from urllib3 import request`
imported the top-level `request()`
function instead of the `urllib3.request` module (`#&#8203;2269
<https://github.com/urllib3/urllib3/issues/2269>`\__).
- Removed support for SSLv3.0 from the `urllib3.contrib.pyopenssl` even
when support is available from the compiled OpenSSL library
(`#&#8203;2233 <https://github.com/urllib3/urllib3/issues/2233>`\__).
- Removed the deprecated `urllib3.contrib.ntlmpool` module
(`#&#8203;2339 <https://github.com/urllib3/urllib3/issues/2339>`\__).
- Removed `DEFAULT_CIPHERS`, `HAS_SNI`,
`USE_DEFAULT_SSLCONTEXT_CIPHERS`, from the private module
`urllib3.util.ssl_` (`#&#8203;2168
<https://github.com/urllib3/urllib3/issues/2168>`\__).
- Removed `urllib3.exceptions.SNIMissingWarning` (`#&#8203;2168
<https://github.com/urllib3/urllib3/issues/2168>`\__).
- Removed the `_prepare_conn` method from `HTTPConnectionPool`.
Previously this was only used to call `HTTPSConnection.set_cert()` by
`HTTPSConnectionPool` (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Removed `tls_in_tls_required` property from `HTTPSConnection`. This is
now determined from the `scheme` parameter in
`HTTPConnection.set_tunnel()` (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Removed the `strict` parameter/attribute from `HTTPConnection`,
`HTTPSConnection`, `HTTPConnectionPool`, `HTTPSConnectionPool`, and
`HTTPResponse` (`#&#8203;2064
<https://github.com/urllib3/urllib3/issues/2064>`\__).

## Deprecated

- Deprecated `HTTPResponse.getheaders()` and `HTTPResponse.getheader()`
which will be removed in urllib3 v2.1.0. Instead use
`HTTPResponse.headers` and `HTTPResponse.headers.get(name, default)`.
(`#&#8203;1543 <https://github.com/urllib3/urllib3/issues/1543>`**,
`#&#8203;2814 <https://github.com/urllib3/urllib3/issues/2814>`**).
- Deprecated `urllib3.contrib.pyopenssl` module which will be removed in
urllib3 v2.1.0 (`#&#8203;2691
<https://github.com/urllib3/urllib3/issues/2691>`\__).
- Deprecated `urllib3.contrib.securetransport` module which will be
removed in urllib3 v2.1.0 (`#&#8203;2692
<https://github.com/urllib3/urllib3/issues/2692>`\__).
- Deprecated `ssl_version` option in favor of `ssl_minimum_version`.
`ssl_version` will be removed in urllib3 v2.1.0 (`#&#8203;2110
<https://github.com/urllib3/urllib3/issues/2110>`\__).
- Deprecated the `strict` parameter of
`PoolManager.connection_from_context()` as it's not longer needed in
Python 3.x. It will be removed in urllib3 v2.1.0 (`#&#8203;2267
<https://github.com/urllib3/urllib3/issues/2267>`\__)
- Deprecated the `NewConnectionError.pool` attribute which will be
removed in urllib3 v2.1.0 (`#&#8203;2271
<https://github.com/urllib3/urllib3/issues/2271>`\__).
- Deprecated `format_header_param_html5` and `format_header_param` in
favor of `format_multipart_header_param` (`#&#8203;2257
<https://github.com/urllib3/urllib3/issues/2257>`\__).
- Deprecated `RequestField.header_formatter` parameter which will be
removed in urllib3 v2.1.0 (`#&#8203;2257
<https://github.com/urllib3/urllib3/issues/2257>`\__).
- Deprecated `HTTPSConnection.set_cert()` method. Instead pass
parameters to the `HTTPSConnection` constructor (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Deprecated `HTTPConnection.request_chunked()` method which will be
removed in urllib3 v2.1.0. Instead pass `chunked=True` to
`HTTPConnection.request()` (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).

## Added

- Added top-level `urllib3.request` function which uses a preconfigured
module-global `PoolManager` instance (`#&#8203;2150
<https://github.com/urllib3/urllib3/issues/2150>`\__).
- Added the `json` parameter to `urllib3.request()`,
`PoolManager.request()`, and `ConnectionPool.request()` methods to send
JSON bodies in requests. Using this parameter will set the header
`Content-Type: application/json` if `Content-Type` isn't already
defined.
Added support for parsing JSON response bodies with
`HTTPResponse.json()` method (`#&#8203;2243
<https://github.com/urllib3/urllib3/issues/2243>`\__).
- Added type hints to the `urllib3` module (`#&#8203;1897
<https://github.com/urllib3/urllib3/issues/1897>`\__).
- Added `ssl_minimum_version` and `ssl_maximum_version` options which
set
`SSLContext.minimum_version` and `SSLContext.maximum_version`
(`#&#8203;2110 <https://github.com/urllib3/urllib3/issues/2110>`\__).
- Added support for Zstandard (RFC 8878) when `zstandard` 1.18.0 or
later is installed.
Added the `zstd` extra which installs the `zstandard` package
(`#&#8203;1992 <https://github.com/urllib3/urllib3/issues/1992>`\__).
- Added `urllib3.response.BaseHTTPResponse` class. All future response
classes will be subclasses of `BaseHTTPResponse` (`#&#8203;2083
<https://github.com/urllib3/urllib3/issues/2083>`\__).
- Added `FullPoolError` which is raised when `PoolManager(block=True)`
and a connection is returned to a full pool (`#&#8203;2197
<https://github.com/urllib3/urllib3/issues/2197>`\__).
- Added `HTTPHeaderDict` to the top-level `urllib3` namespace
(`#&#8203;2216 <https://github.com/urllib3/urllib3/issues/2216>`\__).
- Added support for configuring header merging behavior with
HTTPHeaderDict
When using a `HTTPHeaderDict` to provide headers for a request, by
default duplicate
header values will be repeated. But if `combine=True` is passed into a
call to
`HTTPHeaderDict.add`, then the added header value will be merged in with
an existing
value into a comma-separated list (`X-My-Header: foo, bar`)
(`#&#8203;2242 <https://github.com/urllib3/urllib3/issues/2242>`\__).
- Added `NameResolutionError` exception when a DNS error occurs
(`#&#8203;2305 <https://github.com/urllib3/urllib3/issues/2305>`\__).
- Added `proxy_assert_hostname` and `proxy_assert_fingerprint` kwargs to
`ProxyManager` (`#&#8203;2409
<https://github.com/urllib3/urllib3/issues/2409>`\__).
-   Added a configurable `backoff_max` parameter to the `Retry` class.
    If a custom `backoff_max` is provided to the `Retry` class, it
will replace the `Retry.DEFAULT_BACKOFF_MAX` (`#&#8203;2494
<https://github.com/urllib3/urllib3/issues/2494>`\__).
- Added the `authority` property to the Url class as per RFC 3986 3.2.
This property should be used in place of `netloc` for users who want to
include the userinfo (auth) component of the URI (`#&#8203;2520
<https://github.com/urllib3/urllib3/issues/2520>`\__).
- Added the `scheme` parameter to `HTTPConnection.set_tunnel` to
configure the scheme of the origin being tunnelled to (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Added the `is_closed`, `is_connected` and `has_connected_to_proxy`
properties to `HTTPConnection` (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Added optional `backoff_jitter` parameter to `Retry`. (`#&#8203;2952
<https://github.com/urllib3/urllib3/issues/2952>`\__)

## Changed

- Changed `urllib3.response.HTTPResponse.read` to respect the semantics
of `io.BufferedIOBase` regardless of compression. Specifically, this
method:

- Only returns an empty bytes object to indicate EOF (that is, the
response has been fully consumed).
    -   Never returns more bytes than requested.
    -   Can issue any number of system calls: zero, one or multiple.

If you want each `urllib3.response.HTTPResponse.read` call to issue a
single system call, you need to disable decompression by setting
`decode_content=False` (`#&#8203;2128
<https://github.com/urllib3/urllib3/issues/2128>`\__).
- Changed `urllib3.HTTPConnection.getresponse` to return an instance of
`urllib3.HTTPResponse` instead of `http.client.HTTPResponse`
(`#&#8203;2648 <https://github.com/urllib3/urllib3/issues/2648>`\__).
- Changed `ssl_version` to instead set the corresponding
`SSLContext.minimum_version`
and `SSLContext.maximum_version` values. Regardless of `ssl_version`
passed
`SSLContext` objects are now constructed using `ssl.PROTOCOL_TLS_CLIENT`
(`#&#8203;2110 <https://github.com/urllib3/urllib3/issues/2110>`\__).
- Changed default `SSLContext.minimum_version` to be
`TLSVersion.TLSv1_2` in line with Python 3.10 (`#&#8203;2373
<https://github.com/urllib3/urllib3/issues/2373>`\__).
- Changed `ProxyError` to wrap any connection error (timeout, TLS, DNS)
that occurs when connecting to the proxy (`#&#8203;2482
<https://github.com/urllib3/urllib3/pull/2482>`\__).
- Changed `urllib3.util.create_urllib3_context` to not override the
system cipher suites with
a default value. The new default will be cipher suites configured by the
operating system (`#&#8203;2168
<https://github.com/urllib3/urllib3/issues/2168>`\__).
- Changed `multipart/form-data` header parameter formatting matches the
WHATWG HTML Standard as of 2021-06-10. Control characters in filenames
are no longer percent encoded (`#&#8203;2257
<https://github.com/urllib3/urllib3/issues/2257>`\__).
- Changed the error raised when connecting via HTTPS when the `ssl`
module isn't available from `SSLError` to `ImportError` (`#&#8203;2589
<https://github.com/urllib3/urllib3/issues/2589>`\__).
- Changed `HTTPConnection.request()` to always use lowercase chunk
boundaries when sending requests with `Transfer-Encoding: chunked`
(`#&#8203;2515 <https://github.com/urllib3/urllib3/issues/2515>`\__).
- Changed `enforce_content_length` default to True, preventing silent
data loss when reading streamed responses (`#&#8203;2514
<https://github.com/urllib3/urllib3/issues/2514>`\__).
- Changed internal implementation of `HTTPHeaderDict` to use `dict`
instead of `collections.OrderedDict` for better performance
(`#&#8203;2080 <https://github.com/urllib3/urllib3/issues/2080>`\__).
- Changed the `urllib3.contrib.pyopenssl` module to wrap
`OpenSSL.SSL.Error` with `ssl.SSLError` in
`PyOpenSSLContext.load_cert_chain` (`#&#8203;2628
<https://github.com/urllib3/urllib3/issues/2628>`\__).
- Changed usage of the deprecated `socket.error` to `OSError`
(`#&#8203;2120 <https://github.com/urllib3/urllib3/issues/2120>`\__).
- Changed all parameters in the `HTTPConnection` and `HTTPSConnection`
constructors to be keyword-only except `host` and `port` (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Changed `HTTPConnection.getresponse()` to set the socket timeout from
`HTTPConnection.timeout` value before reading
data from the socket. This previously was done manually by the
`HTTPConnectionPool` calling `HTTPConnection.sock.settimeout(...)`
(`#&#8203;1985 <https://github.com/urllib3/urllib3/issues/1985>`\__).
- Changed the `_proxy_host` property to `_tunnel_host` in
`HTTPConnectionPool` to more closely match how the property is used
(value in `HTTPConnection.set_tunnel()`) (`#&#8203;1985
<https://github.com/urllib3/urllib3/issues/1985>`\__).
- Changed name of `Retry.BACK0FF_MAX` to be `Retry.DEFAULT_BACKOFF_MAX`.
- Changed TLS handshakes to use `SSLContext.check_hostname` when
possible (`#&#8203;2452
<https://github.com/urllib3/urllib3/pull/2452>`\__).
- Changed `server_hostname` to behave like other parameters only used by
`HTTPSConnectionPool` (`#&#8203;2537
<https://github.com/urllib3/urllib3/pull/2537>`\__).
- Changed the default `blocksize` to 16KB to match OpenSSL's default
read amounts (`#&#8203;2348
<https://github.com/urllib3/urllib3/pull/2348>`\__).
- Changed `HTTPResponse.read()` to raise an error when calling with
`decode_content=False` after using `decode_content=True` to prevent data
loss (`#&#8203;2800
<https://github.com/urllib3/urllib3/issues/2800>`\__).

## Fixed

- Fixed thread-safety issue where accessing a `PoolManager` with many
distinct origins would cause connection pools to be closed while
requests are in progress (`#&#8203;1252
<https://github.com/urllib3/urllib3/issues/1252>`\__).
- Fixed an issue where an `HTTPConnection` instance would erroneously
reuse the socket read timeout value from reading the previous response
instead of a newly configured connect timeout.
Instead now if `HTTPConnection.timeout` is updated before sending the
next request the new timeout value will be used (`#&#8203;2645
<https://github.com/urllib3/urllib3/issues/2645>`\__).
- Fixed `socket.error.errno` when raised from pyOpenSSL's
`OpenSSL.SSL.SysCallError` (`#&#8203;2118
<https://github.com/urllib3/urllib3/issues/2118>`\__).
- Fixed the default value of `HTTPSConnection.socket_options` to match
`HTTPConnection` (`#&#8203;2213
<https://github.com/urllib3/urllib3/issues/2213>`\__).
- Fixed a bug where `headers` would be modified by the
`remove_headers_on_redirect` feature (`#&#8203;2272
<https://github.com/urllib3/urllib3/issues/2272>`\__).
- Fixed a reference cycle bug in
`urllib3.util.connection.create_connection()` (`#&#8203;2277
<https://github.com/urllib3/urllib3/issues/2277>`\__).
- Fixed a socket leak if `HTTPConnection.connect()` fails (`#&#8203;2571
<https://github.com/urllib3/urllib3/pull/2571>`\__).
- Fixed `urllib3.contrib.pyopenssl.WrappedSocket` and
`urllib3.contrib.securetransport.WrappedSocket` close methods
(`#&#8203;2970 <https://github.com/urllib3/urllib3/issues/2970>`\__)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR is behind base branch, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/owntracks/android).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi41Ni4wIiwidXBkYXRlZEluVmVyIjoiMzYuNjQuOCIsInRhcmdldEJyYW5jaCI6Im1hc3RlciJ9-->

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DNS lookup error exception
4 participants