diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py index ee810cf4564..3fa52de8d52 100644 --- a/aiohttp/client_reqrep.py +++ b/aiohttp/client_reqrep.py @@ -27,7 +27,7 @@ import chardet -__all__ = ('ClientRequest', 'ClientResponse') +__all__ = ('ClientRequest', 'ClientResponse', 'RequestInfo') RequestInfo = collections.namedtuple( diff --git a/changes/1811.doc b/changes/1811.doc new file mode 100644 index 00000000000..28251039930 --- /dev/null +++ b/changes/1811.doc @@ -0,0 +1,3 @@ +Add a note about possible performance degradation in `await +resp.text()` if charset was not provided by `Content-Type` HTTP +header. Pass explicit encoding to solve it. diff --git a/docs/client_reference.rst b/docs/client_reference.rst index 37478d67163..bddaf09cdfe 100644 --- a/docs/client_reference.rst +++ b/docs/client_reference.rst @@ -102,15 +102,20 @@ The client session supports the context manager protocol for self closing. One example is not processing cookies at all when working in proxy mode. - If no cookie processing is needed, a :class:`aiohttp.helpers.DummyCookieJar` - instance can be provided. + If no cookie processing is needed, a + :class:`aiohttp.helpers.DummyCookieJar` instance can be + provided. .. versionadded:: 0.22 - :param callable json_serialize: Json `serializer` function. (:func:`json.dumps` by default) + :param callable json_serialize: Json *serializer* callable. - :param bool raise_for_status: Automatically call `raise_for_status()` for each response. - (default is False) + By default :func:`json.dumps` function. + + :param bool raise_for_status: + + Automatically call :meth:`ClientResponse.raise_for_status()` for + each response, ``False`` by default. .. versionadded:: 2.0 @@ -153,10 +158,12 @@ The client session supports the context manager protocol for self closing. .. attribute:: requote_redirect_url aiohttp re quote's redirect urls by default, but some servers - require exact url from location header. to disable `re-quote` system - set `requote_redirect_url` to `False`. + require exact url from location header. to disable *re-quote* system + set :attr:`requote_redirect_url` attribute to ``False``. + + .. versionadded:: 2.1 - .. note:: this parameter affects all subsequent requests. + .. note:: This parameter affects all subsequent requests. .. attribute:: loop @@ -198,8 +205,9 @@ The client session supports the context manager protocol for self closing. :param data: Dictionary, bytes, or file-like object to send in the body of the request (optional) - :param json: Any json compatible python object (optional). `json` and `data` - parameters could not be used at the same time. + :param json: Any json compatible python object + (optional). *json* and *data* parameters could not + be used at the same time. :param dict headers: HTTP Headers to send with the request (optional) @@ -226,7 +234,8 @@ The client session supports the context manager protocol for self closing. with a *Content-Encoding* and *Content-Length* headers. ``None`` by default (optional). - :param int chunked: Enable chunked transfer encoding. It is up to the developer + :param int chunked: Enable chunked transfer encoding. + It is up to the developer to decide how to chunk data streams. If chunking is enabled, aiohttp encodes the provided chunks in the "Transfer-encoding: chunked" format. If *chunked* is set, then the *Transfer-encoding* and *content-length* @@ -244,9 +253,11 @@ The client session supports the context manager protocol for self closing. :param aiohttp.BasicAuth proxy_auth: an object that represents proxy HTTP Basic Authorization (optional) - :param int timeout: override the session's timeout (``read_timeout``) for IO operations. + :param int timeout: override the session's timeout + (``read_timeout``) for IO operations. - :return ClientResponse: a :class:`client response ` object. + :return ClientResponse: a :class:`client response ` + object. .. versionadded:: 1.0 @@ -431,24 +442,27 @@ The client session supports the context manager protocol for self closing. :param tuple protocols: Websocket protocols - :param float timeout: Timeout for websocket to close. 10 seconds by default + :param float timeout: Timeout for websocket to close. ``10`` seconds + by default - :param float receive_timeout: Timeout for websocket to receive complete message. - None(unlimited) seconds by default + :param float receive_timeout: Timeout for websocket to receive + complete message. ``None`` (unlimited) + seconds by default :param aiohttp.BasicAuth auth: an object that represents HTTP Basic Authorization (optional) :param bool autoclose: Automatically close websocket connection on close - message from server. If `autoclose` is False + message from server. If *autoclose* is False them close procedure has to be handled manually - :param bool autoping: automatically send `pong` on `ping` + :param bool autoping: automatically send *pong* on *ping* message from server - :param float heartbeat: Send `ping` message every `heartbeat` seconds - and wait `pong` response, if `pong` response is not received - then close connection. + :param float heartbeat: Send *ping* message every *heartbeat* + seconds and wait *pong* response, if + *pong* response is not received then + close connection. :param str origin: Origin header to send to server @@ -526,7 +540,7 @@ certification chaining. :param data: Dictionary, bytes, or file-like object to send in the body of the request (optional) - :param json: Any json compatible python object (optional). `json` and `data` + :param json: Any json compatible python object (optional). *json* and *data* parameters could not be used at the same time. :param dict headers: HTTP Headers to send with the request (optional) @@ -598,8 +612,8 @@ There are standard connectors: All connector classes should be derived from :class:`BaseConnector`. -By default all *connectors* support *keep-alive connections* (behavior is controlled by -*force_close* constructor's parameter). +By default all *connectors* support *keep-alive connections* (behavior +is controlled by *force_close* constructor's parameter). BaseConnector @@ -973,8 +987,9 @@ Response object Reading from the stream may raise :exc:`aiohttp.ClientPayloadError` if the response object is - closed before response receives all data or in case if any transfer encoding - related errors like mis-formed chunked encoding of broken compression data. + closed before response receives all data or in case if any + transfer encoding related errors like mis-formed chunked + encoding of broken compression data. .. attribute:: cookies @@ -1037,13 +1052,16 @@ Response object .. comethod:: release() - It is not required to call `release` on the response object. When the - client fully receives the payload, the underlying connection automatically - returns back to pool. If the payload is not fully read, the connection is closed + It is not required to call `release` on the response + object. When the client fully receives the payload, the + underlying connection automatically returns back to pool. If the + payload is not fully read, the connection is closed .. method:: raise_for_status() - Raise an :exc:`aiohttp.ClientResponseError` if the response status is 400 or higher. + Raise an :exc:`aiohttp.ClientResponseError` if the response + status is 400 or higher. + Do nothing for success responses (less than 400). .. comethod:: text(encoding=None) @@ -1052,7 +1070,10 @@ Response object specified *encoding* parameter. If *encoding* is ``None`` content encoding is autocalculated - using :term:`cchardet` or :term:`chardet` as fallback if + using ``Content-Type`` HTTP header and *chardet* tool if the + header is not provided by server. + + :term:`cchardet` is used with fallback to :term:`chardet` if *cchardet* is not available. Close underlying connection if data reading gets an error, @@ -1064,7 +1085,19 @@ Response object :return str: decoded *BODY* - .. comethod:: json(encoding=None, loads=json.loads, content_type='application/json') + .. note:: + + If response has no ``charset`` info in ``Content-Type`` HTTP + header :term:`cchardet` / :term:`chardet` is used for content + encoding autodetection. + + It may hurt performance. If page encoding is known passing + explicit *encoding* parameter might help:: + + await resp.text('ISO-8859-1') + + .. comethod:: json(*, encoding=None, loads=json.loads, \ + content_type='application/json') Read response's body as *JSON*, return :class:`dict` using specified *encoding* and *loader*. @@ -1092,10 +1125,10 @@ Response object :return: *BODY* as *JSON* data parsed by *loads* parameter or ``None`` if *BODY* is empty or contains white-spaces only. - .. attribute:: request_info + .. attribute:: request_info A namedtuple with request URL and headers from :class:`ClientRequest` - object. + object, :class:`aiohttp.RequestInfo` instance. ClientWebSocketResponse @@ -1240,6 +1273,27 @@ Utilities --------- +RequestInfo +^^^^^^^^^^^ + +.. class:: RequestInfo() + + A namedtuple with request URL and headers from :class:`ClientRequest` + object, available as :attr:`ClientResponse.request_info` attribute. + + .. attribute:: url + + Requested *url*, :class:`yarl.URL` instance. + + .. attribute:: method + + Request HTTP method like ``'GET'`` or ``'POST'``, :class:`str`. + + .. attribute:: headers + + HTTP headers for request, :class:`multidict.CIMultiDict` instance. + + BasicAuth ^^^^^^^^^ @@ -1249,7 +1303,7 @@ BasicAuth :param str login: login :param str password: password - :param str encoding: encoding (`'latin1'` by default) + :param str encoding: encoding (``'latin1'`` by default) Should be used for specifying authorization data in client API, @@ -1351,24 +1405,28 @@ CookieJar Client exceptions ^^^^^^^^^^^^^^^^^ -Exception hierarchy has been significantly modified in version 2.0. aiohttp defines only -exceptions that covers connection handling and server response misbehaviors. -For developer specific mistakes, aiohttp uses python standard exceptions -like `ValueError` or `TypeError`. +Exception hierarchy has been significantly modified in version +2.0. aiohttp defines only exceptions that covers connection handling +and server response misbehaviors. For developer specific mistakes, +aiohttp uses python standard exceptions like :exc:`ValueError` or +:exc:`TypeError`. -Reading a response content may raise a :exc:`ClientPayloadError` exception. This exception -indicates errors specific to the payload encoding. Such as invalid compressed data, -malformed chunked-encoded chunks or not enough data that satisfy the content-length header. +Reading a response content may raise a :exc:`ClientPayloadError` +exception. This exception indicates errors specific to the payload +encoding. Such as invalid compressed data, malformed chunked-encoded +chunks or not enough data that satisfy the content-length header. -All exceptions are available as attributes in `aiohttp` module. +All exceptions are available as members of *aiohttp* module. Hierarchy of exceptions: -* `aiohttp.ClientError` - Base class for all client specific exceptions +* :exc:`aiohttp.ClientError` - Base class for all client specific exceptions - - `aiohttp.ClientResponseError` - exceptions that could happen after we get response from server. + - :exc:`aiohttp.ClientResponseError` - exceptions that could happen after + we get response from server. - `request_info` - Instance of `RequestInfo` object, contains information about request. + `request_info` - Instance of `RequestInfo` object, contains + information about request. `history` - History from `ClientResponse` object, if available, else empty tuple.