Skip to content

Commit

Permalink
Fix #1811: Add a note about possible performance dagradation in await…
Browse files Browse the repository at this point in the history 
… resp.text()
  • Loading branch information
@asvetlov
asvetlov committed Jun 26, 2017
1 parent 63b6c0e commit 9dc8423
Show file tree
Hide file tree
Showing 3 changed files with 108 additions and 47 deletions.
2 changes: 1 addition & 1 deletion aiohttp/client_reqrep.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
import chardet


__all__ = ('ClientRequest', 'ClientResponse')
__all__ = ('ClientRequest', 'ClientResponse', 'RequestInfo')


RequestInfo = collections.namedtuple(
Expand Down
3 changes: 3 additions & 0 deletions changes/1811.doc
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Add a note about possible performance degradation in `await
resp.text()` if charset was not provided by `Content-Type` HTTP
header. Pass explicit encoding to solve it.
150 changes: 104 additions & 46 deletions docs/client_reference.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,15 +102,20 @@ The client session supports the context manager protocol for self closing.
One example is not processing cookies at all when working in
proxy mode.

If no cookie processing is needed, a :class:`aiohttp.helpers.DummyCookieJar`
instance can be provided.
If no cookie processing is needed, a
:class:`aiohttp.helpers.DummyCookieJar` instance can be
provided.

.. versionadded:: 0.22

:param callable json_serialize: Json `serializer` function. (:func:`json.dumps` by default)
:param callable json_serialize: Json *serializer* callable.

:param bool raise_for_status: Automatically call `raise_for_status()` for each response.
(default is False)
By default :func:`json.dumps` function.

:param bool raise_for_status:

Automatically call :meth:`ClientResponse.raise_for_status()` for
each response, ``False`` by default.

.. versionadded:: 2.0

Expand Down Expand Up @@ -153,10 +158,12 @@ The client session supports the context manager protocol for self closing.
.. attribute:: requote_redirect_url

aiohttp re quote's redirect urls by default, but some servers
require exact url from location header. to disable `re-quote` system
set `requote_redirect_url` to `False`.
require exact url from location header. to disable *re-quote* system
set :attr:`requote_redirect_url` attribute to ``False``.

.. versionadded:: 2.1

.. note:: this parameter affects all subsequent requests.
.. note:: This parameter affects all subsequent requests.

.. attribute:: loop

Expand Down Expand Up @@ -198,8 +205,9 @@ The client session supports the context manager protocol for self closing.
:param data: Dictionary, bytes, or file-like object to
send in the body of the request (optional)

:param json: Any json compatible python object (optional). `json` and `data`
parameters could not be used at the same time.
:param json: Any json compatible python object
(optional). *json* and *data* parameters could not
be used at the same time.

:param dict headers: HTTP Headers to send with
the request (optional)
Expand All @@ -226,7 +234,8 @@ The client session supports the context manager protocol for self closing.
with a *Content-Encoding* and *Content-Length* headers.
``None`` by default (optional).

:param int chunked: Enable chunked transfer encoding. It is up to the developer
:param int chunked: Enable chunked transfer encoding.
It is up to the developer
to decide how to chunk data streams. If chunking is enabled, aiohttp
encodes the provided chunks in the "Transfer-encoding: chunked" format.
If *chunked* is set, then the *Transfer-encoding* and *content-length*
Expand All @@ -244,9 +253,11 @@ The client session supports the context manager protocol for self closing.
:param aiohttp.BasicAuth proxy_auth: an object that represents proxy HTTP
Basic Authorization (optional)

:param int timeout: override the session's timeout (``read_timeout``) for IO operations.
:param int timeout: override the session's timeout
(``read_timeout``) for IO operations.

:return ClientResponse: a :class:`client response <ClientResponse>` object.
:return ClientResponse: a :class:`client response <ClientResponse>`
object.

.. versionadded:: 1.0

Expand Down Expand Up @@ -431,24 +442,27 @@ The client session supports the context manager protocol for self closing.

:param tuple protocols: Websocket protocols

:param float timeout: Timeout for websocket to close. 10 seconds by default
:param float timeout: Timeout for websocket to close. ``10`` seconds
by default

:param float receive_timeout: Timeout for websocket to receive complete message.
None(unlimited) seconds by default
:param float receive_timeout: Timeout for websocket to receive
complete message. ``None`` (unlimited)
seconds by default

:param aiohttp.BasicAuth auth: an object that represents HTTP
Basic Authorization (optional)

:param bool autoclose: Automatically close websocket connection on close
message from server. If `autoclose` is False
message from server. If *autoclose* is False
them close procedure has to be handled manually

:param bool autoping: automatically send `pong` on `ping`
:param bool autoping: automatically send *pong* on *ping*
message from server

:param float heartbeat: Send `ping` message every `heartbeat` seconds
and wait `pong` response, if `pong` response is not received
then close connection.
:param float heartbeat: Send *ping* message every *heartbeat*
seconds and wait *pong* response, if
*pong* response is not received then
close connection.

:param str origin: Origin header to send to server

Expand Down Expand Up @@ -526,7 +540,7 @@ certification chaining.
:param data: Dictionary, bytes, or file-like object to
send in the body of the request (optional)

:param json: Any json compatible python object (optional). `json` and `data`
:param json: Any json compatible python object (optional). *json* and *data*
parameters could not be used at the same time.

:param dict headers: HTTP Headers to send with the request (optional)
Expand Down Expand Up @@ -598,8 +612,8 @@ There are standard connectors:

All connector classes should be derived from :class:`BaseConnector`.

By default all *connectors* support *keep-alive connections* (behavior is controlled by
*force_close* constructor's parameter).
By default all *connectors* support *keep-alive connections* (behavior
is controlled by *force_close* constructor's parameter).


BaseConnector
Expand Down Expand Up @@ -973,8 +987,9 @@ Response object

Reading from the stream may raise
:exc:`aiohttp.ClientPayloadError` if the response object is
closed before response receives all data or in case if any transfer encoding
related errors like mis-formed chunked encoding of broken compression data.
closed before response receives all data or in case if any
transfer encoding related errors like mis-formed chunked
encoding of broken compression data.

.. attribute:: cookies

Expand Down Expand Up @@ -1037,13 +1052,16 @@ Response object

.. comethod:: release()

It is not required to call `release` on the response object. When the
client fully receives the payload, the underlying connection automatically
returns back to pool. If the payload is not fully read, the connection is closed
It is not required to call `release` on the response
object. When the client fully receives the payload, the
underlying connection automatically returns back to pool. If the
payload is not fully read, the connection is closed

.. method:: raise_for_status()

Raise an :exc:`aiohttp.ClientResponseError` if the response status is 400 or higher.
Raise an :exc:`aiohttp.ClientResponseError` if the response
status is 400 or higher.

Do nothing for success responses (less than 400).

.. comethod:: text(encoding=None)
Expand All @@ -1052,7 +1070,10 @@ Response object
specified *encoding* parameter.

If *encoding* is ``None`` content encoding is autocalculated
using :term:`cchardet` or :term:`chardet` as fallback if
using ``Content-Type`` HTTP header and *chardet* tool if the
header is not provided by server.

:term:`cchardet` is used with fallback to :term:`chardet` if
*cchardet* is not available.

Close underlying connection if data reading gets an error,
Expand All @@ -1064,7 +1085,19 @@ Response object

:return str: decoded *BODY*

.. comethod:: json(encoding=None, loads=json.loads, content_type='application/json')
.. note::

If response has no ``charset`` info in ``Content-Type`` HTTP
header :term:`cchardet` / :term:`chardet` is used for content
encoding autodetection.

It may hurt performance. If page encoding is known passing
explicit *encoding* parameter might help::

await resp.text('ISO-8859-1')

.. comethod:: json(*, encoding=None, loads=json.loads, \
content_type='application/json')

Read response's body as *JSON*, return :class:`dict` using
specified *encoding* and *loader*.
Expand Down Expand Up @@ -1092,10 +1125,10 @@ Response object
:return: *BODY* as *JSON* data parsed by *loads* parameter or
``None`` if *BODY* is empty or contains white-spaces only.

.. attribute:: request_info
.. attribute:: request_info

A namedtuple with request URL and headers from :class:`ClientRequest`
object.
object, :class:`aiohttp.RequestInfo` instance.


ClientWebSocketResponse
Expand Down Expand Up @@ -1240,6 +1273,27 @@ Utilities
---------


RequestInfo
^^^^^^^^^^^

.. class:: RequestInfo()

A namedtuple with request URL and headers from :class:`ClientRequest`
object, available as :attr:`ClientResponse.request_info` attribute.

.. attribute:: url

Requested *url*, :class:`yarl.URL` instance.

.. attribute:: method

Request HTTP method like ``'GET'`` or ``'POST'``, :class:`str`.

.. attribute:: headers

HTTP headers for request, :class:`multidict.CIMultiDict` instance.


BasicAuth
^^^^^^^^^

Expand All @@ -1249,7 +1303,7 @@ BasicAuth

:param str login: login
:param str password: password
:param str encoding: encoding (`'latin1'` by default)
:param str encoding: encoding (``'latin1'`` by default)


Should be used for specifying authorization data in client API,
Expand Down Expand Up @@ -1351,24 +1405,28 @@ CookieJar
Client exceptions
^^^^^^^^^^^^^^^^^

Exception hierarchy has been significantly modified in version 2.0. aiohttp defines only
exceptions that covers connection handling and server response misbehaviors.
For developer specific mistakes, aiohttp uses python standard exceptions
like `ValueError` or `TypeError`.
Exception hierarchy has been significantly modified in version
2.0. aiohttp defines only exceptions that covers connection handling
and server response misbehaviors. For developer specific mistakes,
aiohttp uses python standard exceptions like :exc:`ValueError` or
:exc:`TypeError`.

Reading a response content may raise a :exc:`ClientPayloadError` exception. This exception
indicates errors specific to the payload encoding. Such as invalid compressed data,
malformed chunked-encoded chunks or not enough data that satisfy the content-length header.
Reading a response content may raise a :exc:`ClientPayloadError`
exception. This exception indicates errors specific to the payload
encoding. Such as invalid compressed data, malformed chunked-encoded
chunks or not enough data that satisfy the content-length header.

All exceptions are available as attributes in `aiohttp` module.
All exceptions are available as members of *aiohttp* module.

Hierarchy of exceptions:

* `aiohttp.ClientError` - Base class for all client specific exceptions
* :exc:`aiohttp.ClientError` - Base class for all client specific exceptions

- `aiohttp.ClientResponseError` - exceptions that could happen after we get response from server.
- :exc:`aiohttp.ClientResponseError` - exceptions that could happen after
we get response from server.

`request_info` - Instance of `RequestInfo` object, contains information about request.
`request_info` - Instance of `RequestInfo` object, contains
information about request.

`history` - History from `ClientResponse` object, if available, else empty tuple.

Expand Down

0 comments on commit 9dc8423

Please sign in to comment.