Support asyncio (#1714) (#1726)

All public classes have async versions now. The documentation navigation
was also updated to follow the Diátaxis documentation framework.

(cherry picked from commit b50d538)

Co-authored-by: Miguel Grinberg <[email protected]>

CONTRIBUTING.rst

@@ -25,22 +25,34 @@ The process for contributing to any of the Elasticsearch repositories is similar
    assure our users of the origin and continuing existence of the code. You only
    need to sign the CLA once.
 
-2. Run the test suite to ensure your changes do not break existing code:
+2. Many classes included in this library are offered in two versions, for
+   asynchronous and synchronous Python. When working with these classes, you only
+   need to make changes to the asynchronous code, located in *_async*
+   subdirectories in the source and tests trees. Once you've made your changes,
+   run the following command to automatically generate the corresponding
+   synchronous code:
+
+    .. code:: bash
+
+        $ nox -rs format
+   
+3. Run the test suite to ensure your changes do not break existing code:
 
     .. code:: bash
 
         $ nox -rs lint test
 
-3. Rebase your changes.
+4. Rebase your changes.
    Update your local repository with the most recent code from the main
    elasticsearch-dsl-py repository, and rebase your branch on top of the latest master
    branch. We prefer your changes to be squashed into a single commit.
 
-4. Submit a pull request. Push your local changes to your forked copy of the
+5. Submit a pull request. Push your local changes to your forked copy of the
    repository and submit a pull request. In the pull request, describe what your
    changes do and mention the number of the issue where discussion has taken
    place, eg “Closes #123″.  Please consider adding or modifying tests related to
-   your changes.
+   your changes. Include any generated files in the *_sync* subdirectory in your
+   pull request.
 
 Then sit back and wait. There will probably be discussion about the pull
 request and, if any changes are needed, we would love to work with you to get

docs/api.rst

@@ -4,38 +4,25 @@ API Documentation
 =================
 
 Below please find the documentation for the public classes and functions of ``elasticsearch_dsl``.
+The :ref:`Asynchronous API <async_api>` classes are documented separately.
 
 .. py:module:: elasticsearch_dsl
 
-Search
-------
-
 .. autoclass:: Search
    :members:
 
 .. autoclass:: MultiSearch
    :members:
 
-Document
---------
-
 .. autoclass:: Document
    :members:
 
-Index
------
-
 .. autoclass:: Index
    :members:
 
-Faceted Search
---------------
-
 .. autoclass:: FacetedSearch
    :members:
 
-Update By Query 
-----------------
 .. autoclass:: UpdateByQuery
   :members:
 
@@ -108,5 +95,3 @@ Common field options:
 
 ``required``
   Indicates if a field requires a value for the document to be valid.
-
-

docs/async_api.rst

@@ -0,0 +1,33 @@
+.. _async_api:
+
+Asynchronous API Documentation
+==============================
+
+Below please find the documentation for the asychronous classes of ``elasticsearch_dsl``.
+
+.. py:module:: elasticsearch_dsl
+   :no-index:
+
+.. autoclass:: AsyncSearch
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncMultiSearch
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncDocument
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncIndex
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncFacetedSearch
+   :inherited-members:
+   :members:
+
+.. autoclass:: AsyncUpdateByQuery
+   :inherited-members:
+   :members:

docs/asyncio.rst

@@ -0,0 +1,94 @@
+.. _asyncio:
+
+Using asyncio with Elasticsearch DSL
+====================================
+
+The ``elasticsearch-dsl`` package supports async/await with `asyncio <https://docs.python.org/3/library/asyncio.html>`__.
+To ensure that you have all the required dependencies, install the ``[async]`` extra:
+
+   .. code:: bash
+
+       $ python -m pip install elasticsearch-dsl[async]
+
+Connections
+-----------
+
+Use the ``async_connections`` module to manage your asynchronous connections.
+
+   .. code:: python
+
+       from elasticsearch_dsl import async_connections
+
+       async_connections.create_connection(hosts=['localhost'], timeout=20)
+
+All the options available in the ``connections`` module can be used with ``async_connections``.
+
+How to avoid 'Unclosed client session / connector' warnings on exit
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+These warnings come from the ``aiohttp`` package, which is used internally by the
+``AsyncElasticsearch`` client. They appear often when the application exits and
+are caused by HTTP connections that are open when they are garbage collected. To
+avoid these warnings, make sure that you close your connections.
+
+   .. code:: python
+
+       es = async_connections.get_connection()
+       await es.close()
+
+Search DSL
+----------
+
+Use the ``AsyncSearch`` class to perform asynchronous searches.
+
+   .. code:: python
+
+       from elasticsearch_dsl import AsyncSearch
+
+       s = AsyncSearch().query("match", title="python")
+       async for hit in s:
+           print(hit.title)
+
+Instead of using the ``AsyncSearch`` object as an asynchronous iterator, you can
+explicitly call the ``execute()`` method to get a ``Response`` object.
+
+    .. code:: python
+
+       s = AsyncSearch().query("match", title="python")
+       response = await s.execute()
+       for hit in response:
+           print(hit.title)
+ 
+An ``AsyncMultiSearch`` is available as well.
+
+   .. code:: python
+
+       from elasticsearch_dsl import AsyncMultiSearch
+
+       ms = AsyncMultiSearch(index='blogs')
+
+       ms = ms.add(AsyncSearch().filter('term', tags='python'))
+       ms = ms.add(AsyncSearch().filter('term', tags='elasticsearch'))
+
+       responses = await ms.execute()
+
+       for response in responses:
+           print("Results for query %r." % response.search.query)
+           for hit in response:
+               print(hit.title)
+
+Asynchronous Documents, Indexes, and more
+-----------------------------------------
+
+The ``Document``, ``Index``, ``IndexTemplate``, ``Mapping``, ``UpdateByQuery`` and
+``FacetedSearch`` classes all have asynchronous versions that use the same name
+with an ``Async`` prefix. These classes expose the same interfaces as the
+synchronous versions, but any methods that perform I/O are defined as coroutines.
+
+Auxiliary classes that do not perform I/O do not have asynchronous versions. The
+same classes can be used in synchronous and asynchronous applications.
+
+When using a :ref:`custom analyzer <Analysis>` in an asynchronous application, use
+the ``async_simulate()`` method to invoke the Analyze API on it.
+
+Consult the :ref:`api` section for details about each specific method.