Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hide Nursery constructor, finalize and expose Nursery #1090

Merged
merged 4 commits into from
Jun 14, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/history.rst
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,7 @@ Highlights
:ref:`async-file-io` (`gh-20
<https://github.com/python-trio/trio/pull/20>`__)

* The new nursery :meth:`~The nursery interface.start` method makes it
* The new nursery :meth:`~Nursery.start` method makes it
easy to perform controlled start-up of long-running tasks. For
example, given an appropriate ``http_server_on_random_open_port``
function, you could write::
Expand Down
109 changes: 3 additions & 106 deletions docs/source/reference-core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -819,116 +819,13 @@ The nursery API
.. autofunction:: open_nursery
:async-with: nursery

Nursery objects provide the following interface:

.. interface:: The nursery interface

.. method:: start_soon(async_fn, *args, name=None)

Creates a new child task inside this nursery, and sets it up to
run ``await async_fn(*args)``.

This and :meth:`start` are the two fundamental methods for
creating concurrent tasks in Trio.

Note that this is *not* an async function and you don't use await
when calling it. It sets up the new task, but then returns
immediately, *before* it has a chance to run. The new task won’t
actually get a chance to do anything until some later point when
you execute a checkpoint and the scheduler decides to run it.
If you want to run a function and immediately wait for its result,
then you don't need a nursery; just use ``await async_fn(*args)``.
If you want to wait for the task to initialize itself before
continuing, see :meth:`start()`.

It's possible to pass a nursery object into another task, which
allows that task to start new child tasks in the first task's
nursery.

The child task inherits its parent nursery's cancel scopes.

:param async_fn: An async callable.
:param args: Positional arguments for ``async_fn``. If you want
to pass keyword arguments, use
:func:`functools.partial`.
:param name: The name for this task. Only used for
debugging/introspection
(e.g. ``repr(task_obj)``). If this isn't a string,
:meth:`start_soon` will try to make it one. A
common use case is if you're wrapping a function
before spawning a new task, you might pass the
original function as the ``name=`` to make
debugging easier.
:raises RuntimeError: If this nursery is no longer open
(i.e. its ``async with`` block has
exited).

.. method:: start(async_fn, *args, name=None)
:async:

Like :meth:`start_soon`, but blocks until the new task has
finished initializing itself, and optionally returns some
information from it.

The ``async_fn`` must accept a ``task_status`` keyword argument,
and it must make sure that it (or someone) eventually calls
``task_status.started()``.

The conventional way to define ``async_fn`` is like::

async def async_fn(arg1, arg2, *, task_status=trio.TASK_STATUS_IGNORED):
...
task_status.started()
...

:attr:`trio.TASK_STATUS_IGNORED` is a special global object with
a do-nothing ``started`` method. This way your function supports
being called either like ``await nursery.start(async_fn, arg1,
arg2)`` or directly like ``await async_fn(arg1, arg2)``, and
either way it can call ``task_status.started()`` without
worrying about which mode it's in. Defining your function like
this will make it obvious to readers that it supports being used
in both modes.

Before the child calls ``task_status.started()``, it's
effectively run underneath the call to :meth:`start`: if it
raises an exception then that exception is reported by
:meth:`start`, and does *not* propagate out of the nursery. If
:meth:`start` is cancelled, then the child task is also
cancelled.

When the child calls ``task_status.started()``, it's moved from
out from underneath :meth:`start` and into the given nursery.

If the child task passes a value to
``task_status.started(value)``, then :meth:`start` returns this
value. Otherwise it returns ``None``.

.. attribute:: cancel_scope

Creating a nursery also implicitly creates a cancellation scope,
which is exposed as the :attr:`cancel_scope` attribute. This is
used internally to implement the logic where if an error occurs
then ``__aexit__`` cancels all children, but you can use it for
other things, e.g. if you want to explicitly cancel all children
in response to some external event.

The last two attributes are mainly to enable introspection of the
task tree, for example in debuggers.

.. attribute:: parent_task

The :class:`~trio.hazmat.Task` that opened this nursery.

.. attribute:: child_tasks

A :class:`frozenset` containing all the child
:class:`~trio.hazmat.Task` objects which are still running.

.. autoclass:: Nursery()
:members:

.. attribute:: TASK_STATUS_IGNORED

See :meth:`~The nursery interface.start`.
See :meth:`~Nursery.start`.


Working with :exc:`MultiError`\s
Expand Down
7 changes: 4 additions & 3 deletions newsfragments/1021.misc.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
Any attempt to inherit from :class:`CancelScope` now raises TypeError.
(Trio has never been able to safely support subclassing here;
this change just makes it more obvious.)
Any attempt to inherit from `CancelScope` or `Nursery` now raises
`TypeError`. (Trio has never been able to safely support subclassing
here; this change just makes it more obvious.)
Also exposed as public classes for type-checking, etc.
3 changes: 2 additions & 1 deletion trio/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@
TrioInternalError, RunFinishedError, WouldBlock, Cancelled,
BusyResourceError, ClosedResourceError, MultiError, run, open_nursery,
CancelScope, open_cancel_scope, current_effective_deadline,
TASK_STATUS_IGNORED, current_time, BrokenResourceError, EndOfChannel
TASK_STATUS_IGNORED, current_time, BrokenResourceError, EndOfChannel,
Nursery
)

from ._timeouts import (
Expand Down
117 changes: 112 additions & 5 deletions trio/_core/_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@
)
from .. import _core
from .._deprecate import deprecated
from .._util import Final
from .._util import Final, NoPublicConstructor

# At the bottom of this file there's also some "clever" code that generates
# wrapper functions for runner and io manager methods, and adds them to
Expand All @@ -46,7 +46,7 @@
__all__ = [
"Task", "run", "open_nursery", "open_cancel_scope", "CancelScope",
"checkpoint", "current_task", "current_effective_deadline",
"checkpoint_if_cancelled", "TASK_STATUS_IGNORED"
"checkpoint_if_cancelled", "TASK_STATUS_IGNORED", "Nursery"
]

GLOBAL_RUN_CONTEXT = threading.local()
Expand Down Expand Up @@ -717,7 +717,7 @@ class NurseryManager:
async def __aenter__(self):
self._scope = CancelScope()
self._scope.__enter__()
self._nursery = Nursery(current_task(), self._scope)
self._nursery = Nursery._create(current_task(), self._scope)
return self._nursery

@enable_ki_protection
Expand Down Expand Up @@ -752,7 +752,7 @@ def __exit__(self): # pragma: no cover

def open_nursery():
"""Returns an async context manager which must be used to create a
new ``Nursery``.
new `Nursery`.

It does not block on entry; on exit it blocks until all child tasks
have exited.
Expand All @@ -761,7 +761,28 @@ def open_nursery():
return NurseryManager()


class Nursery:
class Nursery(metaclass=NoPublicConstructor):
"""A context which may be used to spawn (or cancel) child tasks.

Not constructed directly, use `open_nursery` instead.

The nursery will remain open until all child tasks have completed,
or until it is cancelled, at which point it will cancel all its
remaining child tasks and close.

Nurseries ensure the absence of orphaned Tasks, since all running
tasks will belong to an open Nursery.

Attributes:
cancel_scope:
Creating a nursery also implicitly creates a cancellation scope,
which is exposed as the :attr:`cancel_scope` attribute. This is
used internally to implement the logic where if an error occurs
then ``__aexit__`` cancels all children, but you can use it for
other things, e.g. if you want to explicitly cancel all children
in response to some external event.
"""

def __init__(self, parent_task, cancel_scope):
self._parent_task = parent_task
parent_task._child_nurseries.append(self)
Expand All @@ -784,10 +805,13 @@ def __init__(self, parent_task, cancel_scope):

@property
def child_tasks(self):
"""(`frozenset`): Contains all the child :class:`~trio.hazmat.Task`
objects which are still running."""
return frozenset(self._children)

@property
def parent_task(self):
"(`~trio.hazmat.Task`): The Task that opened this nursery."
return self._parent_task

def _add_exc(self, exc):
Expand Down Expand Up @@ -841,9 +865,92 @@ def aborted(raise_cancel):
return MultiError(self._pending_excs)

def start_soon(self, async_fn, *args, name=None):
""" Creates a child task, scheduling ``await async_fn(*args)``.

This and :meth:`start` are the two fundamental methods for
creating concurrent tasks in Trio.

Note that this is *not* an async function and you don't use await
when calling it. It sets up the new task, but then returns
immediately, *before* it has a chance to run. The new task won’t
actually get a chance to do anything until some later point when
you execute a checkpoint and the scheduler decides to run it.
If you want to run a function and immediately wait for its result,
then you don't need a nursery; just use ``await async_fn(*args)``.
If you want to wait for the task to initialize itself before
continuing, see :meth:`start()`.

It's possible to pass a nursery object into another task, which
allows that task to start new child tasks in the first task's
nursery.

The child task inherits its parent nursery's cancel scopes.

Args:
async_fn: An async callable.
args: Positional arguments for ``async_fn``. If you want
to pass keyword arguments, use
:func:`functools.partial`.
name: The name for this task. Only used for
debugging/introspection
(e.g. ``repr(task_obj)``). If this isn't a string,
:meth:`start_soon` will try to make it one. A
common use case is if you're wrapping a function
before spawning a new task, you might pass the
original function as the ``name=`` to make
debugging easier.

Returns:
True if successful, False otherwise.

Raises:
RuntimeError: If this nursery is no longer open
(i.e. its ``async with`` block has
exited).
"""
GLOBAL_RUN_CONTEXT.runner.spawn_impl(async_fn, args, self, name)

async def start(self, async_fn, *args, name=None):
r""" Creates and initalizes a child task.

Like :meth:`start_soon`, but blocks until the new task has
finished initializing itself, and optionally returns some
information from it.

The ``async_fn`` must accept a ``task_status`` keyword argument,
and it must make sure that it (or someone) eventually calls
``task_status.started()``.

The conventional way to define ``async_fn`` is like::

async def async_fn(arg1, arg2, \*, task_status=trio.TASK_STATUS_IGNORED):
...
task_status.started()
...

:attr:`trio.TASK_STATUS_IGNORED` is a special global object with
a do-nothing ``started`` method. This way your function supports
being called either like ``await nursery.start(async_fn, arg1,
arg2)`` or directly like ``await async_fn(arg1, arg2)``, and
either way it can call ``task_status.started()`` without
worrying about which mode it's in. Defining your function like
this will make it obvious to readers that it supports being used
in both modes.

Before the child calls ``task_status.started()``, it's
effectively run underneath the call to :meth:`start`: if it
raises an exception then that exception is reported by
:meth:`start`, and does *not* propagate out of the nursery. If
:meth:`start` is cancelled, then the child task is also
cancelled.

When the child calls ``task_status.started()``, it's moved from
out from underneath :meth:`start` and into the given nursery.

If the child task passes a value to
``task_status.started(value)``, then :meth:`start` returns this
value. Otherwise it returns ``None``.
"""
if self._closed:
raise RuntimeError("Nursery is closed to new arrivals")
try:
Expand Down
24 changes: 24 additions & 0 deletions trio/_core/tests/test_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -2142,6 +2142,30 @@ async def inner():
_core.run(inner)


def test_Nursery_init():
check_Nursery_error = pytest.raises(
TypeError, match='no public constructor available'
)

with check_Nursery_error:
_core._run.Nursery(None, None)


async def test_Nursery_private_init():
# context manager creation should not raise
async with _core.open_nursery() as nursery:
assert False == nursery._closed


def test_Nursery_subclass():
with pytest.raises(
TypeError, match='`Nursery` does not support subclassing'
):

class Subclass(_core._run.Nursery):
pass


def test_Cancelled_init():
check_Cancelled_error = pytest.raises(
TypeError, match='no public constructor available'
Expand Down