Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace MultiError with BaseExceptionGroup #2213

Merged
merged 104 commits into from
Sep 27, 2022
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
104 commits
Select commit Hold shift + click to select a range
31706e8
Add news fragment for #2210
agronholm Jan 25, 2022
743354b
Replace MultiError with (Base)ExceptionGroup
agronholm Jan 15, 2022
6fdd0fe
Collapse exception groups containing a single exception
agronholm Jan 16, 2022
13de11d
Require exceptiongroup unconditionally for tests
agronholm Jan 16, 2022
9ecca9b
Added exceptiongroup to docs requirements
agronholm Jan 25, 2022
21757c8
Fixed dangling references in the documentation
agronholm Jan 25, 2022
61a1646
Removed obsolete check
agronholm Jan 25, 2022
ca05efb
Removed one last MultiError reference in docs
agronholm Jan 25, 2022
0b159e8
Found one more MultiError, in a news fragment
agronholm Jan 25, 2022
e9aaabc
Added a test for exception group collapsing
agronholm Jan 25, 2022
83e054c
Merge branch 'master' into exceptiongroup
pquentin Jan 27, 2022
370e40f
Merge branch 'master' into exceptiongroup
agronholm Jan 30, 2022
e2fbec0
Updated an example to refer to BaseExceptionGroup
agronholm Jan 30, 2022
95a7b9e
Improved the docs regarding exception groups
agronholm Jan 30, 2022
745f3cf
Added Python 3.11 to the CI matrix
agronholm Jan 30, 2022
bc043aa
Attempt at fixing Python 3.11 test runs
agronholm Jan 30, 2022
797c234
Removed Windows and macOS py3.11 test jobs
agronholm Jan 30, 2022
8a09778
Skip test_coroutine_or_error() on Python 3.11
agronholm Jan 30, 2022
c1200de
Merge branch 'master' into exceptiongroup
agronholm Jan 31, 2022
30358d1
Changed case to be consistent in BEG messages
agronholm Jan 31, 2022
54eb0ce
Restored (and deprecated) MultiError
agronholm Feb 3, 2022
f1cbcf7
Removed unwarranted type annotations
agronholm Feb 3, 2022
0f5dc79
Removed useless failing test
agronholm Feb 3, 2022
36e0eba
Removed unused function
agronholm Feb 3, 2022
20df5d8
Update docs/source/reference-core.rst
agronholm Feb 8, 2022
6eeb137
Merge branch 'master' into exceptiongroup
agronholm Feb 8, 2022
2dc1c64
Update newsfragments/2211.deprecated.rst
agronholm Feb 8, 2022
e7faa46
Update docs/source/tutorial/echo-server.py
agronholm Feb 8, 2022
c41517a
Update trio/_highlevel_open_tcp_stream.py
agronholm Feb 8, 2022
50963c6
Don't skip the entire test_coroutine_or_error test on py3.11
agronholm Feb 8, 2022
ba17fbb
Removed weird test code
agronholm Feb 8, 2022
8911e0a
Update newsfragments/2211.deprecated.rst
agronholm Feb 8, 2022
dfd084c
Bump prompt-toolkit from 3.0.26 to 3.0.27
dependabot[bot] Feb 8, 2022
b3c79c8
Bump tomli from 2.0.0 to 2.0.1
dependabot[bot] Feb 9, 2022
81feff2
Bump platformdirs from 2.4.1 to 2.5.0
dependabot[bot] Feb 10, 2022
bcb442b
Bump towncrier from 21.3.0 to 21.9.0
dependabot[bot] Feb 11, 2022
ab78e02
Bump prompt-toolkit from 3.0.27 to 3.0.28
dependabot[bot] Feb 11, 2022
9abddfa
Bump markupsafe from 2.0.1 to 2.1.0
dependabot[bot] Feb 18, 2022
efc8b01
Bump click from 8.0.3 to 8.0.4
dependabot[bot] Feb 21, 2022
9a4f512
Bump platformdirs from 2.5.0 to 2.5.1
dependabot[bot] Feb 21, 2022
6cb5ad9
Merged test requirements
agronholm Mar 9, 2022
ee245a0
Bump pytest from 7.0.0 to 7.0.1
dependabot[bot] Feb 14, 2022
1e36f8c
Tickle sr.ht CI
pquentin Feb 21, 2022
034bd32
Merge two competing newsfragments
pquentin Feb 21, 2022
80a751f
Bump charset-normalizer from 2.0.11 to 2.0.12
dependabot[bot] Feb 21, 2022
9eb2d1c
Tickle sr.ht CI
pquentin Feb 21, 2022
fd56492
Bump version to 0.20.0
pquentin Feb 21, 2022
af6233b
Bump version to 0.20.0+dev
pquentin Feb 21, 2022
a4876bc
Renamed the news fragment
agronholm Mar 9, 2022
5c09f44
Updated wrapt in test requirements
agronholm Mar 9, 2022
c5af0a4
Changed wrapt target git hash so it's compatible with astroid
agronholm Mar 9, 2022
93c81f0
Merge branch 'master' into exceptiongroup
agronholm May 14, 2022
b4791f8
Fixed error in test_coroutine_or_error() on py3.11
agronholm May 14, 2022
70bb109
Updated dependencies
agronholm May 14, 2022
1c403b1
Move Python 3.11 to its separate test job
agronholm May 14, 2022
a28bfac
Fixed bad pytest switch
agronholm May 14, 2022
7694592
Implemented the strict_exception_groups setting
agronholm May 14, 2022
161c0ab
Implemented NonBaseMultiError which inherits from both MultiError and…
agronholm May 14, 2022
226184f
Fixed MultiError reference in version history
agronholm May 14, 2022
748ceb8
Removed Python 3.11 from test matrix
agronholm May 14, 2022
dfdda5b
Documented the MultiError compatibility issue
agronholm May 14, 2022
d49abbb
Merge branch 'master' into exceptiongroup
agronholm May 16, 2022
a877191
Merge branch 'master' into exceptiongroup
agronholm May 16, 2022
e8a1b60
Documentation updates
agronholm May 17, 2022
4bd09bc
Changed open_tcp_listeners() and open_tcp_stream() to use ExceptionGr…
agronholm May 17, 2022
1101ead
Expose NonBaseMultiError in trio (as a deprecated attribute)
agronholm May 17, 2022
cd11213
Changed wording in MultiError deprecation messages
agronholm May 20, 2022
7088be4
Added missing install dependency on exceptiongroup
agronholm May 20, 2022
5bc2bd9
Merge branch 'master' into exceptiongroup
agronholm Jun 7, 2022
e87de6c
Updated trio version in deprecation notes
agronholm Jun 7, 2022
6b5e4f6
Fixed formatting error
agronholm Jun 7, 2022
9ff7a45
Use trio's own deprecation mechanism
agronholm Jul 4, 2022
2140b70
Added comment on _collapse=False in derive()
agronholm Jul 4, 2022
1da1af4
Only modify MultiErrors in collapse_exception_group()
agronholm Jul 4, 2022
f9e11fc
Retain traceback frames from the MultiError itself
agronholm Jul 4, 2022
1d99b23
Improved prevention of MultiError double initialization
agronholm Jul 9, 2022
2591808
Reworded the comments in the exception group handling examples
agronholm Jul 9, 2022
afd798f
Merge branch 'master' into exceptiongroup
agronholm Jul 9, 2022
8c77459
Made all references to (Base)ExceptionGroup into links
agronholm Jul 15, 2022
b621e70
Improved the documentation for exception group handling
agronholm Jul 17, 2022
656fad4
Fixed test collection error on Python 3.11
agronholm Jul 20, 2022
6e86da3
Readded the exceptiongroups label
agronholm Jul 20, 2022
e29c00f
Merge branch 'master' into exceptiongroup
agronholm Sep 4, 2022
16c19b6
Spelled out MultiError.filter() and MultiError.catch()
agronholm Sep 18, 2022
c35fccc
Fixed the instead= part in the deprecation warning
agronholm Sep 18, 2022
7b7acb9
Replaced warn() with warn_deprecated()
agronholm Sep 18, 2022
3b97ce1
Merge branch 'master' into exceptiongroup
agronholm Sep 18, 2022
1075ff0
Fixed the check for StopAsyncIteration
agronholm Sep 18, 2022
458f5e2
Fixed comment in _nested_child_finished()
agronholm Sep 18, 2022
0f39b3f
Fixed deprecation checking in MultiError.catch() tests
agronholm Sep 18, 2022
dc35213
Reversed the order of tracebacks when concatenating
agronholm Sep 18, 2022
f9eff8e
Restored the IPython custom error handler
agronholm Sep 18, 2022
30b8f3b
Added link to specific IPython issue
agronholm Sep 20, 2022
232688b
Updated wording about extracting submodules to independent packages
agronholm Sep 20, 2022
0b8e908
If present, patch Apport to support exception groups
agronholm Sep 25, 2022
878af56
Added test coverage for collapse_exception_group()
agronholm Sep 25, 2022
cc7d76a
Changed cancel scopes to not collapse exceptions by default
agronholm Sep 25, 2022
a20e2d8
Extended test coverage
agronholm Sep 26, 2022
1aba33e
Fixed the apport script test
agronholm Sep 26, 2022
0c283b7
Fixed MultiError.collapse attribute missing on Python 3.11
agronholm Sep 26, 2022
0364b48
Added test coverage for cancel scope encountering an excgroup w/o Can…
agronholm Sep 26, 2022
bb31ed6
Removed an except block that was never entered
agronholm Sep 26, 2022
2fd8309
Fixed excepthook wrapper name
agronholm Sep 27, 2022
80889b2
Removed unused parameter
agronholm Sep 27, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs-requirements.in
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ async_generator >= 1.9
idna
outcome
sniffio
exceptiongroup

# See note in test-requirements.in
immutables >= 0.6
17 changes: 15 additions & 2 deletions docs-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#
# This file is autogenerated by pip-compile
# This file is autogenerated by pip-compile with python 3.7
# To update, run:
#
# pip-compile --output-file docs-requirements.txt docs-requirements.in
# pip-compile --output-file=docs-requirements.txt docs-requirements.in
#
alabaster==0.7.12
# via sphinx
Expand All @@ -28,6 +28,8 @@ docutils==0.17.1
# via
# sphinx
# sphinx-rtd-theme
exceptiongroup==1.0.0rc1
# via -r docs-requirements.in
idna==3.3
# via
# -r docs-requirements.in
Expand All @@ -36,6 +38,8 @@ imagesize==1.2.0
# via sphinx
immutables==0.16
# via -r docs-requirements.in
importlib-metadata==4.10.1
# via click
incremental==21.3.0
# via towncrier
jinja2==3.0.2
Expand Down Expand Up @@ -87,5 +91,14 @@ toml==0.10.2
# via towncrier
towncrier==21.3.0
# via -r docs-requirements.in
typing-extensions==4.0.1
# via
# immutables
# importlib-metadata
urllib3==1.26.7
# via requests
zipp==3.7.0
# via importlib-metadata

# The following packages are considered to be unsafe in a requirements file:
# setuptools
3 changes: 0 additions & 3 deletions docs/source/design.rst
Original file line number Diff line number Diff line change
Expand Up @@ -465,9 +465,6 @@ There are two notable sub-modules that are largely independent of
the rest of Trio, and could (possibly should?) be extracted into their
own independent packages:
Zac-HD marked this conversation as resolved.
Show resolved Hide resolved

* ``_multierror.py``: Implements :class:`MultiError` and associated
infrastructure.

* ``_ki.py``: Implements the core infrastructure for safe handling of
:class:`KeyboardInterrupt`.

Expand Down
10 changes: 5 additions & 5 deletions docs/source/history.rst
Original file line number Diff line number Diff line change
Expand Up @@ -232,9 +232,9 @@ Bugfixes
- On Ubuntu systems, the system Python includes a custom
unhandled-exception hook to perform `crash reporting
<https://wiki.ubuntu.com/Apport>`__. Unfortunately, Trio wants to use
the same hook to print nice `MultiError` tracebacks, causing a
the same hook to print nice ``MultiError`` tracebacks, causing a
conflict. Previously, Trio would detect the conflict, print a warning,
and you just wouldn't get nice `MultiError` tracebacks. Now, Trio has
and you just wouldn't get nice ``MultiError`` tracebacks. Now, Trio has
gotten clever enough to integrate its hook with Ubuntu's, so the two
systems should Just Work together. (`#1065 <https://github.com/python-trio/trio/issues/1065>`__)
- Fixed an over-strict test that caused failures on Alpine Linux.
Expand Down Expand Up @@ -436,7 +436,7 @@ Features
violated. (One common source of such violations is an async generator
that yields within a cancel scope.) The previous behavior was an
inscrutable chain of TrioInternalErrors. (`#882 <https://github.com/python-trio/trio/issues/882>`__)
- MultiError now defines its ``exceptions`` attribute in ``__init__()``
- ``MultiError`` now defines its ``exceptions`` attribute in ``__init__()``
to better support linters and code autocompletion. (`#1066 <https://github.com/python-trio/trio/issues/1066>`__)
- Use ``__slots__`` in more places internally, which should make Trio slightly faster. (`#984 <https://github.com/python-trio/trio/issues/984>`__)

Expand All @@ -457,7 +457,7 @@ Bugfixes
:meth:`~trio.Path.cwd`, are now async functions. Previously, a bug
in the forwarding logic meant :meth:`~trio.Path.cwd` was synchronous
and :meth:`~trio.Path.home` didn't work at all. (`#960 <https://github.com/python-trio/trio/issues/960>`__)
- An exception encapsulated within a :class:`MultiError` doesn't need to be
- An exception encapsulated within a ``MultiError`` doesn't need to be
hashable anymore.

.. note::
Expand Down Expand Up @@ -1248,7 +1248,7 @@ Other changes
interfering with direct use of
:func:`~trio.testing.wait_all_tasks_blocked` in the same test.

* :meth:`MultiError.catch` now correctly preserves ``__context__``,
* ``MultiError.catch()`` now correctly preserves ``__context__``,
despite Python's best attempts to stop us (`#165
<https://github.com/python-trio/trio/issues/165>`__)

Expand Down
109 changes: 3 additions & 106 deletions docs/source/reference-core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -641,7 +641,7 @@ crucial things to keep in mind:

* Any unhandled exceptions are re-raised inside the parent task. If
there are multiple exceptions, then they're collected up into a
single :exc:`MultiError` exception.
single ``BaseExceptionGroup`` or ``ExceptionGroup`` exception.
agronholm marked this conversation as resolved.
Show resolved Hide resolved

Since all tasks are descendents of the initial task, one consequence
of this is that :func:`run` can't finish until all tasks have
Expand Down Expand Up @@ -712,14 +712,9 @@ limitation. Consider code like::
what? In some sense, the answer should be "both of these at once", but
in Python there can only be one exception at a time.

Trio's answer is that it raises a :exc:`MultiError` object. This is a
Trio's answer is that it raises a ``BaseExceptionGroup`` object. This is a
special exception which encapsulates multiple exception objects –
either regular exceptions or nested :exc:`MultiError`\s. To make these
easier to work with, Trio installs a custom `sys.excepthook` that
knows how to print nice tracebacks for unhandled :exc:`MultiError`\s,
and it also provides some helpful utilities like
:meth:`MultiError.catch`, which allows you to catch "part of" a
:exc:`MultiError`.
either regular exceptions or nested ``BaseExceptionGroup``\s.


Spawning tasks without becoming a parent
Expand Down Expand Up @@ -837,104 +832,6 @@ The nursery API
See :meth:`~Nursery.start`.


Working with :exc:`MultiError`\s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this section be updated for ExceptionGroups instead of being removed? I know it's not really "our" feature anymore, but the existing docs on ExceptionGroups are pretty scant since they're not in a released Python, and we're adopting them as part of our API so it would be nice not to lose friendliness in the transition.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there something wrong with the documentation I just added above which specifically shows how to use both except* and catch()?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's nothing wrong with it, but it does feel like a downgrade when the previous set of docs had multiple examples, describing more edge cases, exception chaining, output, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll restore and update the docs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I took the time to evaluate the examples from the removed section, but since the catching API in the backport works in such a different way, I'm not sure what there is to document. The backport doesn't provide any functionality that the except* syntax doesn't. There's no exception conversion, and to reraise exceptions, you just use raise as usual. That said, I've amended the documentation to at least show how to iterate over the exceptions in the group.

++++++++++++++++++++++++++++++++

.. autoexception:: MultiError

.. attribute:: exceptions

The list of exception objects that this :exc:`MultiError`
represents.

.. automethod:: filter

.. automethod:: catch
:with:

Examples:

Suppose we have a handler function that discards :exc:`ValueError`\s::

def handle_ValueError(exc):
if isinstance(exc, ValueError):
return None
else:
return exc

Then these both raise :exc:`KeyError`::

with MultiError.catch(handle_ValueError):
raise MultiError([KeyError(), ValueError()])

with MultiError.catch(handle_ValueError):
raise MultiError([
ValueError(),
MultiError([KeyError(), ValueError()]),
])

And both of these raise nothing at all::

with MultiError.catch(handle_ValueError):
raise MultiError([ValueError(), ValueError()])

with MultiError.catch(handle_ValueError):
raise MultiError([
MultiError([ValueError(), ValueError()]),
ValueError(),
])

You can also return a new or modified exception, for example::

def convert_ValueError_to_MyCustomError(exc):
if isinstance(exc, ValueError):
# Similar to 'raise MyCustomError from exc'
new_exc = MyCustomError(...)
new_exc.__cause__ = exc
return new_exc
else:
return exc

In the example above, we set ``__cause__`` as a form of explicit
context chaining. :meth:`MultiError.filter` and
:meth:`MultiError.catch` also perform implicit exception chaining – if
you return a new exception object, then the new object's
``__context__`` attribute will automatically be set to the original
exception.

We also monkey patch :class:`traceback.TracebackException` to be able
to handle formatting :exc:`MultiError`\s. This means that anything that
formats exception messages like :mod:`logging` will work out of the
box::

import logging

logging.basicConfig()

try:
raise MultiError([ValueError("foo"), KeyError("bar")])
except:
logging.exception("Oh no!")
raise

Will properly log the inner exceptions:

.. code-block:: none

ERROR:root:Oh no!
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
trio.MultiError: ValueError('foo',), KeyError('bar',)

Details of embedded exception 1:

ValueError: foo

Details of embedded exception 2:

KeyError: 'bar'


.. _task-local-storage:

Task-local storage
Expand Down
8 changes: 1 addition & 7 deletions docs/source/tutorial.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,13 @@ Tutorial
still probably read this, because Trio is different.)

Trio turns Python into a concurrent language. It takes the core
async/await syntax introduced in 3.5, and uses it to add three
async/await syntax introduced in 3.5, and uses it to add two
new pieces of semantics:

- cancel scopes: a generic system for managing timeouts and
cancellation
- nurseries: which let your program do multiple things at the same
time
- MultiErrors: for when multiple things go wrong at once

Of course it also provides a complete suite of APIs for doing
networking, file I/O, using worker threads,
Expand Down Expand Up @@ -57,8 +56,6 @@ Tutorial
and demonstrate start()
then point out that you can just use serve_tcp()

exceptions and MultiError

example: catch-all logging in our echo server

review of the three (or four) core language extensions
Expand Down Expand Up @@ -1149,9 +1146,6 @@ TODO: explain :exc:`Cancelled`
TODO: explain how cancellation is also used when one child raises an
exception

TODO: show an example :exc:`MultiError` traceback and walk through its
structure

TODO: maybe a brief discussion of :exc:`KeyboardInterrupt` handling?

..
Expand Down
2 changes: 1 addition & 1 deletion newsfragments/2063.bugfix.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
Trio now avoids creating cyclic garbage when a `MultiError` is generated and filtered,
Trio now avoids creating cyclic garbage when a ``MultiError`` is generated and filtered,
including invisibly within the cancellation system. This means errors raised
through nurseries and cancel scopes should result in less GC latency.
1 change: 1 addition & 0 deletions newsfragments/2210.removal.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove support for Python 3.6.
4 changes: 4 additions & 0 deletions newsfragments/2211.removal.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
``trio.MultiError`` has been removed in favor of the built-in ``BaseExceptionGroup``
pquentin marked this conversation as resolved.
Show resolved Hide resolved
(and its derivative ``ExceptionGroup``), falling back to the backport_ on Python < 3.11.

.. _backport: https://pypi.org/project/exceptiongroup/
1 change: 1 addition & 0 deletions test-requirements.in
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ trustme # for the ssl tests
pylint # for pylint finding all symbols tests
jedi # for jedi code completion tests
cryptography>=36.0.0 # 35.0.0 is transitive but fails
exceptiongroup # for catch()

# Tools
black; implementation_name == "cpython"
Expand Down
2 changes: 2 additions & 0 deletions test-requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ cryptography==36.0.1
# trustme
decorator==5.1.0
# via ipython
exceptiongroup==1.0.0rc1
# via -r test-requirements.in
flake8==4.0.1
# via -r test-requirements.in
idna==3.3
Expand Down
1 change: 0 additions & 1 deletion trio/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
Cancelled,
BusyResourceError,
ClosedResourceError,
MultiError,
run,
open_nursery,
CancelScope,
Expand Down
2 changes: 0 additions & 2 deletions trio/_core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,6 @@
EndOfChannel,
)

from ._multierror import MultiError

from ._ki import (
enable_ki_protection,
disable_ki_protection,
Expand Down
Loading