diff --git a/doc/en/changelog.rst b/doc/en/changelog.rst index 057659c756..5d2c36b1ae 100644 --- a/doc/en/changelog.rst +++ b/doc/en/changelog.rst @@ -34,67 +34,74 @@ pytest 8.0.0rc1 (2023-12-30) Breaking Changes ---------------- -- `#11137 `_: :class:`pytest.Package` is no longer a :class:`pytest.Module` or :class:`pytest.File`. +Old Deprecations Are Now Errors +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - The ``Package`` collector node designates a Python package, that is, a directory with an `__init__.py` file. - Previously ``Package`` was a subtype of ``pytest.Module`` (which represents a single Python module), - the module being the `__init__.py` file. - This has been deemed a design mistake (see :issue:`11137` and :issue:`7777` for details). +- `#7363 `_: **PytestRemovedIn8Warning deprecation warnings are now errors by default.** - The ``path`` property of ``Package`` nodes now points to the package directory instead of the ``__init__.py`` file. + Following our plan to remove deprecated features with as little disruption as + possible, all warnings of type ``PytestRemovedIn8Warning`` now generate errors + instead of warning messages by default. - Note that a ``Module`` node for ``__init__.py`` (which is not a ``Package``) may still exist, - if it is picked up during collection (e.g. if you configured :confval:`python_files` to include ``__init__.py`` files). + **The affected features will be effectively removed in pytest 8.1**, so please consult the + :ref:`deprecations` section in the docs for directions on how to update existing code. + In the pytest ``8.0.X`` series, it is possible to change the errors back into warnings as a + stopgap measure by adding this to your ``pytest.ini`` file: -- `#11151 `_: Dropped support for Python 3.7, which `reached end-of-life on 2023-06-27 `__. + .. code-block:: ini + [pytest] + filterwarnings = + ignore::pytest.PytestRemovedIn8Warning -- `#11282 `_: Sanitized the handling of the ``default`` parameter when defining configuration options. + But this will stop working when pytest ``8.1`` is released. - Previously if ``default`` was not supplied for :meth:`parser.addini ` and the configuration option value was not defined in a test session, then calls to :func:`config.getini ` returned an *empty list* or an *empty string* depending on whether ``type`` was supplied or not respectively, which is clearly incorrect. Also, ``None`` was not honored even if ``default=None`` was used explicitly while defining the option. + **If you have concerns** about the removal of a specific feature, please add a + comment to :issue:`7363`. - Now the behavior of :meth:`parser.addini ` is as follows: - * If ``default`` is NOT passed but ``type`` is provided, then a type-specific default will be returned. For example ``type=bool`` will return ``False``, ``type=str`` will return ``""``, etc. - * If ``default=None`` is passed and the option is not defined in a test session, then ``None`` will be returned, regardless of the ``type``. - * If neither ``default`` nor ``type`` are provided, assume ``type=str`` and return ``""`` as default (this is as per previous behavior). +Version Compatibility +^^^^^^^^^^^^^^^^^^^^^ - The team decided to not introduce a deprecation period for this change, as doing so would be complicated both in terms of communicating this to the community as well as implementing it, and also because the team believes this change should not break existing plugins except in rare cases. +- `#11151 `_: Dropped support for Python 3.7, which `reached end-of-life on 2023-06-27 `__. -- `#11667 `_: pytest's ``setup.py`` file is removed. - If you relied on this file, e.g. to install pytest using ``setup.py install``, - please see `Why you shouldn't invoke setup.py directly `_ for alternatives. +- ``pluggy>=1.3.0`` is now required. -- `#11676 `_: The classes :class:`~_pytest.nodes.Node`, :class:`~pytest.Collector`, :class:`~pytest.Item`, :class:`~pytest.File`, :class:`~_pytest.nodes.FSCollector` are now marked abstract (see :mod:`abc`). +Collection Changes +^^^^^^^^^^^^^^^^^^ - We do not expect this change to affect users and plugin authors, it will only cause errors when the code is already wrong or problematic. +In this version we've made several breaking changes to pytest's collection phase, +particularly around how filesystem directories and Python packages are collected, +fixing deficiencies and allowing for cleanups and improvements to pytest's internals. +A deprecation period for these changes was not possible. -- `#7363 `_: **PytestRemovedIn8Warning deprecation warnings are now errors by default.** +- `#7777 `_: Files and directories are now collected in alphabetical order jointly, unless changed by a plugin. + Previously, files were collected before directories. + See below for an example. - Following our plan to remove deprecated features with as little disruption as - possible, all warnings of type ``PytestRemovedIn8Warning`` now generate errors - instead of warning messages by default. - **The affected features will be effectively removed in pytest 8.1**, so please consult the - :ref:`deprecations` section in the docs for directions on how to update existing code. +- `#8976 `_: Running `pytest pkg/__init__.py` now collects the `pkg/__init__.py` file (module) only. + Previously, it collected the entire `pkg` package, including other test files in the directory, but excluding tests in the `__init__.py` file itself + (unless :confval:`python_files` was changed to allow `__init__.py` file). - In the pytest ``8.0.X`` series, it is possible to change the errors back into warnings as a - stopgap measure by adding this to your ``pytest.ini`` file: + To collect the entire package, specify just the directory: `pytest pkg`. - .. code-block:: ini - [pytest] - filterwarnings = - ignore::pytest.PytestRemovedIn8Warning +- `#11137 `_: :class:`pytest.Package` is no longer a :class:`pytest.Module` or :class:`pytest.File`. - But this will stop working when pytest ``8.1`` is released. + The ``Package`` collector node designates a Python package, that is, a directory with an `__init__.py` file. + Previously ``Package`` was a subtype of ``pytest.Module`` (which represents a single Python module), + the module being the `__init__.py` file. + This has been deemed a design mistake (see :issue:`11137` and :issue:`7777` for details). - **If you have concerns** about the removal of a specific feature, please add a - comment to :issue:`7363`. + The ``path`` property of ``Package`` nodes now points to the package directory instead of the ``__init__.py`` file. + + Note that a ``Module`` node for ``__init__.py`` (which is not a ``Package``) may still exist, + if it is picked up during collection (e.g. if you configured :confval:`python_files` to include ``__init__.py`` files). - `#7777 `_: Added a new :class:`pytest.Directory` base collection node, which all collector nodes for filesystem directories are expected to subclass. @@ -105,20 +112,12 @@ Breaking Changes i.e. contains an ``__init__.py`` file. :class:`pytest.Package` now only collects files in its own directory; previously it collected recursively. - Sub-directories are collected as sub-collector nodes, thus creating a collection tree which mirrors the filesystem hierarchy. + Sub-directories are collected as their own collector nodes, which then collect themselves, thus creating a collection tree which mirrors the filesystem hierarchy. Added a new :class:`pytest.Dir` concrete collection node, a subclass of :class:`pytest.Directory`. This node represents a filesystem directory, which is not a :class:`pytest.Package`, - i.e. does not contain an ``__init__.py`` file. - Similarly to ``Package``, it only collects the files in its own directory, - while collecting sub-directories as sub-collector nodes. - - Added a new hook :hook:`pytest_collect_directory`, - which is called by filesystem-traversing collector nodes, - such as :class:`pytest.Session`, :class:`pytest.Dir` and :class:`pytest.Package`, - to create a collector node for a sub-directory. - It is expected to return a subclass of :class:`pytest.Directory`. - This hook allows plugins to :ref:`customize the collection of directories `. + that is, does not contain an ``__init__.py`` file. + Similarly to ``Package``, it only collects the files in its own directory. :class:`pytest.Session` now only collects the initial arguments, without recursing into directories. This work is now done by the :func:`recursive expansion process ` of directory collector nodes. @@ -126,9 +125,6 @@ Breaking Changes :attr:`session.name ` is now ``""``; previously it was the rootdir directory name. This matches :attr:`session.nodeid <_pytest.nodes.Node.nodeid>` which has always been `""`. - Files and directories are now collected in alphabetical order jointly, unless changed by a plugin. - Previously, files were collected before directories. - The collection tree now contains directories/packages up to the :ref:`rootdir `, for initial arguments that are found within the rootdir. For files outside the rootdir, only the immediate directory/package is collected -- @@ -189,18 +185,40 @@ Breaking Changes Code/plugins which rely on a specific shape of the collection tree might need to update. -- `#8976 `_: Running `pytest pkg/__init__.py` now collects the `pkg/__init__.py` file (module) only. - Previously, it collected the entire `pkg` package, including other test files in the directory, but excluding tests in the `__init__.py` file itself - (unless :confval:`python_files` was changed to allow `__init__.py` file). +- `#11676 `_: The classes :class:`~_pytest.nodes.Node`, :class:`~pytest.Collector`, :class:`~pytest.Item`, :class:`~pytest.File`, :class:`~_pytest.nodes.FSCollector` are now marked abstract (see :mod:`abc`). - To collect the entire package, specify just the directory: `pytest pkg`. + We do not expect this change to affect users and plugin authors, it will only cause errors when the code is already wrong or problematic. + + +Other breaking changes +^^^^^^^^^^^^^^^^^^^^^^ + +These are breaking changes where deprecation was not possible. + + +- `#11282 `_: Sanitized the handling of the ``default`` parameter when defining configuration options. + + Previously if ``default`` was not supplied for :meth:`parser.addini ` and the configuration option value was not defined in a test session, then calls to :func:`config.getini ` returned an *empty list* or an *empty string* depending on whether ``type`` was supplied or not respectively, which is clearly incorrect. Also, ``None`` was not honored even if ``default=None`` was used explicitly while defining the option. + + Now the behavior of :meth:`parser.addini ` is as follows: + + * If ``default`` is NOT passed but ``type`` is provided, then a type-specific default will be returned. For example ``type=bool`` will return ``False``, ``type=str`` will return ``""``, etc. + * If ``default=None`` is passed and the option is not defined in a test session, then ``None`` will be returned, regardless of the ``type``. + * If neither ``default`` nor ``type`` are provided, assume ``type=str`` and return ``""`` as default (this is as per previous behavior). + + The team decided to not introduce a deprecation period for this change, as doing so would be complicated both in terms of communicating this to the community as well as implementing it, and also because the team believes this change should not break existing plugins except in rare cases. + + +- `#11667 `_: pytest's ``setup.py`` file is removed. + If you relied on this file, e.g. to install pytest using ``setup.py install``, + please see `Why you shouldn't invoke setup.py directly `_ for alternatives. - `#9288 `_: :func:`~pytest.warns` now re-emits unmatched warnings when the context closes -- previously it would consume all warnings, hiding those that were not matched by the function. - While this is a new feature, we decided to announce this as a breaking change + While this is a new feature, we announce it as a breaking change because many test suites are configured to error-out on warnings, and will therefore fail on the newly-re-emitted warnings. @@ -209,80 +227,112 @@ Breaking Changes Deprecations ------------ -- `#10465 `_: Test functions returning a value other than None will now issue a :class:`pytest.PytestWarning` instead of :class:`pytest.PytestRemovedIn8Warning`, meaning this will stay a warning instead of becoming an error in the future. +- `#10465 `_: Test functions returning a value other than ``None`` will now issue a :class:`pytest.PytestWarning` instead of :class:`pytest.PytestRemovedIn8Warning`, meaning this will stay a warning instead of becoming an error in the future. - `#3664 `_: Applying a mark to a fixture function now issues a warning: marks in fixtures never had any effect, but it is a common user error to apply a mark to a fixture (for example ``usefixtures``) and expect it to work. - This will become an error in the future. + This will become an error in pytest 9.0. -Features --------- +Features and Improvements +------------------------- + +Improved Diffs +^^^^^^^^^^^^^^ + +These changes improve the diffs that pytest prints when an assertion fails. +Note that syntax highlighting requires the ``pygments`` package. + -- `#10441 `_: Added :func:`ExceptionInfo.group_contains() `, an assertion - helper that tests if an `ExceptionGroup` contains a matching exception. +- `#11520 `_: The very verbose (``-vv``) diff output is now colored as a diff instead of a big chunk of red. + + Python code in error reports is now syntax-highlighted as Python. + + The sections in the error reports are now better separated. + + +- `#1531 `_: The very verbose diff (``-vv``) for every standard library container type is improved. The indentation is now consistent and the markers are on their own separate lines, which should reduce the diffs shown to users. + + Previously, the standard Python pretty printer was used to generate the output, which puts opening and closing + markers on the same line as the first/last entry, in addition to not having consistent indentation. - `#10617 `_: Added more comprehensive set assertion rewrites for comparisons other than equality ``==``, with the following operations now providing better failure messages: ``!=``, ``<=``, ``>=``, ``<``, and ``>``. +Separate Control For Assertion Verbosity +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + - `#11387 `_: Added the new :confval:`verbosity_assertions` configuration option for fine-grained control of failed assertions verbosity. + If you've ever wished that pytest always show you full diffs, but without making everything else verbose, this is for you. + See :ref:`Fine-grained verbosity ` for more details. For plugin authors, :attr:`config.get_verbosity ` can be used to retrieve the verbosity level for a specific verbosity type. -- `#11610 `_: Added :func:`LogCaptureFixture.filtering() ` context manager that - adds a given :class:`logging.Filter` object to the caplog fixture. +Additional Support For Exception Groups and ``__notes__`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +These changes improve pytest's support for exception groups. -- `#7469 `_: :class:`~pytest.FixtureDef` is now exported as ``pytest.FixtureDef`` for typing purposes. +- `#10441 `_: Added :func:`ExceptionInfo.group_contains() `, an assertion helper that tests if an :class:`ExceptionGroup` contains a matching exception. + See :ref:`assert-matching-exception-groups` for an example. -Improvements ------------- -- `#11122 `_: ``pluggy>=1.2.0`` is now required. +- `#11227 `_: Allow :func:`pytest.raises` ``match`` argument to match against `PEP-678 ` ``__notes__``. - pytest now uses "new-style" hook wrappers internally, available since pluggy 1.2.0. - See `pluggy's 1.2.0 changelog `_ and the :ref:`updated docs ` for details. - Plugins which want to use new-style wrappers can do so if they require this version of pytest or later. +Custom Directory collectors +^^^^^^^^^^^^^^^^^^^^^^^^^^^ +- `#7777 `_: Added a new hook :hook:`pytest_collect_directory`, + which is called by filesystem-traversing collector nodes, + such as :class:`pytest.Session`, :class:`pytest.Dir` and :class:`pytest.Package`, + to create a collector node for a sub-directory. + It is expected to return a subclass of :class:`pytest.Directory`. + This hook allows plugins to :ref:`customize the collection of directories `. -- `#11216 `_: If a test is skipped from inside an :ref:`xunit setup fixture `, the test summary now shows the test location instead of the fixture location. +"New-style" Hook Wrappers +^^^^^^^^^^^^^^^^^^^^^^^^^ -- `#11227 `_: Allow :func:`pytest.raises` ``match`` argument to match against `PEP-678 ` ``__notes__``. +- `#11122 `_: pytest now uses "new-style" hook wrappers internally, available since pluggy 1.2.0. + See `pluggy's 1.2.0 changelog `_ and the :ref:`updated docs ` for details. + Plugins which want to use new-style wrappers can do so if they require ``pytest>=8``. -- `#11314 `_: Logging to a file using the ``--log-file`` option will use ``--log-level``, ``--log-format`` and ``--log-date-format`` as fallback - if ``--log-file-level``, ``--log-file-format`` and ``--log-file-date-format`` are not provided respectively. +Other Improvements +^^^^^^^^^^^^^^^^^^ -- `#11447 `_: :func:`pytest.deprecated_call` now also considers warnings of type :class:`FutureWarning`. +- `#11216 `_: If a test is skipped from inside an :ref:`xunit setup fixture `, the test summary now shows the test location instead of the fixture location. -- `#11520 `_: Improved very verbose diff output to color it as a diff instead of only red. +- `#11314 `_: Logging to a file using the ``--log-file`` option will use ``--log-level``, ``--log-format`` and ``--log-date-format`` as fallback + if ``--log-file-level``, ``--log-file-format`` and ``--log-file-date-format`` are not provided respectively. - Improved the error reporting to better separate each section. - Improved the error reporting to syntax-highlight Python code when Pygments is available. +- `#11610 `_: Added the :func:`LogCaptureFixture.filtering() ` context manager which + adds a given :class:`logging.Filter` object to the :fixture:`caplog` fixture. + + +- `#11447 `_: :func:`pytest.deprecated_call` now also considers warnings of type :class:`FutureWarning`. - `#11600 `_: Improved the documentation and type signature for :func:`pytest.mark.xfail `'s ``condition`` param to use ``False`` as the default value. -- `#1531 `_: Improved the very verbose diff for every standard library container types: the indentation is now consistent and the markers are on their own separate lines, which should reduce the diffs shown to users. +- `#7469 `_: :class:`~pytest.FixtureDef` is now exported as ``pytest.FixtureDef`` for typing purposes. - Previously, the default python pretty printer was used to generate the output, which puts opening and closing - markers on the same line as the first/last entry, in addition to not having consistent indentation. +- `#11353 `_: Added typing to :class:`~pytest.PytestPluginManager`. Bug Fixes @@ -308,7 +358,7 @@ Bug Fixes readability adds an underscore when non-unique ids end in a number. -- `#11563 `_: Fixed crash when using an empty string for the same parametrized value more than once. +- `#11563 `_: Fixed a crash when using an empty string for the same parametrized value more than once. - `#11572 `_: Handle an edge case where :data:`sys.stderr` and :data:`sys.__stderr__` might already be closed when :ref:`faulthandler` is tearing down. @@ -317,7 +367,7 @@ Bug Fixes - `#11712 `_: Fixed handling ``NO_COLOR`` and ``FORCE_COLOR`` to ignore an empty value. -- `#7966 `_: Removes unhelpful error message from assertion rewrite mechanism when exceptions raised in __iter__ methods, and instead treats them as un-iterable. +- `#7966 `_: Removed an unhelpful error message from the assertion rewrite mechanism when exceptions are raised in ``__iter__`` methods. They are now treated as un-iterable instead. - `#9036 `_: ``pytest.warns`` and similar functions now capture warnings when an exception is raised inside a ``with`` block. @@ -330,12 +380,10 @@ Improved Documentation - `#11011 `_: Added a warning about modifying the root logger during tests when using ``caplog``. -- `#11065 `_: Use pytestconfig instead of request.config in cache example - - to be consistent with the API documentation. +- `#11065 `_: Use ``pytestconfig`` instead of ``request.config`` in cache example to be consistent with the API documentation. -- `#11091 `_: Updated documentation and tests to refer to hyphonated options: replaced ``--junitxml`` with ``--junit-xml`` and ``--collectonly`` with ``--collect-only``. +- `#11091 `_: Updated documentation and tests to refer to hyphenated options: replaced ``--junitxml`` with ``--junit-xml`` and ``--collectonly`` with ``--collect-only``. @@ -362,9 +410,6 @@ Trivial/Internal Changes The previous spelling ``INCOVATION_DIR`` remains as an alias. -- `#11353 `_: pluggy>=1.3.0 is now required. This adds typing to :class:`~pytest.PytestPluginManager`. - - - `#11638 `_: Fixed the selftests to pass correctly if ``FORCE_COLOR``, ``NO_COLOR`` or ``PY_COLORS`` is set in the calling environment. diff --git a/doc/en/getting-started.rst b/doc/en/getting-started.rst index 82704bfa03..0f74a8ecf0 100644 --- a/doc/en/getting-started.rst +++ b/doc/en/getting-started.rst @@ -98,7 +98,7 @@ Use the :ref:`raises ` helper to assert that some code raises an e f() You can also use the context provided by :ref:`raises ` to -assert that an expected exception is part of a raised ``ExceptionGroup``: +assert that an expected exception is part of a raised :class:`ExceptionGroup`: .. code-block:: python diff --git a/doc/en/how-to/assert.rst b/doc/en/how-to/assert.rst index 2ecc716b25..5c7d125feb 100644 --- a/doc/en/how-to/assert.rst +++ b/doc/en/how-to/assert.rst @@ -143,11 +143,13 @@ Notes: * The ``match`` parameter also matches against `PEP-678 `__ ``__notes__``. +.. _`assert-matching-exception-groups`: + Matching exception groups ~~~~~~~~~~~~~~~~~~~~~~~~~ You can also use the :func:`excinfo.group_contains() ` -method to test for exceptions returned as part of an ``ExceptionGroup``: +method to test for exceptions returned as part of an :class:`ExceptionGroup`: .. code-block:: python