[8.1.x] Revert legacy path removals (#12093)

Co-authored-by: Bruno Oliveira <[email protected]>

changelog/12069.trivial.rst

@@ -0,0 +1,8 @@
+Delayed the deprecation of the following features to ``9.0.0``:
+
+* :ref:`node-ctor-fspath-deprecation`.
+* :ref:`legacy-path-hooks-deprecated`.
+
+It was discovered after ``8.1.0`` was released that the warnings about the impeding removal were not being displayed, so the team decided to revert the removal.
+
+This was the reason for ``8.1.0`` being yanked.

doc/en/conf.py

@@ -200,6 +200,7 @@
     ("py:class", "_tracing.TagTracerSub"),
     ("py:class", "warnings.WarningMessage"),
     # Undocumented type aliases
+    ("py:class", "LEGACY_PATH"),
     ("py:class", "_PluggyPlugin"),
     # TypeVars
     ("py:class", "_pytest._code.code.E"),

doc/en/deprecations.rst

@@ -19,7 +19,45 @@ Below is a complete list of all pytest features which are considered deprecated.
 :class:`~pytest.PytestWarning` or subclasses, which can be filtered using :ref:`standard warning filters <warnings>`.
 
 
-.. _legacy-path-hooks-deprecated:
+.. _node-ctor-fspath-deprecation:
+
+``fspath`` argument for Node constructors replaced with ``pathlib.Path``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. deprecated:: 7.0
+
+In order to support the transition from ``py.path.local`` to :mod:`pathlib`,
+the ``fspath`` argument to :class:`~_pytest.nodes.Node` constructors like
+:func:`pytest.Function.from_parent()` and :func:`pytest.Class.from_parent()`
+is now deprecated.
+
+Plugins which construct nodes should pass the ``path`` argument, of type
+:class:`pathlib.Path`, instead of the ``fspath`` argument.
+
+Plugins which implement custom items and collectors are encouraged to replace
+``fspath`` parameters (``py.path.local``) with ``path`` parameters
+(``pathlib.Path``), and drop any other usage of the ``py`` library if possible.
+
+If possible, plugins with custom items should use :ref:`cooperative
+constructors <uncooperative-constructors-deprecated>` to avoid hardcoding
+arguments they only pass on to the superclass.
+
+.. note::
+    The name of the :class:`~_pytest.nodes.Node` arguments and attributes (the
+    new attribute being ``path``) is **the opposite** of the situation for
+    hooks, :ref:`outlined below <legacy-path-hooks-deprecated>` (the old
+    argument being ``path``).
+
+    This is an unfortunate artifact due to historical reasons, which should be
+    resolved in future versions as we slowly get rid of the :pypi:`py`
+    dependency (see :issue:`9283` for a longer discussion).
+
+Due to the ongoing migration of methods like :meth:`~pytest.Item.reportinfo`
+which still is expected to return a ``py.path.local`` object, nodes still have
+both ``fspath`` (``py.path.local``) and ``path`` (``pathlib.Path``) attributes,
+no matter what argument was used in the constructor. We expect to deprecate the
+``fspath`` attribute in a future release.
+
 
 Configuring hook specs/impls using markers
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -62,6 +100,33 @@ Changed ``hookwrapper`` attributes:
 * ``historic``
 
 
+.. _legacy-path-hooks-deprecated:
+
+``py.path.local`` arguments for hooks replaced with ``pathlib.Path``
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. deprecated:: 7.0
+
+In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
+
+*  :hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
+*  :hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
+*  :hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
+*  :hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
+*  :hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
+
+The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
+
+.. note::
+    The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
+    :ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
+    being ``path``) is **the opposite** of the situation for hooks (the old
+    argument being ``path``).
+
+    This is an unfortunate artifact due to historical reasons, which should be
+    resolved in future versions as we slowly get rid of the :pypi:`py`
+    dependency (see :issue:`9283` for a longer discussion).
+
 Directly constructing internal classes
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -208,73 +273,6 @@ an appropriate period of deprecation has passed.
 
 Some breaking changes which could not be deprecated are also listed.
 
-.. _node-ctor-fspath-deprecation:
-
-``fspath`` argument for Node constructors replaced with ``pathlib.Path``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. deprecated:: 7.0
-
-In order to support the transition from ``py.path.local`` to :mod:`pathlib`,
-the ``fspath`` argument to :class:`~_pytest.nodes.Node` constructors like
-:func:`pytest.Function.from_parent()` and :func:`pytest.Class.from_parent()`
-is now deprecated.
-
-Plugins which construct nodes should pass the ``path`` argument, of type
-:class:`pathlib.Path`, instead of the ``fspath`` argument.
-
-Plugins which implement custom items and collectors are encouraged to replace
-``fspath`` parameters (``py.path.local``) with ``path`` parameters
-(``pathlib.Path``), and drop any other usage of the ``py`` library if possible.
-
-If possible, plugins with custom items should use :ref:`cooperative
-constructors <uncooperative-constructors-deprecated>` to avoid hardcoding
-arguments they only pass on to the superclass.
-
-.. note::
-    The name of the :class:`~_pytest.nodes.Node` arguments and attributes (the
-    new attribute being ``path``) is **the opposite** of the situation for
-    hooks, :ref:`outlined below <legacy-path-hooks-deprecated>` (the old
-    argument being ``path``).
-
-    This is an unfortunate artifact due to historical reasons, which should be
-    resolved in future versions as we slowly get rid of the :pypi:`py`
-    dependency (see :issue:`9283` for a longer discussion).
-
-Due to the ongoing migration of methods like :meth:`~pytest.Item.reportinfo`
-which still is expected to return a ``py.path.local`` object, nodes still have
-both ``fspath`` (``py.path.local``) and ``path`` (``pathlib.Path``) attributes,
-no matter what argument was used in the constructor. We expect to deprecate the
-``fspath`` attribute in a future release.
-
-
-``py.path.local`` arguments for hooks replaced with ``pathlib.Path``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-.. deprecated:: 7.0
-.. versionremoved:: 8.0
-
-In order to support the transition from ``py.path.local`` to :mod:`pathlib`, the following hooks now receive additional arguments:
-
-*  :hook:`pytest_ignore_collect(collection_path: pathlib.Path) <pytest_ignore_collect>` as equivalent to ``path``
-*  :hook:`pytest_collect_file(file_path: pathlib.Path) <pytest_collect_file>` as equivalent to ``path``
-*  :hook:`pytest_pycollect_makemodule(module_path: pathlib.Path) <pytest_pycollect_makemodule>` as equivalent to ``path``
-*  :hook:`pytest_report_header(start_path: pathlib.Path) <pytest_report_header>` as equivalent to ``startdir``
-*  :hook:`pytest_report_collectionfinish(start_path: pathlib.Path) <pytest_report_collectionfinish>` as equivalent to ``startdir``
-
-The accompanying ``py.path.local`` based paths have been deprecated: plugins which manually invoke those hooks should only pass the new ``pathlib.Path`` arguments, and users should change their hook implementations to use the new ``pathlib.Path`` arguments.
-
-.. note::
-    The name of the :class:`~_pytest.nodes.Node` arguments and attributes,
-    :ref:`outlined above <node-ctor-fspath-deprecation>` (the new attribute
-    being ``path``) is **the opposite** of the situation for hooks (the old
-    argument being ``path``).
-
-    This is an unfortunate artifact due to historical reasons, which should be
-    resolved in future versions as we slowly get rid of the :pypi:`py`
-    dependency (see :issue:`9283` for a longer discussion).
-
-
 .. _nose-deprecation:
 
 Support for tests written for nose

src/_pytest/compat.py

@@ -1,5 +1,6 @@
 # mypy: allow-untyped-defs
 """Python version compatibility code."""
+
 from __future__ import annotations
 
 import dataclasses
@@ -16,6 +17,22 @@
 from typing import Final
 from typing import NoReturn
 
+import py
+
+
+#: constant to prepare valuing pylib path replacements/lazy proxies later on
+#  intended for removal in pytest 8.0 or 9.0
+
+# fmt: off
+# intentional space to create a fake difference for the verification
+LEGACY_PATH = py.path. local
+# fmt: on
+
+
+def legacy_path(path: str | os.PathLike[str]) -> LEGACY_PATH:
+    """Internal wrapper to prepare lazy proxies for legacy_path instances"""
+    return LEGACY_PATH(path)
+
 
 # fmt: off
 # Singleton type for NOTSET, as described in:

src/_pytest/config/__init__.py

@@ -38,12 +38,14 @@
 from typing import Union
 import warnings
 
+import pluggy
 from pluggy import HookimplMarker
 from pluggy import HookimplOpts
 from pluggy import HookspecMarker
 from pluggy import HookspecOpts
 from pluggy import PluginManager
 
+from .compat import PathAwareHookProxy
 from .exceptions import PrintHelp as PrintHelp
 from .exceptions import UsageError as UsageError
 from .findpaths import determine_setup
@@ -1068,7 +1070,7 @@ def __init__(
         self._store = self.stash
 
         self.trace = self.pluginmanager.trace.root.get("config")
-        self.hook = self.pluginmanager.hook  # type: ignore[assignment]
+        self.hook: pluggy.HookRelay = PathAwareHookProxy(self.pluginmanager.hook)  # type: ignore[assignment]
         self._inicache: Dict[str, Any] = {}
         self._override_ini: Sequence[str] = ()
         self._opt2dest: Dict[str, str] = {}

src/_pytest/config/compat.py

@@ -0,0 +1,85 @@
+from __future__ import annotations
+
+import functools
+from pathlib import Path
+from typing import Any
+from typing import Mapping
+import warnings
+
+import pluggy
+
+from ..compat import LEGACY_PATH
+from ..compat import legacy_path
+from ..deprecated import HOOK_LEGACY_PATH_ARG
+
+
+# hookname: (Path, LEGACY_PATH)
+imply_paths_hooks: Mapping[str, tuple[str, str]] = {
+    "pytest_ignore_collect": ("collection_path", "path"),
+    "pytest_collect_file": ("file_path", "path"),
+    "pytest_pycollect_makemodule": ("module_path", "path"),
+    "pytest_report_header": ("start_path", "startdir"),
+    "pytest_report_collectionfinish": ("start_path", "startdir"),
+}
+
+
+def _check_path(path: Path, fspath: LEGACY_PATH) -> None:
+    if Path(fspath) != path:
+        raise ValueError(
+            f"Path({fspath!r}) != {path!r}\n"
+            "if both path and fspath are given they need to be equal"
+        )
+
+
+class PathAwareHookProxy:
+    """
+    this helper wraps around hook callers
+    until pluggy supports fixingcalls, this one will do
+
+    it currently doesn't return full hook caller proxies for fixed hooks,
+    this may have to be changed later depending on bugs
+    """
+
+    def __init__(self, hook_relay: pluggy.HookRelay) -> None:
+        self._hook_relay = hook_relay
+
+    def __dir__(self) -> list[str]:
+        return dir(self._hook_relay)
+
+    def __getattr__(self, key: str) -> pluggy.HookCaller:
+        hook: pluggy.HookCaller = getattr(self._hook_relay, key)
+        if key not in imply_paths_hooks:
+            self.__dict__[key] = hook
+            return hook
+        else:
+            path_var, fspath_var = imply_paths_hooks[key]
+
+            @functools.wraps(hook)
+            def fixed_hook(**kw: Any) -> Any:
+                path_value: Path | None = kw.pop(path_var, None)
+                fspath_value: LEGACY_PATH | None = kw.pop(fspath_var, None)
+                if fspath_value is not None:
+                    warnings.warn(
+                        HOOK_LEGACY_PATH_ARG.format(
+                            pylib_path_arg=fspath_var, pathlib_path_arg=path_var
+                        ),
+                        stacklevel=2,
+                    )
+                if path_value is not None:
+                    if fspath_value is not None:
+                        _check_path(path_value, fspath_value)
+                    else:
+                        fspath_value = legacy_path(path_value)
+                else:
+                    assert fspath_value is not None
+                    path_value = Path(fspath_value)
+
+                kw[path_var] = path_value
+                kw[fspath_var] = fspath_value
+                return hook(**kw)
+
+            fixed_hook.name = hook.name  # type: ignore[attr-defined]
+            fixed_hook.spec = hook.spec  # type: ignore[attr-defined]
+            fixed_hook.__name__ = key
+            self.__dict__[key] = fixed_hook
+            return fixed_hook  # type: ignore[return-value]

src/_pytest/deprecated.py

@@ -36,6 +36,21 @@
 PRIVATE = PytestDeprecationWarning("A private pytest class or function was used.")
 
 
+HOOK_LEGACY_PATH_ARG = UnformattedWarning(
+    PytestRemovedIn9Warning,
+    "The ({pylib_path_arg}: py.path.local) argument is deprecated, please use ({pathlib_path_arg}: pathlib.Path)\n"
+    "see https://docs.pytest.org/en/latest/deprecations.html"
+    "#py-path-local-arguments-for-hooks-replaced-with-pathlib-path",
+)
+
+NODE_CTOR_FSPATH_ARG = UnformattedWarning(
+    PytestRemovedIn9Warning,
+    "The (fspath: py.path.local) argument to {node_type_name} is deprecated. "
+    "Please use the (path: pathlib.Path) argument instead.\n"
+    "See https://docs.pytest.org/en/latest/deprecations.html"
+    "#fspath-argument-for-node-constructors-replaced-with-pathlib-path",
+)
+
 HOOK_LEGACY_MARKING = UnformattedWarning(
     PytestDeprecationWarning,
     "The hook{type} {fullname} uses old-style configuration options (marks or attributes).\n"