Merge branch 'master' into unstick-virtualenv-windows

CHANGELOG.md

@@ -1,6 +1,19 @@
 # Change Log
 
 
+## [1.4.2] - 2023-04-02
+
+### Changed
+
+- When trying to install wheels with invalid `RECORD` files, Poetry does not fail anymore but only prints a warning.
+  This mitigates an unintended change introduced in Poetry 1.4.1 ([#7694](https://github.com/python-poetry/poetry/pull/7694)).
+
+### Fixed
+
+- Fix an issue where relative git submodule urls were not parsed correctly ([#7017](https://github.com/python-poetry/poetry/pull/7017)).
+- Fix an issue where Poetry could freeze when building a project with a build script if it generated enough output to fill the OS pipe buffer ([#7699](https://github.com/python-poetry/poetry/pull/7699)).
+
+
 ## [1.4.1] - 2023-03-19
 
 ### Fixed
@@ -1786,7 +1799,8 @@ Initial release
 
 
 
-[Unreleased]: https://github.com/python-poetry/poetry/compare/1.4.1...master
+[Unreleased]: https://github.com/python-poetry/poetry/compare/1.4.2...master
+[1.4.2]: https://github.com/python-poetry/poetry/releases/tag/1.4.2
 [1.4.1]: https://github.com/python-poetry/poetry/releases/tag/1.4.1
 [1.4.0]: https://github.com/python-poetry/poetry/releases/tag/1.4.0
 [1.3.2]: https://github.com/python-poetry/poetry/releases/tag/1.3.2

docs/cli.md

@@ -225,6 +225,14 @@ If you want to skip this installation, use the `--no-root` option.
 poetry install --no-root
 ```
 
+Similar to `--no-root` you can use `--no-directory` to skip directory path dependencies:
+
+```bash
+poetry install --no-directory
+```
+
+This is mainly useful for caching in CI or when building Docker images. See the [FAQ entry]({{< relref "faq#poetry-busts-my-docker-cache-because-it-requires-me-to-copy-my-source-files-in-before-installing-3rd-party-dependencies" >}}) for more information on this option.
+
 By default `poetry` does not compile Python source files to bytecode during installation.
 This speeds up the installation process, but the first execution may take a little more
 time because Python then compiles source files to bytecode automatically.
@@ -240,6 +248,7 @@ The `--compile` option has no effect if `installer.modern-installation`
 is set to `false` because the old installer always compiles source files to bytecode.
 {{% /note %}}
 
+
 ### Options
 
 * `--without`: The dependency groups to ignore.
@@ -248,6 +257,7 @@ is set to `false` because the old installer always compiles source files to byte
 * `--only-root`: Install only the root project, exclude all dependencies.
 * `--sync`: Synchronize the environment with the locked packages and the specified groups.
 * `--no-root`: Do not install the root package (your project).
+* `--no-directory`: Skip all directory path dependencies (including transitive ones).
 * `--dry-run`: Output the operations but do not execute anything (implicitly enables --verbose).
 * `--extras (-E)`: Features to install (multiple values allowed).
 * `--all-extras`: Install all extra features (conflicts with --extras).

docs/faq.md

@@ -33,21 +33,54 @@ Once Poetry has cached the releases' information on your machine, the dependency
 will be much faster.
 {{% /note %}}
 
-### Why are unbound version constraints a bad idea?
+### What kind of versioning scheme does Poetry use for itself?
+
+Poetry uses "major.minor.micro" version identifiers as mentioned in
+[PEP 440](https://peps.python.org/pep-0440/#final-releases).
+
+Version bumps are done similar to Python's versioning:
+* A major version bump (incrementing the first number) is only done for breaking changes
+  if a deprecation cycle is not possible and many users have to perform some manual steps
+  to migrate from one version to the next one.
+* A minor version bump (incrementing the second number) may include new features as well
+  as new deprecations and drop features deprecated in an earlier minor release.
+* A micro version bump (incrementing the third number) usually only includes bug fixes.
+  Deprecated features will not be dropped in a micro release.
+
+### Why does Poetry not adhere to semantic versioning?
+
+Because of its large user base, even small changes not considered relevant by most users
+can turn out to be a breaking change for some users in hindsight.
+Sticking to strict [semantic versioning](https://semver.org) and (almost) always bumping
+the major version instead of the minor version does not seem desirable
+since the minor version will not carry any meaning anymore.
+
+### Are unbound version constraints a bad idea?
 
 A version constraint without an upper bound such as `*` or `>=3.4` will allow updates to any future version of the dependency.
 This includes major versions breaking backward compatibility.
 
 Once a release of your package is published, you cannot tweak its dependencies anymore in case a dependency breaks BC
 – you have to do a new release but the previous one stays broken.
+(Users can still work around the broken dependency by restricting it by themselves.)
 
-The only good alternative is to define an upper bound on your constraints,
+To avoid such issues you can define an upper bound on your constraints,
 which you can increase in a new release after testing that your package is compatible
 with the new major version of your dependency.
 
-For example instead of using `>=3.4` you should use `^3.4` which allows all versions `<4.0`.
+For example instead of using `>=3.4` you can use `^3.4` which allows all versions `<4.0`.
 The `^` operator works very well with libraries following [semantic versioning](https://semver.org).
 
+However, when defining an upper bound, users of your package are not able to update
+a dependency beyond the upper bound even if it does not break anything
+and is fully compatible with your package.
+You have to release a new version of your package with an increased upper bound first.
+
+If your package will be used as a library in other packages, it might be better to avoid
+upper bounds and thus unnecessary dependency conflicts (unless you already know for sure
+that the next release of the dependency will break your package).
+If your package will be used as an application, it might be worth to define an upper bound.
+
 ### Is tox supported?
 
 **Yes**. By using the [isolated builds](https://tox.readthedocs.io/en/latest/config.html#conf-isolated_build) `tox` provides,
@@ -156,3 +189,37 @@ This is done so to be compliant with the broader Python ecosystem.
 
 For example, if Poetry builds a distribution for a project that uses a version that is not valid according to
 [PEP 440](https://peps.python.org/pep-0440), third party tools will be unable to parse the version correctly.
+
+
+### Poetry busts my Docker cache because it requires me to COPY my source files in before installing 3rd party dependencies
+
+By default running `poetry install ...` requires you to have your source files present (both the "root" package and any directory path dependencies you might have).
+This interacts poorly with Docker's caching mechanisms because any change to a source file will make any layers (subsequent commands in your Dockerfile) re-run.
+For example, you might have a Dockerfile that looks something like this:
+
+```text
+FROM python
+COPY pyproject.toml poetry.lock .
+COPY src/ ./src
+RUN pip install poetry && poetry install --no-dev
+```
+
+As soon as *any* source file changes, the cache for the `RUN` layer will be invalidated, which forces all 3rd party dependencies (likely the slowest step out of these) to be installed again if you changed any files in `src/`.
+
+To avoid this cache busting you can split this into two steps:
+
+1. Install 3rd party dependencies.
+2. Copy over your source code and install just the source code.
+
+This might look something like this:
+
+```text
+FROM python
+COPY pyproject.toml poetry.lock .
+RUN pip install poetry && poetry install --no-root --no-directory
+COPY src/ ./src
+RUN poetry install --no-dev
+```
+
+The two key options we are using here are `--no-root` (skips installing the project source) and `--no-directory` (skips installing any local directory path dependencies, you can omit this if you don't have any).
+[More information on the options available for `poetry install`]({{< relref "cli#install" >}}).

docs/libraries.md

@@ -19,10 +19,11 @@ This chapter will tell you how to make your library installable through Poetry.
 
 Poetry requires [PEP 440](https://peps.python.org/pep-0440)-compliant versions for all projects.
 
-While Poetry does not enforce any release convention, it does encourage the use of
+While Poetry does not enforce any release convention, it used to encourage the use of
 [semantic versioning](https://semver.org/) within the scope of
-[PEP 440](https://peps.python.org/pep-0440/#semantic-versioning). This has many advantages for the end users
-and allows them to set appropriate [version constraints]({{< relref "dependency-specification#version-constraints" >}}).
+[PEP 440](https://peps.python.org/pep-0440/#semantic-versioning) and supports
+[version constraints]({{< relref "dependency-specification/#caret-requirements" >}})
+that are especially suitable for semver.
 
 {{% note %}}
 

docs/managing-dependencies.md

@@ -163,7 +163,7 @@ poetry install --only docs
 ```
 
 {{% note %}}
-If you only want to install the project's runtime dependencies, you can do so  with the
+If you only want to install the project's runtime dependencies, you can do so with the
 `--only main` notation:
 
 ```bash

src/poetry/config/config.py

@@ -11,12 +11,12 @@
 from typing import Any
 
 from packaging.utils import canonicalize_name
-from poetry.core.toml import TOMLFile
 
 from poetry.config.dict_config_source import DictConfigSource
 from poetry.config.file_config_source import FileConfigSource
 from poetry.locations import CONFIG_DIR
 from poetry.locations import DEFAULT_CACHE_DIR
+from poetry.toml import TOMLFile
 
 
 if TYPE_CHECKING:

src/poetry/config/file_config_source.py

@@ -13,9 +13,10 @@
 if TYPE_CHECKING:
     from collections.abc import Iterator
 
-    from poetry.core.toml.file import TOMLFile
     from tomlkit.toml_document import TOMLDocument
 
+    from poetry.toml.file import TOMLFile
+
 
 class FileConfigSource(ConfigSource):
     def __init__(self, file: TOMLFile, auth_config: bool = False) -> None:

src/poetry/console/commands/check.py

@@ -58,9 +58,8 @@ def validate_classifiers(
         return errors, warnings
 
     def handle(self) -> int:
-        from poetry.core.pyproject.toml import PyProjectTOML
-
         from poetry.factory import Factory
+        from poetry.pyproject.toml import PyProjectTOML
 
         # Load poetry config and display errors, if any
         poetry_file = self.poetry.file.path

src/poetry/console/commands/config.py

@@ -86,11 +86,11 @@ def handle(self) -> int:
         from pathlib import Path
 
         from poetry.core.pyproject.exceptions import PyProjectException
-        from poetry.core.toml.file import TOMLFile
 
         from poetry.config.config import Config
         from poetry.config.file_config_source import FileConfigSource
         from poetry.locations import CONFIG_DIR
+        from poetry.toml.file import TOMLFile
 
         config = Config.create()
         config_file = TOMLFile(CONFIG_DIR / "config.toml")

src/poetry/console/commands/group_command.py

@@ -1,11 +1,13 @@
 from __future__ import annotations
 
+from collections import defaultdict
 from typing import TYPE_CHECKING
 
 from cleo.helpers import option
 from poetry.core.packages.dependency_group import MAIN_GROUP
 
 from poetry.console.commands.command import Command
+from poetry.console.exceptions import GroupNotFound
 
 
 if TYPE_CHECKING:
@@ -78,6 +80,7 @@ def activated_groups(self) -> set[str]:
                 for groups in self.option(key, "")
                 for group in groups.split(",")
             }
+        self._validate_group_options(groups)
 
         for opt, new, group in [
             ("no-dev", "only", MAIN_GROUP),
@@ -107,3 +110,22 @@ def project_with_activated_groups_only(self) -> ProjectPackage:
         return self.poetry.package.with_dependency_groups(
             list(self.activated_groups), only=True
         )
+
+    def _validate_group_options(self, group_options: dict[str, set[str]]) -> None:
+        """
+        Raises en error if it detects that a group is not part of pyproject.toml
+        """
+        invalid_options = defaultdict(set)
+        for opt, groups in group_options.items():
+            for group in groups:
+                if not self.poetry.package.has_dependency_group(group):
+                    invalid_options[group].add(opt)
+        if invalid_options:
+            message_parts = []
+            for group in sorted(invalid_options):
+                opts = ", ".join(
+                    f"<fg=yellow;options=bold>--{opt}</>"
+                    for opt in sorted(invalid_options[group])
+                )
+                message_parts.append(f"{group} (via {opts})")
+            raise GroupNotFound(f"Group(s) not found: {', '.join(message_parts)}")

src/poetry/console/commands/init.py

@@ -73,11 +73,11 @@ def __init__(self) -> None:
     def handle(self) -> int:
         from pathlib import Path
 
-        from poetry.core.pyproject.toml import PyProjectTOML
         from poetry.core.vcs.git import GitConfig
 
         from poetry.config.config import Config
         from poetry.layouts import layout
+        from poetry.pyproject.toml import PyProjectTOML
         from poetry.utils.env import EnvManager
 
         project_path = Path.cwd()

src/poetry/console/commands/install.py

@@ -30,6 +30,16 @@ class InstallCommand(InstallerCommand):
         option(
             "no-root", None, "Do not install the root package (the current project)."
         ),
+        option(
+            "no-directory",
+            None,
+            (
+                "Do not install any directory path dependencies; useful to install"
+                " dependencies without source code, e.g. for caching of Docker layers)"
+            ),
+            flag=True,
+            multiple=False,
+        ),
         option(
             "dry-run",
             None,
@@ -148,6 +158,7 @@ def handle(self) -> int:
             with_synchronization = True
 
         self.installer.only_groups(self.activated_groups)
+        self.installer.skip_directory(self.option("no-directory"))
         self.installer.dry_run(self.option("dry-run"))
         self.installer.requires_synchronization(with_synchronization)
         self.installer.executor.enable_bytecode_compilation(self.option("compile"))

src/poetry/console/commands/run.py

@@ -63,6 +63,9 @@ def run_script(self, script: str | dict[str, str], args: list[str]) -> int:
             if script_path.exists():
                 args = [str(script_path), *args[1:]]
                 break
+        else:
+            # If we reach this point, the script is not installed
+            self._warning_not_installed_script(args[0])
 
         if isinstance(script, dict):
             script = script["callable"]
@@ -81,3 +84,14 @@ def run_script(self, script: str | dict[str, str], args: list[str]) -> int:
         ]
 
         return self.env.execute(*cmd)
+
+    def _warning_not_installed_script(self, script: str) -> None:
+        message = f"""\
+Warning: '{script}' is an entry point defined in pyproject.toml, but it's not \
+installed as a script. You may get improper `sys.argv[0]`.
+
+The support to run uninstalled scripts will be removed in a future release.
+
+Run `poetry install` to resolve and get rid of this message.
+"""
+        self.line_error(message, style="warning")

src/poetry/console/commands/self/self_command.py

@@ -5,11 +5,11 @@
 
 from poetry.core.packages.dependency import Dependency
 from poetry.core.packages.project_package import ProjectPackage
-from poetry.core.pyproject.toml import PyProjectTOML
 
 from poetry.__version__ import __version__
 from poetry.console.commands.installer_command import InstallerCommand
 from poetry.factory import Factory
+from poetry.pyproject.toml import PyProjectTOML
 from poetry.utils.env import EnvManager
 from poetry.utils.env import SystemEnv
 from poetry.utils.helpers import directory

src/poetry/console/exceptions.py

@@ -5,3 +5,7 @@
 
 class PoetryConsoleError(CleoError):
     pass
+
+
+class GroupNotFound(PoetryConsoleError):
+    pass