Breaking changes to poetry install?

[stephenleonard8]

Issue Summary

With the release of poetry-core 1.1.0b1, some users installing the latest Poetry pre-release (1.2.0b1) in CI environments via the install script can encounter certain issues due to either bug fixes and/or improvements in poetry-core that require related changes that are only available in Poetry master branch.

Resolution

Invalid cosntraint parsing regression has been resolved with https://github.com/python-poetry/poetry-core/releases/tag/1.1.0b2

Mitigation

Users impacted by this issue can either downgrade version of poetry-core #5681 (comment) or use Poetry from git #5681 (comment).

Issue (original)

Here’s how we install poetry:

wget https://install.python-poetry.org/install-poetry.py
python ./install-poetry.py --version 1.2.0b1

Our poetry.lock file contains:

[[package]]
name = "webcolors"
version = "1.11.1"
description = "A library for working with color names and color values formats defined by HTML and CSS."
category = "main"
optional = false
python-versions = ">=3.5,"

Note that the python-versions constraint seems to be malformed (extra trailing comma).

Prior to 2022-05-23 poetry install (on 1.2.0b1, with this malformed version specification) worked fine. This morning we re-installed poetry 1.2.0b1 using the same install procedure above, and we now get an error:

Could not parse version constraint: >=3.5,

Something seems to have changed out from under us even with specifying a version for poetry that is causing an error and our production pipeline to fail.

Additionally, running poetry lock --no-update on today’s re-install of 1.2.0b1 seems to be changing the category of our packages (within poetry.lock) from “main” to “dev”.

What has changed with the poetry installation, and how do we get it back? This is currently breaking production systems for us.

[KyleKaniecki]

I am also running into this issue!

❯ poetry lock
Updating dependencies
Resolving dependencies... (1.8s)

  ParseConstraintError

  Could not parse version constraint: >=3.7,

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/core/semver/helpers.py:144 in parse_single_constraint
      140│         return version
      141│
      142│     from poetry.core.semver.exceptions import ParseConstraintError
      143│
    → 144│     raise ParseConstraintError(f"Could not parse version constraint: {constraint}")
      145│

[KyleKaniecki]

I am also running into this issue!

❯ poetry lock
Updating dependencies
Resolving dependencies... (1.8s)

  ParseConstraintError

  Could not parse version constraint: >=3.7,

  at ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/core/semver/helpers.py:144 in parse_single_constraint
      140│         return version
      141│
      142│     from poetry.core.semver.exceptions import ParseConstraintError
      143│
    → 144│     raise ParseConstraintError(f"Could not parse version constraint: {constraint}")
      145│

Confirmed that it's the trailing comma in the version constraint. Here's a REPL of the regex they use here. Not sure if the best solution here is to fix the regex, or trim the trailing commas off the version constraint before sending it into the regex match. Input welcomed.

>>> from packaging.version import VERSION_PATTERN
>>> import re
>>> BASIC_CONSTRAINT = re.compile(
...     rf"^(?P<op><>|!=|>=?|<=?|==?)?\s*(?P<version>{VERSION_PATTERN}|dev)(\.\*)?$",
...     re.VERBOSE | re.IGNORECASE,
... )
>>> m = BASIC_CONSTRAINT.match(">=3.7,")
>>> type(m)
<class 'NoneType'>
>>> m = BASIC_CONSTRAINT.match(">=3.7")
>>> m
<re.Match object; span=(0, 5), match='>=3.7'>
>>>

[radoering]

For anyone who needs a quick workaround: This seems to be related to the release of poetry-core 1.1.0b1. Downgrading to poetry-core 1.1.0a7 should restore the old behavior.

[gavrieladler]

Hi, we are also experiencing production issues that started yesterday evening (EST). In our pyproject.toml we have:

[tool.poetry.dependencies]
torchvision = { url = "https://download.pytorch.org/whl/lts/1.8/cu111/torchvision-0.9.2%2Bcu111-cp38-cp38-linux_x86_64.whl" }

We have been installing this package on poetry 1.2.0b1 for a few months in CI with no issues, but as of last night we now see:

 Installing dependencies from lock file

  RuntimeError

  [BUG] torchvision (0.9.2+cu111 url) is not satisfied.

  at ~/.local/share/pypoetry/venv/lib/python3.8/site-packages/poetry/mixology/partial_solution.py:204 in satisfier
      200│             # As soon as we have enough assignments to satisfy term, return them.
      201│             if assigned_term.satisfies(term):
      202│                 return assignment
      203│ 
    → 204│         raise RuntimeError(f"[BUG] {term} is not satisfied.")
      205│ 
      206│     def satisfies(self, term: Term) -> bool:
      207│         return self.relation(term) == SetRelation.SUBSET
      208│ 
Error: Process completed with exit code 1.

Each CI run installs poetry fresh via curl -sSL https://install.python-poetry.org/ | python3 - --version 1.2.0b1, so we also believe somehow the release has changed out from under us. How do we get this rolled back? Thanks!

[tfromme]

For anyone who needs a quick workaround: This seems to be related to the release of poetry-core 1.1.0b1. Downgrading to poetry-core 1.1.0a7 should restore the old behavior.

How do you downgrade poetry-core? I've tried the line requires = ["poetry-core==1.1.0a7"] in my pyproject.toml as well as including it as one of the dependencies, but it doesn't seem to be working

[abn]

The issue has to do with Poetry's runtime environment. When you install using the script, a venv is created and poetry is installed via pip. Since poetry 1.2.0b1 says Requires-Dist: poetry-core>=1.1.0a7 the latest beta of poetry core gets installed.

A quick way to downgrade core is to execute this. Note this is only tested for linux environments.

${POETRY_HOME:-${XDG_DATA_HOME:-~/.local/share}}/pypoetry/venv/bin/python -m pip install --force --no-deps poetry-core==1.1.0a7

Ref: https://github.com/python-poetry/install.python-poetry.org/blob/0fab38103455fdb6d630f25ce0d4ccb11f90f9cb/install-poetry.py#L6-L14

[gavrieladler]

The issue has to do with Poetry's runtime environment. When you install using the script, a venv is created and poetry is installed via pip. Since poetry 1.2.0b1 says Requires-Dist: poetry-core>=1.1.0a7 the latest beta gets installed.

A quick way to downgrade core is to execute this. Note this is only tested for linux environments.

${POETRY_HOME:-${XDG_DATA_HOME:-~/.local/share}}/pypoetry/venv/bin/python -m pip install --force --no-deps poetry-core==1.1.0a7

Ref: https://github.com/python-poetry/install.python-poetry.org/blob/0fab38103455fdb6d630f25ce0d4ccb11f90f9cb/install-poetry.py#L6-L14

Hi @abn, thanks for the reply and the explanation.

In general, is there a reason that a specific release of poetry is not tied to a specific release of poetry-core? I just made an internal push for my organization to move our build to poetry, but it's hard to justify using poetry for a production system if we could wake up any morning and find CI is suddenly failing even though no code or dependencies changed on our end.

[radoering]

@gavrieladler Can you please check if you get the [BUG] torchvision (0.9.2+cu111 url) is not satisfied. when using poetry master? If you get the error, please create a separate issue with a complete pyproject.toml to reproduce it. Otherwise that may go under because it seems not related to the original "trailing comma issue" and I'm not sure if it is only with poetry 1.2.0b1 + poetry-core 1.1.0b1 or if it still exists in master.

[abn]

Hi, we are also experiencing production issues that started yesterday evening (EST). In our pyproject.toml we have:

@gavrieladler the downgrade suggested about should mitigate your solver issue. But, I would also recommend that you test out the latest change in master for your project as that might have a fix that works with the latest poetry-core beta.

curl -sSL https://install.python-poetry.org | python3 - --git https://github.com/python-poetry/poetry.git@master

[abn]

In general, is there a reason that a specific release of poetry is not tied to a specific release of poetry-core?

Typically we restrict to a minor version of poetry-core as we do not normally expect things to break between pre-releases. In this case from 1.1.0a7 to 1.1.0b1 of poetry-core. This is usually sufficient restriction. As far as I can tell most of the issues reported were either misfeatures or outright bugs in 1.2.0b1 which were fixed after the release either via changes in Poetry or poetry-core. Note that this is a new minor for core and poetry simultaneously.

This caught us off guard as well. There seems to be a gap in our test matrix that does not ensure that an upcoming release of poetry-core does not break a previously released version of Poetry. We only test the current master for Poetry.

All that said, I suspect in the future, we might restrict core version a bit further, atleast prior to a stable release.

but it's hard to justify using poetry for a production system if we could wake up any morning and find CI is suddenly failing even though no code or dependencies changed on our end.

The frustration is understandable. I suspect part of the issue here is that the 1.2.0 release has been piling on fixes and features forcing the community to use a prerelease in production environments. I am hopeful that once the 1.2.0 release is done we should be able to get new fixes out with better cadence. This should aleaviate the risk of relying on unstable and prone to breaking releases.

For environments that need a higher degree of confidence, I would even recommend that a constraints file be used when installing poetry. Hopefully though, this is merely a transitional issue.

[gavrieladler]

@gavrieladler Can you please check if you get the [BUG] torchvision (0.9.2+cu111 url) is not satisfied. when using poetry master? If you get the error, please create a separate issue with a complete pyproject.toml to reproduce it. Otherwise that may go under because it seems not related to the original "trailing comma issue" and I'm not sure if it is only with poetry 1.2.0b1 + poetry-core 1.1.0b1 or if it still exists in master.

I just confirmed that the issue is resolved when using master.

[gavrieladler]

In general, is there a reason that a specific release of poetry is not tied to a specific release of poetry-core?

Typically we restrict to a minor version of poetry-core as we do not normally expect things to break between pre-releases. In this case from 1.1.0a7 to 1.1.0b1 of poetry-core. This is usually sufficient restriction. As far as I can tell most of the issues reported were either misfeatures or outright bugs in 1.2.0b1 which were fixed after the release either via changes in Poetry or poetry-core. Note that this is a new minor for core and poetry simultaneously.

This caught us off guard as well. There seems to be a gap in our test matrix that does not ensure that an upcoming release of poetry-core does not break a previously released version of Poetry. We only test the current master for Poetry.

All that said, I suspect in the future, we might restrict core version a bit further, atleast prior to a stable release.

but it's hard to justify using poetry for a production system if we could wake up any morning and find CI is suddenly failing even though no code or dependencies changed on our end.

The frustration is understandable. I suspect part of the issue here is that the 1.2.0 release has been piling on fixes and features forcing the community to use a prerelease in production environments. I am hopeful that once the 1.2.0 release is done we should be able to get new fixes out with better cadence. This should aleaviate the risk of relying on unstable and prone to breaking releases.

For environments that need a higher degree of confidence, I would even recommend that a constraints file be used when installing poetry. Hopefully though, this is merely a transitional issue.

Thanks for the detailed explanation, appreciated. Do you have a rough ETA on the 1.2.0 release?

[abn]

Do you have a rough ETA on the 1.2.0 release?

The current plan is to have another beta (~1 week), followed by an rc then stable release.

[radoering]

@stephenleonard8 I created python-poetry/poetry-core#371 for the trailing commas issue. Can you please check if the "main"/"dev" change also occurs when using poetry@master and create a separate issue with an example to reproduce the issue?

[gavrieladler]

@abn @radoering thank you again for being so quick to respond and thorough in your explanations / assistance!

[abn]

Core release 1.1.0b2 is now available.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.