Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow pre-releases when using compatible release syntax #2685

Open
matthewfeickert opened this issue Mar 27, 2024 · 4 comments
Open

Allow pre-releases when using compatible release syntax #2685

matthewfeickert opened this issue Mar 27, 2024 · 4 comments
Labels
enhancement New feature or improvement to existing functionality

Comments

@matthewfeickert
Copy link

matthewfeickert commented Mar 27, 2024 •
edited
Loading

Description

👋 If you use the compatible release syntax with uv pip install the install can fail if there is a dev release for the target package as that can be selected by the compatible release, but uv will then error and hint that

hint: papermill was requested with a pre-release marker (e.g., papermill>2.5.0,<2.6.dev0), but pre-releases weren't enabled (try: `--prerelease=allow`)

The desired solution would allow for using the compatible release syntax without needing to use --prerelease=allow.

Requested information

  • A minimal code snippet that reproduces the bug.
uv pip install 'papermill~=2.5.0'

or, more fully

$ docker run --rm -ti python:3.12 bin/bash
root@277786dfca2e:/# python -m venv venv && . venv/bin/activate
(venv) root@277786dfca2e:/# pip install uv
Collecting uv
  Downloading uv-0.1.24-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (25 kB)
Downloading uv-0.1.24-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.4/11.4 MB 42.6 MB/s eta 0:00:00
Installing collected packages: uv
Successfully installed uv-0.1.24
(venv) root@277786dfca2e:/# uv pip install --verbose 'papermill~=2.5.0'
INFO Found a virtualenv through VIRTUAL_ENV at: /venv
DEBUG Cached interpreter info for Python 3.12.2, skipping probing: venv/bin/python
DEBUG Using Python 3.12.2 environment at venv/bin/python
DEBUG Using registry request timeout of 300s
DEBUG Solving with target Python version 3.12.2
DEBUG Adding direct dependency: papermill>=2.5.0, <2.6.dev0
DEBUG Found fresh response for: https://pypi.org/simple/papermill/
DEBUG Searching for a compatible version of papermill (>=2.5.0, <2.6.dev0)
DEBUG Selecting: papermill==2.5.0 (papermill-2.5.0-py3-none-any.whl)
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/ef/df/a81912b5c70852db6cce7f4f7671ecfa3ac457b59d20062b06ac6c89b938/papermill-2.5.0-py3-none-any.whl.metadata
DEBUG Adding transitive dependency: click*
DEBUG Adding transitive dependency: pyyaml*
DEBUG Adding transitive dependency: nbformat>=5.1.2
DEBUG Adding transitive dependency: nbclient>=0.2.0
DEBUG Adding transitive dependency: tqdm>=4.32.2
DEBUG Adding transitive dependency: requests*
DEBUG Adding transitive dependency: entrypoints*
DEBUG Adding transitive dependency: tenacity>=5.0.2
DEBUG Adding transitive dependency: aiohttp==3.9.0b0
DEBUG Found fresh response for: https://pypi.org/simple/click/
DEBUG Found fresh response for: https://pypi.org/simple/nbformat/
DEBUG Found fresh response for: https://pypi.org/simple/nbclient/
DEBUG Found fresh response for: https://pypi.org/simple/tenacity/
DEBUG Found fresh response for: https://pypi.org/simple/tqdm/
DEBUG Found fresh response for: https://pypi.org/simple/requests/
DEBUG Found fresh response for: https://pypi.org/simple/entrypoints/
DEBUG Found fresh response for: https://pypi.org/simple/pyyaml/
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/b0/4c/20b6c8b6d7cc17b0481eb49c18f23b76f913ab3e6580a57515cd9727ca43/nbformat-5.10.3-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/66/e8/00517a23d3eeaed0513e718fbc94aab26eaa1758f5690fc8578839791c79/nbclient-0.10.0-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/f4/f1/990741d5bb2487d529d20a433210ffa136a367751e454214013b441c4575/tenacity-8.2.3-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/2a/14/e75e52d521442e2fcc9f1df3c5e456aead034203d4797867980de558ab34/tqdm-4.66.2-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/35/a8/365059bbcd4572cbc41de17fd5b682be5868b218c3c5479071865cab9078/entrypoints-0.4-py3-none-any.whl.metadata
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/b4/33/720548182ffa8344418126017aa1d4ab4aeec9a2275f04ce3f3573d8ace8/PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG Searching for a compatible version of click (*)
DEBUG Selecting: click==8.1.7 (click-8.1.7-py3-none-any.whl)
DEBUG Searching for a compatible version of pyyaml (*)
DEBUG Selecting: pyyaml==6.0.1 (PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl)
DEBUG Searching for a compatible version of nbformat (>=5.1.2)
DEBUG Selecting: nbformat==5.10.3 (nbformat-5.10.3-py3-none-any.whl)
DEBUG Adding transitive dependency: fastjsonschema*
DEBUG Adding transitive dependency: jsonschema>=2.6
DEBUG Adding transitive dependency: jupyter-core*
DEBUG Adding transitive dependency: traitlets>=5.1
DEBUG Searching for a compatible version of nbclient (>=0.2.0)
DEBUG Selecting: nbclient==0.10.0 (nbclient-0.10.0-py3-none-any.whl)
DEBUG Adding transitive dependency: jupyter-client>=6.1.12
DEBUG Adding transitive dependency: jupyter-core>=4.12, <5.0.dev0 | >=5.1.dev0
DEBUG Adding transitive dependency: nbformat>=5.1
DEBUG Adding transitive dependency: traitlets>=5.4
DEBUG Searching for a compatible version of tqdm (>=4.32.2)
DEBUG Selecting: tqdm==4.66.2 (tqdm-4.66.2-py3-none-any.whl)
DEBUG Searching for a compatible version of requests (*)
DEBUG Selecting: requests==2.31.0 (requests-2.31.0-py3-none-any.whl)
DEBUG Adding transitive dependency: charset-normalizer>=2, <4
DEBUG Adding transitive dependency: idna>=2.5, <4
DEBUG Adding transitive dependency: urllib3>=1.21.1, <3
DEBUG Adding transitive dependency: certifi>=2017.4.17
DEBUG Searching for a compatible version of entrypoints (*)
DEBUG Selecting: entrypoints==0.4 (entrypoints-0.4-py3-none-any.whl)
DEBUG Searching for a compatible version of tenacity (>=5.0.2)
DEBUG Selecting: tenacity==8.2.3 (tenacity-8.2.3-py3-none-any.whl)
DEBUG No cache entry for: https://pypi.org/simple/traitlets/
DEBUG No credentials found for: https://pypi.org/simple/traitlets/
DEBUG Found fresh response for: https://pypi.org/simple/fastjsonschema/
DEBUG Found fresh response for: https://pypi.org/simple/jsonschema/
DEBUG Found fresh response for: https://pypi.org/simple/jupyter-core/
DEBUG Found fresh response for: https://pypi.org/simple/idna/
DEBUG Found fresh response for: https://pypi.org/simple/urllib3/
DEBUG Found fresh response for: https://pypi.org/simple/jupyter-client/
DEBUG Found fresh response for: https://pypi.org/simple/certifi/
DEBUG Found fresh response for: https://pypi.org/simple/charset-normalizer/
DEBUG No cache entry for: https://files.pythonhosted.org/packages/9c/b9/79691036d4a8f9857e74d1728b23f34f583b81350a27492edda58d5604e1/fastjsonschema-2.19.1-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/9c/b9/79691036d4a8f9857e74d1728b23f34f583b81350a27492edda58d5604e1/fastjsonschema-2.19.1-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/c2/e7/a82b05cf63a603df6e68d59ae6a68bf5064484a0718ea5033660af4b54a9/idna-3.6-py3-none-any.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/c2/e7/a82b05cf63a603df6e68d59ae6a68bf5064484a0718ea5033660af4b54a9/idna-3.6-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/c2/e7/a82b05cf63a603df6e68d59ae6a68bf5064484a0718ea5033660af4b54a9/idna-3.6-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/c9/fb/108ecd1fe961941959ad0ee4e12ee7b8b1477247f30b1fdfd83ceaf017f0/jupyter_core-5.7.2-py3-none-any.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/c9/fb/108ecd1fe961941959ad0ee4e12ee7b8b1477247f30b1fdfd83ceaf017f0/jupyter_core-5.7.2-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/c9/fb/108ecd1fe961941959ad0ee4e12ee7b8b1477247f30b1fdfd83ceaf017f0/jupyter_core-5.7.2-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/39/9d/b035d024c62c85f2e2d4806a59ca7b8520307f34e0932fbc8cc75fe7b2d9/jsonschema-4.21.1-py3-none-any.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/39/9d/b035d024c62c85f2e2d4806a59ca7b8520307f34e0932fbc8cc75fe7b2d9/jsonschema-4.21.1-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/39/9d/b035d024c62c85f2e2d4806a59ca7b8520307f34e0932fbc8cc75fe7b2d9/jsonschema-4.21.1-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/a2/73/a68704750a7679d0b6d3ad7aa8d4da8e14e151ae82e6fee774e6e0d05ec8/urllib3-2.2.1-py3-none-any.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/a2/73/a68704750a7679d0b6d3ad7aa8d4da8e14e151ae82e6fee774e6e0d05ec8/urllib3-2.2.1-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/a2/73/a68704750a7679d0b6d3ad7aa8d4da8e14e151ae82e6fee774e6e0d05ec8/urllib3-2.2.1-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/75/6d/d7b55b9c1ac802ab066b3e5015e90faab1fffbbd67a2af498ffc6cc81c97/jupyter_client-8.6.1-py3-none-any.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/75/6d/d7b55b9c1ac802ab066b3e5015e90faab1fffbbd67a2af498ffc6cc81c97/jupyter_client-8.6.1-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/75/6d/d7b55b9c1ac802ab066b3e5015e90faab1fffbbd67a2af498ffc6cc81c97/jupyter_client-8.6.1-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/ba/06/a07f096c664aeb9f01624f858c3add0a4e913d6c96257acb4fce61e7de14/certifi-2024.2.2-py3-none-any.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/ba/06/a07f096c664aeb9f01624f858c3add0a4e913d6c96257acb4fce61e7de14/certifi-2024.2.2-py3-none-any.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/ba/06/a07f096c664aeb9f01624f858c3add0a4e913d6c96257acb4fce61e7de14/certifi-2024.2.2-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/ee/fb/14d30eb4956408ee3ae09ad34299131fb383c47df355ddb428a7331cfa1e/charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG No credentials found for already-seen URL: https://files.pythonhosted.org/packages/ee/fb/14d30eb4956408ee3ae09ad34299131fb383c47df355ddb428a7331cfa1e/charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG No credentials found for: https://files.pythonhosted.org/packages/ee/fb/14d30eb4956408ee3ae09ad34299131fb383c47df355ddb428a7331cfa1e/charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG Found fresh response for: https://pypi.org/simple/aiohttp/
DEBUG Searching for a compatible version of aiohttp (==3.9.0b0)
DEBUG No compatible version found for: aiohttp
DEBUG Searching for a compatible version of papermill (>2.5.0, <2.6.dev0)
DEBUG No compatible version found for: papermill
  × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of aiohttp==3.9.0b0 and papermill==2.5.0 depends on aiohttp==3.9.0b0, we can conclude that papermill==2.5.0 cannot be used.
      And because only the following versions of papermill are available:
          papermill<=2.5.0
          papermill>=2.6.dev0
      and you require papermill>=2.5.0, we can conclude that the requirements are unsatisfiable.

      hint: papermill was requested with a pre-release marker (e.g., papermill>2.5.0,<2.6.dev0), but pre-releases weren't enabled (try: `--prerelease=allow`)

      hint: aiohttp was requested with a pre-release marker (e.g., aiohttp==3.9.0b0), but pre-releases weren't enabled (try: `--prerelease=allow`)
(venv) root@277786dfca2e:/#
  • The command you invoked (e.g., uv pip sync requirements.txt), ideally including the --verbose flag.
uv pip install --verbose 'papermill~=2.5.0'
  • The current uv platform: manylinux2014_x86_64
  • The current uv version (uv --version): uv 0.1.24

Related Issues and PRs
This was referenced Mar 28, 2024
Merged
Closed
@charliermarsh charliermarsh added the enhancement New feature or improvement to existing functionality label Apr 7, 2024
@matthewfeickert
Copy link
Author

So to give a more accurate reproducer (feel free to update the Issue title) the problem here is nteract/papermill#785 (comment) where the target papermill==2.5.0 has a pre-release package as a hard dependency. So even though the target is being requested correctly, uv still raises the issue of missing --prerelease=allow, which isn't desired for all other packages that might be installed.

$ docker run --rm -ti python:3.12 /bin/bash
root@a06ed016cede:/# python -m venv .venv && . .venv/bin/activate
(.venv) root@a06ed016cede:/# python -m pip --quiet install uv
(.venv) root@a06ed016cede:/# uv pip install --verbose 'papermill==2.5.0'
INFO Found a virtualenv through VIRTUAL_ENV at: /.venv
DEBUG Probing interpreter info for: /.venv/bin/python
DEBUG Found Python 3.12.3 for: /.venv/bin/python
DEBUG Using Python 3.12.3 environment at .venv/bin/python
DEBUG Trying to lock if free: .venv/.lock
DEBUG Using registry request timeout of 30s
DEBUG Solving with target Python version 3.12.3
DEBUG Adding direct dependency: papermill==2.5.0
DEBUG No cache entry for: https://pypi.org/simple/papermill/
DEBUG Searching for a compatible version of papermill (==2.5.0)
DEBUG Selecting: papermill==2.5.0 (papermill-2.5.0-py3-none-any.whl)
DEBUG No cache entry for: https://files.pythonhosted.org/packages/ef/df/a81912b5c70852db6cce7f4f7671ecfa3ac457b59d20062b06ac6c89b938/papermill-2.5.0-py3-none-any.whl.metadata
DEBUG Adding transitive dependency for papermill==2.5.0: click*
DEBUG Adding transitive dependency for papermill==2.5.0: pyyaml*
DEBUG Adding transitive dependency for papermill==2.5.0: nbformat>=5.1.2
DEBUG Adding transitive dependency for papermill==2.5.0: nbclient>=0.2.0
DEBUG Adding transitive dependency for papermill==2.5.0: tqdm>=4.32.2
DEBUG Adding transitive dependency for papermill==2.5.0: requests*
DEBUG Adding transitive dependency for papermill==2.5.0: entrypoints*
DEBUG Adding transitive dependency for papermill==2.5.0: tenacity>=5.0.2
DEBUG Adding transitive dependency for papermill==2.5.0: aiohttp==3.9.0b0
DEBUG No cache entry for: https://pypi.org/simple/click/
DEBUG No cache entry for: https://pypi.org/simple/pyyaml/
DEBUG No cache entry for: https://pypi.org/simple/nbformat/
DEBUG No cache entry for: https://pypi.org/simple/nbclient/
DEBUG No cache entry for: https://pypi.org/simple/tqdm/
DEBUG No cache entry for: https://pypi.org/simple/tenacity/
DEBUG No cache entry for: https://pypi.org/simple/requests/
DEBUG No cache entry for: https://pypi.org/simple/entrypoints/
DEBUG No cache entry for: https://pypi.org/simple/aiohttp/
DEBUG No cache entry for: https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/b4/33/720548182ffa8344418126017aa1d4ab4aeec9a2275f04ce3f3573d8ace8/PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/a9/82/0340caa499416c78e5d8f5f05947ae4bc3cba53c9f038ab6e9ed964e22f1/nbformat-5.10.4-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/66/e8/00517a23d3eeaed0513e718fbc94aab26eaa1758f5690fc8578839791c79/nbclient-0.10.0-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/2a/14/e75e52d521442e2fcc9f1df3c5e456aead034203d4797867980de558ab34/tqdm-4.66.2-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/35/a8/365059bbcd4572cbc41de17fd5b682be5868b218c3c5479071865cab9078/entrypoints-0.4-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/f4/f1/990741d5bb2487d529d20a433210ffa136a367751e454214013b441c4575/tenacity-8.2.3-py3-none-any.whl.metadata
DEBUG No cache entry for: https://files.pythonhosted.org/packages/70/8e/0e2d847013cb52cd35b38c009bb167a1a26b2ce6cd6965bf26b47bc0bf44/requests-2.31.0-py3-none-any.whl.metadata
DEBUG Searching for a compatible version of aiohttp (==3.9.0b0)
DEBUG No compatible version found for: aiohttp
  × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of aiohttp==3.9.0b0 and papermill==2.5.0 depends on aiohttp==3.9.0b0, we can conclude that papermill==2.5.0 cannot be used.
      And because you require papermill==2.5.0, we can conclude that the requirements are unsatisfiable.

      hint: aiohttp was requested with a pre-release marker (e.g., aiohttp==3.9.0b0), but pre-releases weren't enabled (try: `--prerelease=allow`)
(.venv) root@a06ed016cede:/#

@henryiii
Copy link
Contributor

Allow prereleases in pip only allows ranges that include non-prereleases to include pre-releases; a range that only includes prereleases can still include pre-releases. So numpy>=1.26rc1 would allow numpy to select prereleases, but not all other requirements. Personally, I wish pip was a bit smarter here; numpy>=1.26rc1 should select numpy==1.26rc2, but IMO it should not select numpy==2.0.0rc1 (which it does, which broke a package I had that was originally using this on Python 3.12).

@notatallshaw
Copy link
Contributor

Allow prereleases in pip only allows ranges that include non-prereleases to include pre-releases; a range that only includes prereleases can still include pre-releases. So numpy>=1.26rc1 would allow numpy to select prereleases, but not all other requirements. Personally, I wish pip was a bit smarter here; numpy>=1.26rc1 should select numpy==1.26rc2, but IMO it should not select numpy==2.0.0rc1 (which it does, which broke a package I had that was originally using this on Python 3.12).

This is unrealted to OPs issue and in this case pip, uv, and other package installers, are following the spec. I would rather uv tried to failthfully follow the spec, excluding specific design choices that make it difficult/impossible, rather than making arbitary choices.

The spec should be discussed on the Python packaging discourse: https://discuss.python.org/c/packaging/14

@notatallshaw
Copy link
Contributor

notatallshaw commented Apr 26, 2024 •
edited
Loading

So even though the target is being requested correctly, uv still raises the issue of missing --prerelease=allow, which isn't desired for all other packages that might be installed.

This is a known compatability issue: https://github.com/astral-sh/uv/blob/main/PIP_COMPATIBILITY.md#pre-release-compatibility

You can workaround by adding to a requirement file or constraint file the specific packages you would like to allow for prereleases, e.g. package_name >= 0.0.dev0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or improvement to existing functionality
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@charliermarsh @henryiii @matthewfeickert @notatallshaw