Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

solver doesn't find correct version of package with a number at the end of the name #6996

Closed
samthurston opened this issue Nov 8, 2022 · 11 comments · Fixed by #9189
Closed
Labels
area/solver Related to the dependency resolver kind/bug Something isn't working as expected status/confirmed Issue is reproduced and confirmed version/1.2.2

Comments

@samthurston
Copy link

  • [ x] I am on the latest stable Poetry version, installed using a recommended method.
  • [ x] I have searched the issues of this repo and believe that this is not a duplicate.
  • [ x] I have consulted the FAQ and blog for any relevant entries or release notes.
  • [x ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When installing a package with an integer at the end of the package name, the solver is not finding the correct package. The specific package I am having trouble with is flask-restful-swagger-2. latest at the time of writing is 0.35

adding unbounded:

$ poetry add flask-restful-swagger-2
Using version ^0.20 for flask-restful-swagger-2

Updating dependencies
Resolving dependencies... (548.9s)

Writing lock file

Package operations: 13 installs, 0 updates, 0 removals

  • Installing markupsafe (2.1.1)
  • Installing zipp (3.10.0)
  • Installing click (8.1.3)
  • Installing importlib-metadata (5.0.0)
  • Installing itsdangerous (2.1.2)
  • Installing jinja2 (3.1.2)
  • Installing werkzeug (2.2.2)
  • Installing aniso8601 (9.0.1)
  • Installing flask (2.2.2)
  • Installing pytz (2022.6)
  • Installing six (1.16.0)
  • Installing flask-restful (0.3.9)
  • Installing flask-restful-swagger-2 (0.20)

adding the explicit latest version bound:

$ poetry add [email protected]
Could not find a matching version of package flask-restful-swagger-2

manually updating pyproject.toml to flask-restful-swagger-2 = "^0.35" and rerunning install:

Installing dependencies from lock file
Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.

Because poetry-test depends on flask-restful-swagger-2 (^0.35) which doesn't match any versions, version solving failed.

run with -vvv

$ poetry install -vvv
Loading configuration file /home/sam/.config/pypoetry/config.toml
Using virtualenv: /home/sam/.cache/pypoetry/virtualenvs/poetry-test-yR2o0HPF-py3.8
Project environment contains an empty path in sys_path, ignoring.
Installing dependencies from lock file
Warning: poetry.lock is not consistent with pyproject.toml. You may be getting improper dependencies. Run `poetry lock [--no-update]` to fix it.

Finding the necessary packages for the current system
Source (poetry-repo): 0 packages found for flask-restful-swagger-2 0.35
Falling back to installed packages to discover metadata for flask-restful-swagger-2
Found 0 compatible packages for flask-restful-swagger-2

  Stack trace:

  4  ~/.local/lib/python3.8/site-packages/poetry/puzzle/solver.py:151 in _solve
      149│ 
      150│         try:
    → 151│             result = resolve_version(
      152│                 self._package, self._provider, locked=locked, use_latest=use_latest
      153│             )

  3  ~/.local/lib/python3.8/site-packages/poetry/mixology/__init__.py:24 in resolve_version
       22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       23│ 
    →  24│     return solver.solve()
       25│ 

  2  ~/.local/lib/python3.8/site-packages/poetry/mixology/version_solver.py:126 in solve
      124│             next: str | None = self._root.name
      125│             while next is not None:
    → 126│                 self._propagate(next)
      127│                 next = self._choose_package_version()
      128│ 

  1  ~/.local/lib/python3.8/site-packages/poetry/mixology/version_solver.py:165 in _propagate
      163│                     # where that incompatibility will allow us to derive new assignments
      164│                     # that avoid the conflict.
    → 165│                     root_cause = self._resolve_conflict(incompatibility)
      166│ 
      167│                     # Back jumping erases all the assignments we did at the previous

  SolveFailure

  Because poetry-test depends on flask-restful-swagger-2 (0.35) which doesn't match any versions, version solving failed.

  at ~/.local/lib/python3.8/site-packages/poetry/mixology/version_solver.py:364 in _resolve_conflict
      360│             )
      361│             self._log(f'! which is caused by "{most_recent_satisfier.cause}"')
      362│             self._log(f"! thus: {incompatibility}")
      363│ 
    → 364│         raise SolveFailure(incompatibility)
      365│ 
      366│     def _choose_package_version(self) -> str | None:
      367│         """
      368│         Tries to select a version of a required package.

The following error occurred when trying to handle this error:


  Stack trace:

  11  ~/.local/lib/python3.8/site-packages/cleo/application.py:329 in run
       327│ 
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

  10  ~/.local/lib/python3.8/site-packages/poetry/console/application.py:185 in _run
       183│         self._load_plugins(io)
       184│ 
     → 185│         exit_code: int = super()._run(io)
       186│         return exit_code
       187│ 

   9  ~/.local/lib/python3.8/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│ 
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│ 

   8  ~/.local/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
       463│ 
       464│         if error is not None:
     → 465│             raise error
       466│ 
       467│         return event.exit_code

   7  ~/.local/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
       447│ 
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   6  ~/.local/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   5  ~/.local/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
        81│ 
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

   4  ~/.local/lib/python3.8/site-packages/poetry/console/commands/install.py:146 in handle
       144│         self.installer.verbose(self.io.is_verbose())
       145│ 
     → 146│         return_code = self.installer.run()
       147│ 
       148│         if return_code != 0:

   3  ~/.local/lib/python3.8/site-packages/poetry/installation/installer.py:115 in run
       113│             self._execute_operations = False
       114│ 
     → 115│         return self._do_install()
       116│ 
       117│     def dry_run(self, dry_run: bool = True) -> Installer:

   2  ~/.local/lib/python3.8/site-packages/poetry/installation/installer.py:321 in _do_install
       319│ 
       320│         with solver.use_environment(self._env):
     → 321│             ops = solver.solve(use_latest=self._whitelist).calculate_operations(
       322│                 with_uninstalls=self._requires_synchronization,
       323│                 synchronize=self._requires_synchronization,

   1  ~/.local/lib/python3.8/site-packages/poetry/puzzle/solver.py:73 in solve
        71│         with self._provider.progress():
        72│             start = time.time()
     →  73│             packages, depths = self._solve(use_latest=use_latest)
        74│             end = time.time()
        75│ 

  SolverProblemError

  Because poetry-test depends on flask-restful-swagger-2 (0.35) which doesn't match any versions, version solving failed.

  at ~/.local/lib/python3.8/site-packages/poetry/puzzle/solver.py:159 in _solve
      155│             packages = result.packages
      156│         except OverrideNeeded as e:
      157│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      158│         except SolveFailure as e:
    → 159│             raise SolverProblemError(e)
      160│ 
      161│         combined_nodes = depth_first_search(PackageNode(self._package, packages))
      162│         results = dict(aggregate_package_nodes(nodes) for nodes in combined_nodes)
      163│

Incidentally, ^0.20 is the latest of the package flask-restful-swagger (package name doesn't end in -2)

@samthurston samthurston added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Nov 8, 2022
@samthurston
Copy link
Author

following directions from the extended exception output:

$ poetry lock --no-update
Resolving dependencies... (0.1s)

Because poetry-test depends on flask-restful-swagger-2 (0.35) which doesn't match any versions, version solving failed.

@neersighted
Copy link
Member

This is still missing verbose output as requested by the issue template.

root@f11cc50a82f9:/foo# poetry add -vvv flask-restful-swagger-2==0.35
Using virtualenv: /root/.cache/pypoetry/virtualenvs/foo-b2TG5iYf-py3.8
Project environment contains an empty path in sys_path, ignoring.
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
No suitable keyring backend found
No suitable keyring backends were found
Keyring is not available, credentials will be stored and retrieved from configuration files as plaintext.
Creating new session for pypi.org
Skipping url (https://files.pythonhosted.org/packages/ad/ff/058bbd1e0a282e79a678456d2b0dff09122f46e8674bdba7578639179e61/flask-restful-swagger-2-0.20.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/eb/94/6eb85b2d67eca56057ad5dfab94a7b76bc2e40305b0fb82cdaf1978dcee3/flask-restful-swagger-2-0.21.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/48/ef/8646e3a568e1877b257f8851f91ad58c3df1e0f42177ed958dde799f4ff9/flask-restful-swagger-2-0.22.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/55/58/77f1f0ba0c76614bf4aec603b3a06e3a0c356a85624b71314717da703aab/flask-restful-swagger-2-0.23.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/1f/1a/b964bb988d1b8f8f277337e25d45116cd926470c7ba4dcb41a9f724e30d6/flask-restful-swagger-2-0.24.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/b2/c1/af6e02b26425d61a076b18e53a4b26128d74d1e5122a311e5e025a5dfbb4/flask-restful-swagger-2-0.25.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/df/b3/147c25c6ac0d4d32bec6328075030cfb373f240b01550518c0d8af794ff5/flask-restful-swagger-2-0.26.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/68/2a/5333b31ac091c294b1523d9a85b154f962d4d9cd656c232f837bb0557fe4/flask-restful-swagger-2-0.27.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/4a/50/9ab37d04a7e95e00d15aea958e45ff2280f3663fdde956714fe59cc1231b/flask-restful-swagger-2-0.28.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/8a/8b/1e3b748d123b84932276f042d37466d57acbca737194f2fddfa5cb8b67be/flask-restful-swagger-2-0.29.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/fd/ec/712f7e305ae0d0dda7c5a3992d72c68f4548003b040c66ab4352dc9c0cf9/flask-restful-swagger-2-0.30.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/e7/ba/94d8de73a637880619b5463f6e34f93056c79449f52a9adcd181e52437e1/flask-restful-swagger-2-0.31.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/c8/b6/13d898a91442c7a0e9fae9e90f6a9545d008dbd689483340c58e6fbc31b0/flask-restful-swagger-2-0.32.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/42/10/6779b64fb18bf0de363d4774b48ef8f835a780e63afe1600ec61adcbd4db/flask-restful-swagger-2-0.33.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/0d/ad/46019e218e87259e425bed01cafa4755da56cb92d1d7311ca06329c2956e/flask-restful-swagger-2-0.34.tar.gz) due to invalid version (None)
Skipping url (https://files.pythonhosted.org/packages/2f/2e/ad9e34b9c92148f2fc9ac8e8086ffc65e228a7c302a2671dbab1da74723e/flask-restful-swagger-2-0.35.tar.gz) due to invalid version (None)
Source (PyPI): 0 packages found for flask-restful-swagger-2 0.35

  Stack trace:

  9  /usr/local/lib/python3.8/site-packages/cleo/application.py:329 in run
      327│
      328│             try:
    → 329│                 exit_code = self._run(io)
      330│             except Exception as e:
      331│                 if not self._catch_exceptions:

  8  /usr/local/lib/python3.8/site-packages/poetry/console/application.py:185 in _run
      183│         self._load_plugins(io)
      184│
    → 185│         exit_code: int = super()._run(io)
      186│         return exit_code
      187│

  7  /usr/local/lib/python3.8/site-packages/cleo/application.py:423 in _run
      421│             io.input.set_stream(stream)
      422│
    → 423│         exit_code = self._run_command(command, io)
      424│         self._running_command = None
      425│

  6  /usr/local/lib/python3.8/site-packages/cleo/application.py:465 in _run_command
      463│
      464│         if error is not None:
    → 465│             raise error
      466│
      467│         return event.exit_code

  5  /usr/local/lib/python3.8/site-packages/cleo/application.py:449 in _run_command
      447│
      448│             if event.command_should_run():
    → 449│                 exit_code = command.run(io)
      450│             else:
      451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  4  /usr/local/lib/python3.8/site-packages/cleo/commands/base_command.py:119 in run
      117│         io.input.validate()
      118│
    → 119│         status_code = self.execute(io)
      120│
      121│         if status_code is None:

  3  /usr/local/lib/python3.8/site-packages/cleo/commands/command.py:83 in execute
       81│
       82│         try:
    →  83│             return self.handle()
       84│         except KeyboardInterrupt:
       85│             return 1

  2  /usr/local/lib/python3.8/site-packages/poetry/console/commands/add.py:158 in handle
      156│             return 0
      157│
    → 158│         requirements = self._determine_requirements(
      159│             packages,
      160│             allow_prereleases=self.option("allow-prereleases"),

  1  /usr/local/lib/python3.8/site-packages/poetry/console/commands/init.py:375 in _determine_requirements
      373│                 # check that the specified version/constraint exists
      374│                 # before we proceed
    → 375│                 name, _ = self._find_best_version_for_package(
      376│                     requirement["name"],
      377│                     requirement["version"],

  ValueError

  Could not find a matching version of package flask-restful-swagger-2

  at /usr/local/lib/python3.8/site-packages/poetry/console/commands/init.py:404 in _find_best_version_for_package
      400│         )
      401│
      402│         if not package:
      403│             # TODO: find similar
    → 404│             raise ValueError(f"Could not find a matching version of package {name}")
      405│
      406│         return package.pretty_name, selector.find_recommended_require_version(package)
      407│
      408│     def _parse_requirements(self, requirements: list[str]) -> list[dict[str, Any]]:

It looks like this is because of an ambiguously named sdist. @dimbleby @radoering PTAL -- is there anything we can do here, other than request the author re-upload correctly named sdists?

@neersighted neersighted added area/solver Related to the dependency resolver status/confirmed Issue is reproduced and confirmed version/1.2.2 and removed status/triage This issue needs to be triaged labels Nov 8, 2022
@samthurston
Copy link
Author

I have tested some other integer-ended packages and it does indeed seem specific to this one.

@neersighted
Copy link
Member

neersighted commented Nov 8, 2022

It's because the source distribution names are out of spec, but pip does handle that case as it wasn't codified until recent-ish.

(and your previous issue was for a solved bug that caused issues with other integer-suffixed packages -- this bug is unique because of the sdist names)

@dimbleby
Copy link
Contributor

dimbleby commented Nov 8, 2022

I can't say I find myself very interested in doing anything with this, but if anyone does then the code to change should be here - that's what's parsing the package version from the filename.

Main thing to worry about if making a change would likely be to make sure that the main line modern case - with exactly one hyphen in the filename - is the fast path, then the oddities don't matter so much.

@clementw168
Copy link

clementw168 commented Jan 20, 2023

If anyone comes here, this is a quick fix for poetry.

  • Delete the .venv and the poetry.lock file
  • Run poetry run pip install iso-639
  • Add iso-639 = "^0.4.5" in your pyproject.toml
  • Run poetry lock
  • And finally, run poetry install

@C-Borne
Copy link

C-Borne commented Feb 17, 2023

Another possible solution is to add the package from git directly, instead of using the pypi one

  • iso-639 = {git = "https://github.com/noumar/iso639.git", tag = "0.4.5"}

@andrewvaughan
Copy link

andrewvaughan commented Jan 22, 2024

If anyone comes here, this is a quick fix for poetry.

  • Delete the .venv and the poetry.lock file

  • Run poetry run pip install iso-639

  • Add iso-639 = "^0.4.5" in your pyproject.toml

  • Run poetry lock

  • And finally, run poetry install

While not optimal, I wonder if this could be a bodge workaround in the CLI tool. It's hacky, but it could work.

(Coding on my phone, just assume this is pseudocode):

import re
import sys
import subprocess
import toml

from importlib.metadata import version

if re.match(r".*[0-9]+$", package_name):
  try:
     subprocess.check_call([sys.executable, '-m', 'pip', install', package_name])

  except subprocess.CalledProcessError:
    # Handle a bad package
    print("You done a bad package.")
    raise
  
  # Requires Python > 3.8
  package_version = version(package_name)

  # Update pyproject.toml (I don't do a lot with TOML - this is likely terrible)
  with open("pyproject.toml", "rb") as cfg:
    project_toml = toml.load(cfg)

  project_toml["tool.poetry.dependencies"][package_name] = package_version

  with open("pyproject.toml", "w") as cfg:
    toml.dump(project_toml, cfg)

  # Update poetry's stuff (whatever the equivalent is)
  poetry.lock()
  poetry.install()

I'm doing this all from memory... so don't hate on the bad code... just a concept.

@ttencate
Copy link

The workaround of manually installing iso-639 or any other problematic package is no longer sufficient. Since Poetry 1.8.0, poetry lock and other commands subsequently fail with the error list index out of range.

Unless another workaround is found, this makes it essentially impossible to include any integer-ending package in a Poetry project.

@dimbleby
Copy link
Contributor

restating things already in the thread but becoming buried: it is not all integer-ending packages, the bug requires a more specific combination of hyphens and mis-named distributions too

eg https://pypi.org/project/iso-639/#files iso-639-0.4.5.tar.gz should have been uploaded as iso_639-0.4.5.tar.gz

#6996 (comment) points at the code if anyone would like to contribute a fix

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/solver Related to the dependency resolver kind/bug Something isn't working as expected status/confirmed Issue is reproduced and confirmed version/1.2.2
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants