Poetry 1.8.0 unable to lock files (worked in 1.7.1)

[matejsp]

Description

Running poetry lock with latest poetry 1.8.0 seems broken with:
Sonatype Nexus Repository OSS 3.52.0-01


Updating dependencies
Resolving dependencies... (12.6s)

404 Client Error: Not Found for url: https://myuser:[email protected]/repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl#sha256=a8ba463124b3de17fe11c4a9cf2080511ec9d5da8bf890b20f5e5a614dfa266c

[[tool.poetry.source]]
name = "nexus"
url = "https://myuser:[email protected]/repository/pypi-blessed/simple"
priority = "primary"

(.venv) ➜ curl -v https://myuser:[email protected]/repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl\#sha256\=a8ba463124b3de17fe11c4a9cf2080511ec9d5da8bf890b20f5e5a614dfa266c 
* using HTTP/1.1
* Server auth using Basic with user 'myuser'
> GET /repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl HTTP/1.1
> Host: nexus.mycompany.net
> Authorization: Basic ....
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx/1.24.0
< Date: Mon, 26 Feb 2024 08:56:04 GMT
< Content-Type: application/zip
< Content-Length: 8864
< Last-Modified: Wed, 06 Dec 2023 06:59:37 GMT
< ETag: "{SHA1{9192aaf7a657752fa183da289f64e9a1c39d6e81}}"
< 
Warning: Binary output can mess up your terminal. Use "--output -" to tell 

Workarounds

no known workarounds except downgrade. 
tried: poetry config http-basic.nexus <username> <password>

Poetry Installation Method

pipx

Operating System

MacOSX

Poetry Version

Poetry (version 1.8.0)

Poetry Configuration

cache-dir = "/Users/myuser/Library/Caches/pypoetry"
experimental.system-git-client = false
installer.max-workers = null
installer.modern-installation = true
installer.no-binary = null
installer.parallel = true
keyring.enabled = true
repositories.nexus.url = "https://myuser:[email protected]/repository/pypi-blessed/simple"
solver.lazy-wheel = true
virtualenvs.create = true
virtualenvs.in-project = true
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.no-setuptools = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /Users/myuser/Library/Caches/pypoetry/virtualenvs
virtualenvs.prefer-active-python = false
virtualenvs.prompt = "{project_name}-py{python_version}"
warnings.export = false

Python Sysconfig

No response

Example pyproject.toml

[tool.poetry]
name = "bugreport"
version = "0.0.0"
description = "Bla"
authors = ["bla"]

[tool.poetry.dependencies]
python = ">=3.11, <4"
django-stubs-ext = "==4.2.7"

[[tool.poetry.source]]
name = "nexus"
url = "https://myuser:[email protected]/repository/pypi-blessed/simple"
priority = "primary"

Poetry Runtime Logs

(.venv) ➜  poetrylock poetry -vvv lock
Adding repository nexus (https://myuser:[email protected]/repository/pypi-blessed/simple) and setting it as primary
Deactivating the PyPI repository
Using virtualenv: /Users/myuser/projects/myproject/.venv
Updating dependencies
Resolving dependencies...
   1: fact: bugreport is 0.0.0
   1: derived: bugreport
   1: fact: bugreport depends on django-stubs-ext (==4.2.7)
   1: selecting bugreport (0.0.0)
   1: derived: django-stubs-ext (==4.2.7)
Creating new session for myuser:[email protected]
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): nexus.mycompany.net:443
[urllib3:urllib3.connectionpool] https://nexus.mycompany.net:443 "GET /repository/pypi-blessed/simple/django-stubs-ext/ HTTP/1.1" 200 2736
Source (nexus): 1 packages found for django-stubs-ext 4.2.7
begin fetching content length
initial bytes request: bytes=-10000
[urllib3:urllib3.connectionpool] https://nexus.mycompany.net:443 "GET /repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl HTTP/1.1" 404 1824
   1: Version solving took 0.172 seconds.
   1: Tried 1 solutions.

  ValueError

  Package('django-stubs-ext', '4.2.7') is not in list

  at ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/legacy_repository.py:66 in package
       62│         Note that this will be cached so the subsequent operations
       63│         should be much faster.
       64│         """
       65│         try:
    →  66│             index = self._packages.index(Package(name, version))
       67│ 
       68│             return self._packages[index]
       69│         except ValueError:
       70│             package = super().package(name, version, extras)

The following error occurred when trying to handle this error:


  Stack trace:

  34  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:327 in run
       325│ 
       326│             try:
     → 327│                 exit_code = self._run(io)
       328│             except BrokenPipeError:
       329│                 # If we are piped to another process, it may close early and send a

  33  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/application.py:190 in _run
       188│         self._load_plugins(io)
       189│ 
     → 190│         exit_code: int = super()._run(io)
       191│         return exit_code
       192│ 

  32  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:431 in _run
       429│             io.input.interactive(interactive)
       430│ 
     → 431│         exit_code = self._run_command(command, io)
       432│         self._running_command = None
       433│ 

  31  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:473 in _run_command
       471│ 
       472│         if error is not None:
     → 473│             raise error
       474│ 
       475│         return terminate_event.exit_code

  30  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/application.py:457 in _run_command
       455│ 
       456│             if command_event.command_should_run():
     → 457│                 exit_code = command.run(io)
       458│             else:
       459│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  29  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/base_command.py:117 in run
       115│         io.input.validate()
       116│ 
     → 117│         return self.execute(io) or 0
       118│ 
       119│     def merge_application_definition(self, merge_args: bool = True) -> None:

  28  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/cleo/commands/command.py:61 in execute
        59│ 
        60│         try:
     →  61│             return self.handle()
        62│         except KeyboardInterrupt:
        63│             return 1

  27  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/console/commands/lock.py:55 in handle
        53│         self.installer.lock(update=not self.option("no-update"))
        54│ 
     →  55│         return self.installer.run()
        56│ 

  26  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/installer.py:104 in run
       102│             self.verbose(True)
       103│ 
     → 104│         return self._do_install()
       105│ 
       106│     def dry_run(self, dry_run: bool = True) -> Installer:

  25  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/installer.py:241 in _do_install
       239│                 source_root=self._env.path.joinpath("src")
       240│             ):
     → 241│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       242│         else:
       243│             self._io.write_line("Installing dependencies from lock file")

  24  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/puzzle/solver.py:71 in solve
        69│         with self._progress(), self._provider.use_latest_for(use_latest or []):
        70│             start = time.time()
     →  71│             packages, depths = self._solve()
        72│             end = time.time()
        73│ 

  23  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/puzzle/solver.py:154 in _solve
       152│ 
       153│         try:
     → 154│             result = resolve_version(self._package, self._provider)
       155│ 
       156│             packages = result.packages

  22  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/mixology/__init__.py:18 in resolve_version
        16│     solver = VersionSolver(root, provider)
        17│ 
     →  18│     return solver.solve()
        19│ 

  21  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/mixology/version_solver.py:175 in solve
       173│             while next is not None:
       174│                 self._propagate(next)
     → 175│                 next = self._choose_package_version()
       176│ 
       177│             return self._result()

  20  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/mixology/version_solver.py:514 in _choose_package_version
       512│             package = locked
       513│ 
     → 514│         package = self._provider.complete_package(package)
       515│ 
       516│         conflict = False

  19  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/puzzle/provider.py:489 in complete_package
       487│                 dependency_package = DependencyPackage(
       488│                     dependency,
     → 489│                     self._pool.package(
       490│                         package.pretty_name,
       491│                         package.version,

  18  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/repository_pool.py:204 in package
       202│         for repo in self.repositories:
       203│             try:
     → 204│                 return repo.package(name, version, extras=extras)
       205│             except PackageNotFound:
       206│                 continue

  17  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/legacy_repository.py:70 in package
        68│             return self._packages[index]
        69│         except ValueError:
     →  70│             package = super().package(name, version, extras)
        71│             package._source_type = "legacy"
        72│             package._source_url = self._url

  16  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/cached_repository.py:75 in package
        73│         extras: list[str] | None = None,
        74│     ) -> Package:
     →  75│         return self.get_release_info(canonicalize_name(name), version).to_package(
        76│             name=name, extras=extras
        77│         )

  15  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/cached_repository.py:52 in get_release_info
        50│             return PackageInfo.load(self._get_release_info(name, version))
        51│ 
     →  52│         cached = self._release_cache.remember(
        53│             f"{name}:{version}", lambda: self._get_release_info(name, version)
        54│         )

  14  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/cache.py:147 in remember
       145│         value = self.get(key)
       146│         if value is None:
     → 147│             value = callback() if callable(callback) else callback
       148│             self.put(key, value, minutes)
       149│         return value

  13  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/cached_repository.py:53 in <lambda>
        51│ 
        52│         cached = self._release_cache.remember(
     →  53│             f"{name}:{version}", lambda: self._get_release_info(name, version)
        54│         )
        55│ 

  12  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/legacy_repository.py:123 in _get_release_info
       121│         yanked = page.yanked(name, version)
       122│ 
     → 123│         return self._links_to_data(
       124│             links,
       125│             PackageInfo(

  11  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/http_repository.py:360 in _links_to_data
       358│ 
       359│         # drop yanked files unless the entire release is yanked
     → 360│         info = self._get_info_from_links(links, ignore_yanked=not data.yanked)
       361│ 
       362│         data.summary = info.summary

  10  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/http_repository.py:310 in _get_info_from_links
       308│                 return self._get_info_from_metadata(
       309│                     universal_python3_wheel
     → 310│                 ) or self._get_info_from_wheel(universal_python3_wheel)
       311│ 
       312│             if universal_python2_wheel:

   9  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/repositories/http_repository.py:123 in _get_info_from_wheel
       121│             try:
       122│                 package_info = PackageInfo.from_metadata(
     → 123│                     metadata_from_wheel_url(link.filename, link.url, self.session)
       124│                 )
       125│             except HTTPRangeRequestUnsupported:

   8  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:77 in metadata_from_wheel_url
        75│         # After context manager exit, wheel.name will point to a deleted file path.
        76│         # Add `delete_backing_file=False` to disable this for debugging.
     →  77│         with LazyWheelOverHTTP(url, session) as lazy_file:
        78│             metadata_bytes = lazy_file.read_metadata(name)
        79│ 

   7  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:289 in __enter__
       287│     def __enter__(self: U) -> U:
       288│         super().__enter__()
     → 289│         self._setup_content()
       290│         return self
       291│ 

   6  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:357 in _setup_content
       355│         if self._length is None:
       356│             logger.debug("begin fetching content length")
     → 357│             self._length = self._fetch_content_length()
       358│             logger.debug("done fetching content length (is: %d)", self._length)
       359│             # Enable us to seek and write anywhere in the backing file up to this

   5  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:508 in _fetch_content_length
       506│         """
       507│         initial_chunk_size = self._initial_chunk_length()
     → 508│         ret_length, tail = self._extract_content_length(initial_chunk_size)
       509│ 
       510│         # Need to explicitly truncate here in order to perform the write and seek

   4  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:609 in _extract_content_length
       607│         try:
       608│             # Initial range request for just the end of the file.
     → 609│             file_length, tail = self._try_initial_chunk_request(initial_chunk_size)
       610│         except HTTPError as e:
       611│             resp = e.response

   3  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/inspection/lazy_wheel.py:572 in _try_initial_chunk_request
       570│ 
       571│         self._request_count += 1
     → 572│         tail = self._session.get(self._url, headers=headers, stream=True)
       573│         tail.raise_for_status()
       574│ 

   2  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/authenticator.py:267 in get
       265│ 
       266│     def get(self, url: str, **kwargs: Any) -> requests.Response:
     → 267│         return self.request("get", url, **kwargs)
       268│ 
       269│     def head(self, url: str, **kwargs: Any) -> requests.Response:

   1  ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/utils/authenticator.py:245 in request
       243│                 if resp.status_code not in STATUS_FORCELIST or is_last_attempt:
       244│                     if raise_for_status:
     → 245│                         resp.raise_for_status()
       246│                     return resp
       247│ 

  HTTPError

  404 Client Error: Not Found for url: https://myuser:[email protected]/repository/pypi-blessed/packages/django-stubs-ext/4.2.7/django_stubs_ext-4.2.7-py3-none-any.whl#sha256=a8ba463124b3de17fe11c4a9cf2080511ec9d5da8bf890b20f5e5a614dfa266c

  at ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/requests/models.py:1021 in raise_for_status
      1017│                 f"{self.status_code} Server Error: {reason} for url: {self.url}"
      1018│             )
      1019│ 
      1020│         if http_error_msg:
    → 1021│             raise HTTPError(http_error_msg, response=self)
      1022│ 
      1023│     def close(self):
      1024│         """Releases the connection back to the pool. Once this method has been
      1025│         called the underlying ``raw`` object must not be accessed again.
(.venv) ➜  poetrylock

[sssilvar]

Same here. Tested on MacOS and Ubuntu installed using curl.

[dimbleby]

404 is the server telling you that the wheel does not exist. is it true?

If not, you should talk to Nexus about why they are returning 404.

cf #9026

[david-nano]

404 is the server telling you that the wheel does not exist. is it true?

If not, you should talk to Nexus about why they are returning 404.

cf #9026

The issue is not on Nexus side, here is my error:

Creating virtualenv app-v2-mock-_KxWvR52-py3.8 in /root/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... Downloading http://nexus.domain.local:8081/repository/deepypigr/packages/app-v2/0.0.3370/app_mode_separation_block-0.0.3370-py3-none-any.whl#md5=8d70b702f866be9ad21a8c55f631aab9Resolving dependencies... (8.4s)

404 Client Error: Not Found for url: http://nexus.domain.local:8081/repository/deepypigr/packages/app-plugins/0.0.85/app_plugins-0.0.85-py3-none-any.whl#md5=9e110805fdb4f32a6add6279624f1b86

When trying to reach http://nexus.domain.local:8081/repository/deepypigr/packages/app-plugins/0.0.85/app_plugins-0.0.85-py3-none-any.whl#md5=9e110805fdb4f32a6add6279624f1b86 it's start a download of the whl file.
the same export is working on poetry==1.7.1 it's a bug that we are facing only in 1.8.0.

[abn]

Can you try the workaround mentioned at #9026 (comment) please?

The 404 is interesting because this implies that either a file that we expect the index to have does not exist or is not accessible for some reason. Alternatively, poetry might be expecting incorrectly a file to exist.

I'd definitely verify if the issue is an actual 404 or Nexus returning 404 because we are not allowed to do a ranged read as it might treat this as a "not authorized" scenario. Some servers send a 404 when a 400/401/403 occurs.

Disabling the lazy wheel feature should tell us more.

[radoering]

You can also try: curl -I <link-to-file> vs. curl -I -H "Range: bytes=-10000" <link-to-file> and check the status code to verify the server behavior.

[skrese]

The status of curl -I <link-to-file> is 200.

[abn]

The status of curl -I <link-to-file> is 200.

How about curl -I -H "Range: bytes=-10000" <link-to-file>?

[skrese]

Same thing with curl -I -H "Range: bytes=-10000" <link-to-file>

[david-nano]

Can you try the workaround mentioned at #9026 (comment) please?

The 404 is interesting because this implies that either a file that we expect the index to have does not exist or is not accessible for some reason. Alternatively, poetry might be expecting incorrectly a file to exist.

I'd definitely verify if the issue is an actual 404 or Nexus returning 404 because we are not allowed to do a ranged read as it might treat this as a "not authorized" scenario. Some servers send a 404 when a 400/401/403 occurs.

Disabling the lazy wheel feature should tell us more.

It's working, but my current workaround is to keep working with the old version before change my whole CI.

[abn]

@david-nano makes sense. I suspect we will make a patch release for this as it might be an issue for a lot of folks.

[abn]

@david-nano can you also please try the curl commands @radoering posted, in the environment where you got the failure?

[david-nano]

curl -I -H "Range: bytes=-10000"

HTTP/1.1 200 OK
Date: Mon, 26 Feb 2024 11:13:25 GMT
Server: Nexus/3.38.1-01 (OSS)
X-Content-Type-Options: nosniff
Content-Security-Policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
X-XSS-Protection: 1; mode=block
ETag: "{SHA1{6b145fc2d72079f4a7cb24feddbd2bc094de02ad}}"
Content-Type: application/zip
Content-Length: 7111

[abn]

@radoering something else might be off with the client request. But would need a problematic index server to test against. :(

[radoering]

@david-nano @skrese Can you please try if #9030 fixes your issue? I'm not 100 % sure because we don't understand Nexus' behavior completely.

[abn]

You can try using pipx like so.

pipx install --suffix=@9030 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/9030/head'
poetry@9030 lock

[skrese]

For me POETRY_SOLVER_LAZY_WHEEL=0 poetry lock it works. @radoering

[radoering]

POETRY_SOLVER_LAZY_WHEEL=0 is basically what 1.7.1 does. My request is to try #9030 with lazy-wheel active. (And I suppose you have to clear your cache or copy the folder for backup if you do not want to clear it, to get meaningful test results.)

[matejsp]

There is something wrong with Range: bytes=-10000 ... that nexus does not accept and this changed in latest poetry

Curl with -I works.
-I is just a HEAD response in CURL and it returns 200 (since it is not sending range bytes anyway).

However:

(.venv) ➜  poetrylock curl -v -H "Range: bytes=-10000" https://myuser:[email protected]/repository/pypi-blessed/packages/mccabe/0.7.0/mccabe-0.7.0-py2.py3-none-any.whl\#sha256\=3259e7631b90a2a25934467501437109b9bead9db61aedf3539e79873759a6c0   
*   Trying x.x.x.x:443...
* using HTTP/1.1
* Server auth using Basic with user 'bitstamp'
> GET /repository/pypi-blessed/packages/mccabe/0.7.0/mccabe-0.7.0-py2.py3-none-any.whl HTTP/1.1
> Host: nexus.mycompany.net
> Authorization: Basic XXXXXXXXXXXXXXXXXXXX=
> User-Agent: curl/8.4.0
> Accept: */*
> Range: bytes=-10000
> 
< HTTP/1.1 404 Not Found
< Server: nginx/1.24.0
< Date: Mon, 26 Feb 2024 12:04:24 GMT
< Content-Type: text/html
< Content-Length: 1824
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Content-Security-Policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate, post-check=0, pre-check=0
< Pragma: no-cache
< Expires: 0
< X-Frame-Options: DENY
< 

[matejsp]

curl -v -H "Range: bytes=0-100" https://myuser:[email protected]/repository/pypi-blessed/packages/mccabe/0.7.0/mccabe-0.7.0-py2.py3-none-any.whl\#sha256\=3259e7631b90a2a25934467501437109b9bead9db61aedf3539e79873759a6c0   

< HTTP/1.1 206 Partial Content
< Server: nginx/1.24.0
< Date: Mon, 26 Feb 2024 12:10:20 GMT
< Content-Type: application/zip
< Content-Length: 101
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Content-Security-Policy: sandbox allow-forms allow-modals allow-popups allow-presentation allow-scripts allow-top-navigation
< X-XSS-Protection: 1; mode=block
< Content-Range: bytes 0-100/7348
< Last-Modified: Mon, 13 Jun 2022 13:35:05 GMT
< ETag: "{SHA1{6e0fd745753c9ec01a68b98e738da71382411d32}}"

According to mozilla -10000 is not correct ... it should be 0-10000.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests

[radoering]

@matejsp Thanks a lot. 🙏 So my reproducer with curl was just not good enough. 🤦

Then, it's clear that Nexus supports range requests but not negative offsets and instead of returning not implemented or method not allowed it returns not found. I'm confident now that #9030 will fix this.

[abn]

Resolved in #9030 #9032

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.