Latest 1.4.0 release causes unclear Issue with aws_psycopg2 install

[jakebrinkmann]

• Poetry version: 1.4.0
• Python version: python3.9

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

When using the latest version, 1.4.0 we get the following runtime error:

  Installing sqlalchemy (1.4.46)

  AssertionError

  In /builds/project-0/.cache/pypoetry/artifacts/39/96/f1/0f021ed26225e8d9035c9bac5a886b024840f841af84bd33f2be466dcb/aws_psycopg2-1.3.8-py3-none-any.whl, psycopg2/_psycopg.cpython-38-aarch64-linux-gnu.so is not mentioned in RECORD

  at venv/lib/python3.9/site-packages/installer/sources.py:158 in get_contents
      154│             if item.filename[-1:] == "/":  # looks like a directory
      155│                 continue
      156│ 
      157│             record = record_mapping.pop(item.filename, None)
    → 158│             assert record is not None, "In {}, {} is not mentioned in RECORD".format(
      159│                 self._zipfile.filename,
      160│                 item.filename,
      161│             )  # should not happen for valid wheels
      162│ 

However, when we downgrade to poetry==1.3.2 this problem goes away...

[dimbleby]

the new installer uses https://github.com/pypa/installer, which apparently is stricter about these things. Your choices are

• ask the aws_psycopg2 project to build valid wheels
• ask the installer project to tolerate invalid wheels
• disable the new installer

https://packaging.python.org/en/latest/specifications/binary-distribution-format/ is clear that

installation will fail if any file in the archive is not both mentioned and correctly hashed in RECORD

so installer (and therefore poetry) is behaving correctly here

[dimbleby]

unreleased pypa/installer#147 refactors this, it's not entirely clear to me but I think that the result will be that (by default) this validation does not happen.

so I expect poetry will pick that up in due course if and when the next installer release arrives

[dimbleby]

fwiw when that new installer release arrives IMO poetry should call the now-optional validation: per the reference above, a compliant installer should fail to install a wheel that is mis-built in this way

[miguelvalente]

I had the same issue and going back to 1.3.2 didn't work. Mind you that I'm doing this inside a Dockerfile.
Using poetry config installer.modern-installation false did it for me whilst using 1.4.0. The reference is #7572

Hey everyone,

thanks a lot for reporting. This seems to be an issue within the installer package which is now used for installing packages. I'm able to reproduce this error with the code example given in the docs. @pradyunsg can you have a look into this?

For now one can disable the new installer with poetry config installer.modern-installation false. (https://python-poetry.org/docs/master/configuration/#installermodern-installation)

fin swimmer

My working Dockerfile.

...
RUN pip install poetry


COPY pyproject.toml poetry.lock ./

RUN poetry config virtualenvs.create false
RUN poetry --version
RUN poetry config installer.modern-installation false
RUN poetry  install --no-interaction --no-ansi --no-root

...

[dimbleby]

@miguelvalente this issue is also not the one that contains the comment that you are replying to. Please stop pasting this comment in irrelevant places.

[miguelvalente]

@miguelvalente this issue is also not the one that contains the comment that you are replying to. Please stop pasting this comment in irrelevant places.

It actually is. I was having the same issue as @jakebrinkmann. Dowgrading the version did not help. Disabling the instaler did as suggested in another issue.

[neersighted]

The key difference between 1.3.2 and 1.4.0 is the new installer. Downgrading not solving this exact stack trace is impossible as this code is not used on 1.3.2.

I'm going to close this issue as a duplicate of #7585.

[dimbleby]

this issue is 7585, I don't think it has a duplicate

(specifically this one is about installer being strict in validation of malformed wheels)

[neersighted]

Oh, shoot, I apparently crossed a wire pretty badly this morning.

[zEdS15B3GCwq]

+1 package producing the same error: pyqt5-qt5==5.15.2

Poetry: 1.4.0 release & current master
Python: 3.10.10

  • Installing pyqt5-qt5 (5.15.2): Failed

  AssertionError

  In C:\Users\R\AppData\Local\pypoetry\Cache\artifacts\5a\64\78\eb3dc30bf7097835ba9ef2d55725a0480f1b97738287b90a24cd9edf1d\PyQt5_Qt5-5.15.2-py3-none-win_amd64.whl, PyQt5_Qt5-5.15.2.dist-info/RECORD is not mentioned in RECORD

  at ~\AppData\Roaming\pypoetry\venv\Lib\site-packages\installer\sources.py:158 in get_contents
      154│             if item.filename[-1:] == "/":  # looks like a directory
      155│                 continue
      156│
      157│             record = record_mapping.pop(item.filename, None)
    → 158│             assert record is not None, "In {}, {} is not mentioned in RECORD".format(
      159│                 self._zipfile.filename,
      160│                 item.filename,
      161│             )  # should not happen for valid wheels
      162│

pyproject.toml:

[tool.poetry]
name = "temp"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "~3.10"
pyqt5-qt5 = "^5.15.2"


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Installation is successful with pre-1.4, non-modern installer (installer.modern-installation=false).

[dimbleby]

NB while this is marked fixed by the merge of #7671, it's actually intentionally left such that poetry will refuse to install aws_psycopg2. Please ask that project to build valid wheels, and meanwhile do not use poetry's modern installer.

Other examples mentioned in this thread are actually somewhat different and should be fixed by #7671

[radoering]

As already mentioned in #7671 failure to install packages that do not validate correctly is what the spec demands:

installation will fail if any file in the archive is not both mentioned and correctly hashed in RECORD

Workaround to install invalid wheels: set installer.modern-installation to false

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.