Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

uv venv --seed doesn't seem to be installing setuptools for pypy #1488

Closed
bendoerry opened this issue Feb 16, 2024 · 5 comments
Closed

uv venv --seed doesn't seem to be installing setuptools for pypy #1488

bendoerry opened this issue Feb 16, 2024 · 5 comments
Assignees
@BurntSushi
Labels
bug Something isn't working pypy Related to PyPy support virtualenv Related to virtual environments

Comments

@bendoerry
Copy link

Not sure whether title is "correct" here, but it presents that way.

Alembic is illustrative here, I've also had this occur with psycogreen, psycopg2cffi, flask-migrate, docopt (this is a non-exhaustive list)

With CPython (using rye to ensure same python version)

> uv venv --python <HOME>/.rye/py/[email protected]/install/bin/python3 --seed
Using Python 3.9.16 interpreter at <HOME>/.rye/py/[email protected]/install/bin/python3.9
Creating virtualenv at: .venv
 + setuptools==69.1.0
 + pip==24.0
 + wheel==0.42.0

> source .venv/bin/activate
> python3 --version
Python 3.9.16

> uv pip install alembic==1.0.11
Resolved 9 packages in 12ms
Installed 9 packages in 8ms
 + alembic==1.0.11
 + greenlet==3.0.3
 + mako==1.3.2
 + markupsafe==2.1.5
 + python-dateutil==2.8.2
 + python-editor==1.0.4
 + six==1.16.0
 + sqlalchemy==2.0.27
 + typing-extensions==4.9.0

With PyPy (This also occurs with system PyPy, not just rye managed)

> uv venv --python <HOME>/.rye/py/[email protected]/bin/python3 --seed
Using Python 3.9.16 interpreter at <HOME>/.rye/py/[email protected]/bin/pypy3.9
Creating virtualenv at: .venv
 + setuptools==69.1.0
 + pip==24.0
 + wheel==0.42.0

> source .venv/bin/activate
> python3 --version
Python 3.9.16 (feeb267ead3e6771d3f2f49b83e1894839f64fb7, Dec 29 2022, 14:23:21)
[PyPy 7.3.11 with GCC 10.2.1 20210130 (Red Hat 10.2.1-11)]

> uv pip install alembic==1.0.11
Resolved 9 packages in 11ms
error: Failed to download distributions
  Caused by: Failed to fetch wheel: greenlet==3.0.3
  Caused by: Failed to build: greenlet==3.0.3
  Caused by: Build backend failed to determine extra requires with `build_wheel()`:
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 4, in <module>
ModuleNotFoundError: No module named 'setuptools'
---

> # Next one included since error is slightly different
> uv pip install alembic==1.0.11 --index-url <index with prebuilt pypy wheels>
error: Failed to download and build: alembic==1.0.11
  Caused by: Failed to build: alembic==1.0.11
  Caused by: Build backend failed to determine metadata through `prepare_metadata_for_build_wheel`:
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 4, in <module>
ModuleNotFoundError: No module named 'setuptools'
---
@zanieb zanieb added bug Something isn't working virtualenv Related to virtual environments labels Feb 16, 2024
@BurntSushi BurntSushi self-assigned this Feb 19, 2024
@edgarrmondragon edgarrmondragon mentioned this issue Feb 20, 2024
Closed
@edgarrmondragon
Copy link
Contributor

edgarrmondragon commented Feb 20, 2024 •
edited
Loading

I think this is a general issue with build requirements in PyPy. See also tox-dev/tox-uv#15.

@edgarrmondragon edgarrmondragon mentioned this issue Feb 21, 2024
Merged
@konstin konstin added the pypy Related to PyPy support label Feb 21, 2024
@edgarrmondragon edgarrmondragon mentioned this issue Feb 22, 2024
Closed
BurntSushi added a commit that referenced this issue Feb 29, 2024
@BurntSushi
virtualenv: determine 'site-packages' based on implementation name
a43314a 
I'm not at all sure whether this is a correct fix or not, but it does
seem to make `pypy` work in at least some cases with `uv`. Previously,
I couldn't get it to work at all. Namely the virtualenv was created
with a `lib/python3.10/site-packages`, but whenever I did a `uv
pip install` in that virtualenv, it was looking for a non-existent
`lib/pypy3.10/site-packages` directory.

With this PR, the workflow reported as not working in #1488 now works
for me:

```
$ pypy3 --version
Python 3.10.13 (fc59e61cfbff, Jan 17 2024, 05:35:45)
[PyPy 7.3.15 with GCC 13.2.1 20230801]

$ uv venv --python $(which pypy3) --seed
Using Python 3.10.13 interpreter at: /usr/bin/pypy3
Creating virtualenv at: .venv
 + pip==24.0
 + setuptools==69.1.1
 + wheel==0.42.0
Activate with: source .venv/bin/activate

$ uv pip install 'alembic==1.0.11'
Resolved 9 packages in 8ms
Installed 9 packages in 14ms
 + alembic==1.0.11
 + greenlet==3.0.3
 + mako==1.3.2
 + markupsafe==2.1.5
 + python-dateutil==2.8.2
 + python-editor==1.0.4
 + six==1.16.0
 + sqlalchemy==2.0.27
 + typing-extensions==4.10.0
```

Where as previously (current `main`), I was hitting this error:

```
$ uv venv --python $(which pypy3) --seed
Using Python 3.10.13 interpreter at: /usr/bin/pypy3
Creating virtualenv at: .venv
 + pip==24.0
 + setuptools==69.1.1
 + wheel==0.42.0
Activate with: source .venv/bin/activate

$ uv pip install 'alembic==1.0.11'
error: Failed to list installed packages
  Caused by: failed to read directory `/home/andrew/astral/issues/uv/i1488/.venv/lib/pypy3.10/site-packages`
  Caused by: No such file or directory (os error 2)
```

Notice though that neither outcome above matches the error reported in #1488,
so this is likely not a complete fix. There are perhaps other lurking
issues.

Ref #1488
BurntSushi added a commit that referenced this issue Feb 29, 2024
@BurntSushi
virtualenv: determine 'site-packages' based on implementation name
0d480b7 
I'm not at all sure whether this is a correct fix or not, but it does
seem to make `pypy` work in at least some cases with `uv`. Previously,
I couldn't get it to work at all. Namely the virtualenv was created
with a `lib/python3.10/site-packages`, but whenever I did a `uv
pip install` in that virtualenv, it was looking for a non-existent
`lib/pypy3.10/site-packages` directory.

With this PR, the workflow reported as not working in #1488 now works
for me:

```
$ pypy3 --version
Python 3.10.13 (fc59e61cfbff, Jan 17 2024, 05:35:45)
[PyPy 7.3.15 with GCC 13.2.1 20230801]

$ uv venv --python $(which pypy3) --seed
Using Python 3.10.13 interpreter at: /usr/bin/pypy3
Creating virtualenv at: .venv
 + pip==24.0
 + setuptools==69.1.1
 + wheel==0.42.0
Activate with: source .venv/bin/activate

$ uv pip install 'alembic==1.0.11'
Resolved 9 packages in 8ms
Installed 9 packages in 14ms
 + alembic==1.0.11
 + greenlet==3.0.3
 + mako==1.3.2
 + markupsafe==2.1.5
 + python-dateutil==2.8.2
 + python-editor==1.0.4
 + six==1.16.0
 + sqlalchemy==2.0.27
 + typing-extensions==4.10.0
```

Where as previously (current `main`), I was hitting this error:

```
$ uv venv --python $(which pypy3) --seed
Using Python 3.10.13 interpreter at: /usr/bin/pypy3
Creating virtualenv at: .venv
 + pip==24.0
 + setuptools==69.1.1
 + wheel==0.42.0
Activate with: source .venv/bin/activate

$ uv pip install 'alembic==1.0.11'
error: Failed to list installed packages
  Caused by: failed to read directory `/home/andrew/astral/issues/uv/i1488/.venv/lib/pypy3.10/site-packages`
  Caused by: No such file or directory (os error 2)
```

Notice though that neither outcome above matches the error reported in #1488,
so this is likely not a complete fix. There are perhaps other lurking
issues.

Ref #1488
@BurntSushi BurntSushi mentioned this issue Feb 29, 2024
Merged
BurntSushi added a commit that referenced this issue Feb 29, 2024
@BurntSushi
virtualenv: determine 'site-packages' based on implementation name
8122d80 
I'm not at all sure whether this is a correct fix or not, but it does
seem to make `pypy` work in at least some cases with `uv`. Previously,
I couldn't get it to work at all. Namely the virtualenv was created
with a `lib/python3.10/site-packages`, but whenever I did a `uv
pip install` in that virtualenv, it was looking for a non-existent
`lib/pypy3.10/site-packages` directory.

With this PR, the workflow reported as not working in #1488 now works
for me:

```
$ pypy3 --version
Python 3.10.13 (fc59e61cfbff, Jan 17 2024, 05:35:45)
[PyPy 7.3.15 with GCC 13.2.1 20230801]

$ uv venv --python $(which pypy3) --seed
Using Python 3.10.13 interpreter at: /usr/bin/pypy3
Creating virtualenv at: .venv
 + pip==24.0
 + setuptools==69.1.1
 + wheel==0.42.0
Activate with: source .venv/bin/activate

$ uv pip install 'alembic==1.0.11'
Resolved 9 packages in 8ms
Installed 9 packages in 14ms
 + alembic==1.0.11
 + greenlet==3.0.3
 + mako==1.3.2
 + markupsafe==2.1.5
 + python-dateutil==2.8.2
 + python-editor==1.0.4
 + six==1.16.0
 + sqlalchemy==2.0.27
 + typing-extensions==4.10.0
```

Where as previously (current `main`), I was hitting this error:

```
$ uv venv --python $(which pypy3) --seed
Using Python 3.10.13 interpreter at: /usr/bin/pypy3
Creating virtualenv at: .venv
 + pip==24.0
 + setuptools==69.1.1
 + wheel==0.42.0
Activate with: source .venv/bin/activate

$ uv pip install 'alembic==1.0.11'
error: Failed to list installed packages
  Caused by: failed to read directory `/home/andrew/astral/issues/uv/i1488/.venv/lib/pypy3.10/site-packages`
  Caused by: No such file or directory (os error 2)
```

Notice though that neither outcome above matches the error reported in #1488,
so this is likely not a complete fix. There are perhaps other lurking
issues.

Ref #1488
@BurntSushi
Copy link
Member

With #2094 merged, I was able to run your original set of steps successfully. Could you try again when the next release is out? (Or build from main if you want to try earlier.) If it still doesn't work, it might help to share a bit more about your environment. Linux? macOS? And show the output of RUST_LOG=trace uv pip install -v ....

@charliermarsh
Copy link
Member

This actually might be fixed... Let's see.

@bendoerry
Copy link
Author

So this does seem to work for us now, we're able to install all our dependencies on PyPy and so far our tests seem to pass (haven't had time to run the full test suite though).

@charliermarsh
Copy link
Member

Okay cool -- closing for now but if you run into other PyPy issues, please file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@BurntSushi BurntSushi

Labels
bug Something isn't working pypy Related to PyPy support virtualenv Related to virtual environments
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@BurntSushi @charliermarsh @zanieb @konstin @bendoerry @edgarrmondragon