Skip to content

Commit

Permalink
Remove pip-tools, use uv for all operations (#1226)
Browse files Browse the repository at this point in the history
  • Loading branch information
nazq authored Aug 21, 2024
1 parent bf3ccf8 commit 2c69050
Show file tree
Hide file tree
Showing 20 changed files with 180 additions and 718 deletions.
5 changes: 0 additions & 5 deletions docs/guide/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,11 +81,6 @@ force-rye-managed = false
# virtual environments.
global-python = false

# When set to `true`, Rye will use `uv` for package resolution and installation.
# Set to `false` to fall back to the `pip-tools` resolver.
# Learn more about uv here: https://github.com/astral-sh/uv
use-uv = true

# Enable or disable automatic `sync` after `add` and `remove`. This defaults
# to `true` when uv is enabled and `false` otherwise.
autosync = true
Expand Down
34 changes: 27 additions & 7 deletions docs/guide/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,26 @@ This guide requires some familiarity with Docker and Dockerfiles.
- Your `pyproject.toml` should contain `virtual = true` under the `[tool.rye]` section. If it's not there, add it and run `rye sync`.
- If you're just setting up a project, run `rye init --virtual` instead of `rye init`.

1. Create a `Dockerfile` in your project root with the following content:
2. Create a `Dockerfile` in your project root with the following content, using [`uv`](https://github.com/astral-sh/uv):

```Dockerfile
FROM python:slim

RUN pip install uv

WORKDIR /app
COPY requirements.lock ./
uv pip install --no-cache -r requirements.lock

COPY src .
CMD python main.py
```

Or, using `pip`:

```Dockerfile
FROM python:slim

WORKDIR /app
COPY requirements.lock ./
RUN PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir -r requirements.lock
Expand All @@ -27,7 +42,7 @@ This guide requires some familiarity with Docker and Dockerfiles.
CMD python main.py
```

2. You can now build your image like this:
3. You can now build your image like this:

```bash
docker build .
Expand Down Expand Up @@ -56,11 +71,12 @@ The `Dockerfile`s in this guide are examples. Some adjustments you might want to
If your code is an installable package, it's recommended that you first build it, then install it inside your Docker image.
This way you can be sure that the image is exactly the same as what a user installation would be.

An example `Dockerfile` might look like this:
An example `Dockerfile` might look like this with [`uv`](https://github.com/astral-sh/uv):

```Dockerfile
FROM python:slim
RUN --mount=source=dist,target=/dist PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir /dist/*.whl
RUN pip install uv
RUN --mount=source=dist,target=/dist uv pip install --no-cache /dist/*.whl
CMD python -m my_package
```

Expand All @@ -78,8 +94,12 @@ The [Dockerfile adjustments from the previous section](#dockerfile-adjustments)

## Explanations

Rye's lock file standard is the `requirements.txt` format from `pip`, so you don't actually need `rye` in your container to be able to install dependencies.
Rye's lockfile standard is the `requirements.txt` format from `pip` (and used by [`uv`](https://github.com/astral-sh/uv)), so you don't actually need `rye` in your container to be able to install dependencies.
This makes the Dockerfile much simpler and avoids the necessity for multi-stage builds if small images are desired.

The `PYTHONDONTWRITEBYTECODE=1` env var and `--no-cache-dir` parameters when invoking Pip both make the image smaller by not writing any temporary files.
Both Bytecode and Pip caches are speeding things up when running Pip multiple times, but since we are working in a container, we can be pretty sure that we'll never invoke pip again.
The `--no-cache-dir` and `--no-cache` parameters, passed to `pip` and `uv` respectively, make the image smaller by not
writing any temporary files. While caching can speed up subsequent builds, it's not necessary in a container where the
image is built once and then used many times.

Similarly, the `PYTHONDONTWRITEBYTECODE=1` environment variable is set to avoid writing `.pyc` files, which are not
needed in a container. (`uv` skips writing `.pyc` files by default.)
8 changes: 0 additions & 8 deletions docs/guide/rust.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,14 +41,6 @@ it as a global tool:
rye install maturin
```

Note that `maturin develop` requires `pip` to be installed into the virtualenv. Before
you can use it you need to add it:

```
rye add --dev pip
rye sync
```

Rye recommends mixed python/rust modules. In that case you can save some valuable
iteration time by running `maturin develop --skip-install`:

Expand Down
29 changes: 16 additions & 13 deletions docs/guide/sync.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,9 @@
# Syncing and Locking

Rye supports two systems to manage dependencies:
[uv](https://github.com/astral-sh/uv) and
[pip-tools](https://github.com/jazzband/pip-tools). It currently defaults to
`uv` as it offers significantly better performance, but will offer you the
option to use `pip-tools` instead.
Rye uses [`uv`](https://github.com/astral-sh/uv) to manage dependencies.

In order to download dependencies rye creates two "lockfiles" (called
`requirements.lock` and `requirements-dev.lock`). These are not real lockfiles
`requirements.lock` and `requirements-dev.lock`). These are not real lockfiles,
but they fulfill a similar purpose until a better solution has been implemented.

Whenever `rye sync` is called, it will update lockfiles as well as the
Expand Down Expand Up @@ -66,12 +62,12 @@ rye lock Flask --pre
+++ 0.18.0

By default (unless the `tool.rye.lock-with-sources` config key is set to `true` in the
`pyproject.toml`) lock files are not generated with source references. This means that
if custom sources are used the lock file cannot be installed via `pip` unless also
`pyproject.toml`) lockfiles are not generated with source references. This means that
if custom sources are used the lockfile cannot be installed via `uv` or `pip`, unless
`--find-links` and other parameters are manually passed. This can be particularly useful
when the lock file is used for docker image builds.
when the lockfile is used for Docker image builds.

When this flag is passed then the lock file is generated with references to `--index-url`,
When this flag is passed then the lockfile is generated with references to `--index-url`,
`--extra-index-url` or `--find-links`.

```
Expand Down Expand Up @@ -100,11 +96,18 @@ lockfile (`requirements-dev.lock`).
rye sync --no-dev
```

## Limitations
## Platform Compatibility

Lockfiles depend on the platform they were generated on. This is a known limitation
in pip-tools.
By default, lockfiles depend on the platform they were generated on.

For example, if your project relies on platform-specific packages and you generate
lockfiles on Windows, these lockfiles will include Windows-specific projects.
Consequently, they won't be compatible with other platforms like Linux or macOS.

To generate a cross-platform lockfile, you can enable uv's `universal` setting
by adding the following to your `pyproject.toml`:

```toml
[tool.rye]
universal = true
```
15 changes: 6 additions & 9 deletions docs/philosophy.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,13 @@ on my mind when I built it:
of dependencies. Not even `pip` or `setuptools` are installed into it. Rye
manages the virtualenv from outside the virtualenv.

- **No Core Non Standard Stuff:** Rye (with the exception of it's own `tool` section
- **No Core Non-Standard Stuff:** Rye (with the exception of its own `tool` section
in the `pyproject.toml`) uses standardized keys. That means it uses regular
requirements as you would expect. It also does not use a custom lock file
format and uses [`uv`](https://github.com/astral-sh/uv) and
[`pip-tools`](https://github.com/jazzband/pip-tools) behind the scenes.
format and uses [`uv`](https://github.com/astral-sh/uv).

- **No Pip:** Rye uses pip, but it does not expose it. It manages dependencies in
`pyproject.toml` only.
- **No Pip:** Rye uses [`uv`](https://github.com/astral-sh/uv) to manage dependencies,
through `pyproject.toml` only.

- **No System Python:** I can't deal with any more linux distribution weird Python
installations or whatever mess there is on macOS. I used to build my own Pythons
Expand Down Expand Up @@ -53,10 +52,8 @@ lack of standardization. Here is what this project ran into over the years:
which allows both remote and local references to co-exist and it rewrites them
on publish.

- **No Exposed Pip:** pip is intentionally not exposed. If you were to install something
into the virtualenv, it disappears next time you sync. If you symlink `rye` to
`~/.rye/shims/pip` you can get access to pip without installing it into the
virtualenv. There be dragons.
- **No Exposed Pip:** pip is intentionally not exposed. If you install something
into the virtualenv with pip, it disappears next time you sync.

- **No Workspace Spec:** for monorepos and things of that nature, the Python ecosystem
would need a definition of workspaces. Today that does not exist which forces every
Expand Down
5 changes: 2 additions & 3 deletions rye/src/bootstrap.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ use once_cell::sync::Lazy;
use tempfile::tempdir_in;

use crate::config::Config;
use crate::piptools::LATEST_PIP;
use crate::platform::{
get_app_dir, get_canonical_py_path, get_python_bin_within, get_toolchain_python_bin,
list_known_toolchains,
Expand Down Expand Up @@ -154,8 +153,8 @@ pub fn ensure_self_venv_with_toolchain(
let uv_venv = uv.venv(&venv_dir, &py_bin, &version, None)?;
// write our marker
uv_venv.write_marker()?;
// update pip and our requirements
uv_venv.update(LATEST_PIP, SELF_REQUIREMENTS)?;
// update our requirements
uv_venv.update_requirements(SELF_REQUIREMENTS)?;

// Update the shims
let shims = app_dir.join("shims");
Expand Down
Loading

0 comments on commit 2c69050

Please sign in to comment.