feat: add a uv backend

[henryiii]

UV just came out from the Ruff folks, and it's insanely fast - it makes venvs faster (25ms) than Python can start up (50ms). And much faster than virtualenv (400ms) and venv (nearly 5 seconds). The package install speeds are also unreal compared to pip.

There are some differences - namely name @ . is required instead of . if an install is not editable (for now). But it's really nice to try it out!

0001-1698.mp4

Nox's docs build in 4 seconds instead of 22 seconds using the uv backend.

[henryiii]

I pulled your changes in here, thanks @alex! (Nice username, by the way!)

[alex]

Happy to help! Looking forward to this.

[henryiii]

I'm thinking a nice followup could be to allow fallback backends. So uv|virtualenv would use UV if available, and fallback to virtualenv if not available. Same with mamba|conda.

[henryiii]

Is there a way to ignore a line from coverage only on Python 3.7?

[edgarrmondragon]

Is there a way to ignore a line from coverage only on Python 3.7?

Maybe https://github.com/asottile/covdefaults?tab=readme-ov-file#version-specific--pragma-no-cover?

[alex]

I don't believe there's a way to skip coverage by python version with just coverage.py. I think you have two choices:

1. #pragma: no cover the line always
2. Install uv outside the virtualenv, with the system python interpreter, so that it's on the $PATH in CI.

[henryiii]

@FollowTheProcess, @DiddiLeija, @cjolowicz, or @theacodes, I think this is ready. Also, there are quite a few other PRs that are basically ready and useful to have open too!

[henryiii]

Also, Tox added UV support yesterday: https://github.com/tox-dev/tox-uv

[cjolowicz]

Thank you @henryiii and @alex 🎉

I'll let this sit for a day to give the other maintainers a chance to chime in.

[slafs]

Fantastic work! Thank you all! My 3 cents:

1. [minor] The uv option is not mentioned in the help text of --default-venv-backend nor --force-venv-backend CLI options.
2. It's also not mentioned in the docs for those options ("Changing the sessions default backend" and "Forcing the sessions backend" in usage.rst).
3. Speaking of docs, would it make sense to add a note that uv venv does NOT install pip by default? Unlike venv or virtualenv. I've run into a (rather obscure) caveat when trying out this branch. Namely you can't switch backends from uv (fresh) to venv (re-use) without re-creating the whole virtual environment, because pip is missing. It's OK to go from venv (fresh) to uv (re-use), though. Again, this is probably a non-issue in a day-to-day usage and only relevant when exploring backends or seeing that ridiculous speed difference 😅. Nevertheless, I feel like it wouldn't hurt to mention that in the docs.

Also, thanks @skshetry for pointing out that bug in uv 😬.

I'm thinking a nice followup could be to allow fallback backends. So uv|virtualenv would use UV if available, and fallback to virtualenv if not available. Same with mamba|conda.

Yes! 💯 That would be great!

[edgarrmondragon]

Suggested change

	"virtualenv",
	"virtualenv",
	"--seed",

See point 3 in #762 (comment)

[henryiii]

IMO, there's no need for pip (or setuptools or wheel for Python <3.11) in the environments - since we have .install instead of manually interacting with pip (hurray!), we don't have to guarantee pip is present (especially since this is opt-in; if we change the default to use uv if present, then we could install pip for back-compat purposes, maybe). Someone can install with session.install("pip").

[henryiii]

(I think the issue in 3. is still solvable, by the way, another way)

[henryiii]

I kind of forgot about docs because I was initially just making it work, will add!

[FollowTheProcess]

@henryiii nice work! Sorry been a bit swamped recently and haven't had much time to look at Nox too much. I think once the docs get an update calling out the "no pip" behaviour as mentioned above then I'm happy and excited to get this in 👍🏻

[henryiii]

@slafs: Thanks! I've addressed (1) and (2). I think I also fixed (3) by detecting uv created venvs, would be great if you could check!

[alex]

Thank you for following through on this!

Is there a good place to follow along in terms of when this will be in a release?

[henryiii]

Watch -> Custom -> releases (in the GH UI) is what I do. You can also get GitHub Releases via RSS, though I don't use that much anymore.

I think we have a few small followups (rebuild on env change & environment fallback, maybe #776), and we might want #768 as I'd like the action to work on the new AS runners, but after that I think @theacodes has said that a release is in the near future.

[alex]

Thanks

…

On Fri, Feb 23, 2024 at 12:19 PM Henry Schreiner ***@***.***> wrote: Watch -> Custom -> releases (in the GH UI) is what I do. You can also get GitHub Releases via RSS, though I don't use that much anymore. I think we have a few small followups (rebuild on env change & environment fallback, maybe #776 <#776>), and we might want #768 <#768> as I'd like the action to work on the new AS runners, but after that I think @theacodes <https://github.com/theacodes> has said that a release is in the near future. — Reply to this email directly, view it on GitHub <#762 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBBJ6VK4G7NC3RAIG4TYVDFSPAVCNFSM6AAAAABDMYCT5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRRG4YDONZXGQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[henryiii]

FYI, release is out (including on PyPI and homebrew). I've been setting NOX_DEFAULT_VENV_BACKEND=uv and it's so fast. It's also teaching me where I unknowingly had dependencies on pip and need to list it explicitly.

After the new release is out for a bit I may start moving over to the new uv|virtualenv syntax.

[alex]

Thank you!

…

On Mon, Mar 4, 2024 at 11:48 AM Henry Schreiner ***@***.***> wrote: FYI, release is out (including on PyPI and homebrew). I've been setting NOX_DEFAULT_VENV_BACKEND=uv and it's *so* fast. It's also teaching me where I unknowingly had dependencies on pip and need to it listed explicitly. After the new release is out for a bit I may start moving over to the new uv|virtualenv syntax. — Reply to this email directly, view it on GitHub <#762 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBE2O77KAZL5FHD3TNLYWSQWZAVCNFSM6AAAAABDMYCT5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNZXGAZDQNZVGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- All that is necessary for evil to succeed is for good people to do nothing.

[pfmoore]

Looking at this feature now it's been released, I have a couple of comments.

1. The uv backend isn't mentioned in the documentation of the session API. It took me a while to find the information in the "Usage" section.
2. While it's noted that uv is fast, and that it doesn't install pip by default, it's not obvious from the way that the documentation covers this, that a significant part of the speed gain is because pip isn't installed. By adding a venv_params argument of --without-pip (for venv or virtualenv) you get much of the speed gain (along with the "pip isn't installed" disadvantage, of course), while still keeping the (currently) more compatible standard behaviour otherwise.

[cjolowicz]

I've been wondering if we shouldn't support using an external pip in a more seamless way.

[theacodes]

I'd be fine with that.

…

On Wed, Mar 6, 2024 at 9:42 AM Claudio Jolowicz ***@***.***> wrote: I've been wondering if we shouldn't support using an external pip in a more seamless way. — Reply to this email directly, view it on GitHub <#762 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB5I47DKXZ73HFKRFETVGDYW4TNLAVCNFSM6AAAAABDMYCT5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBRGAZDGNBUG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[cjolowicz]

That said, IIUC every session.install would incur pip startup time twice, right?

[pfmoore]

That said, IIUC every session.install would incur pip startup time twice, right?

Does the uv backend use uv for session.install as well? That's not obvious from the description, where it's described as being a venv backend, not an installer backend. Although thinking about it now, it seems very reasonable to be that it's actually an "environment" backend in the broader sense of creating the environment and populating it (I've never used the conda backend, but I guess that works similarly).

I've been wondering if we shouldn't support using an external pip in a more seamless way.

Assuming there's a pip available in the environment nox is installed in, and it's sufficiently recent, you could easily use that pip (with the --python command line option) to do installs into the session. At that point, using --without-pip by default in virtualenv/venv created environments is likely to be possible without too much disruption.

I'd be willing to look at putting together a PR to do this, if there's interest. I've not contributed to the nox codebase before, so advice (in particular on the best user interface) would be welcome.

[henryiii]

Build 1.1 uses external pip if it can automatically. But of course, nox users can interact with pip. I’m wondering if the following would be seamless enough:

• Don’t install pip if pip is present and new enough in nox’s environment.

• Special case the installer (pip or uv) if it’s the first arg. Convert it to an absolute path and add any required args. I think conda does something like this for “python”.

Not installing pip only makes the setup faster, and it’s only about half a second for virtualenv (the default). That’s an absolute time, so great for simple environments, not as noticeable for big ones.

We have to pay the Python startup cost every Pip interaction due to no public API, also in build, etc. ;)

By the way, another speedup is uv doesn’t compile bytecode by default. For build, this makes sense, as you really just want to compile what you use and anything else is wasted. But not sure for nox. Most of the time, that probably is desired, but if you make a dev environment, it’s better to precompile. I’d want to measure the cost first, but it’s apparently enough that uv didn’t want to make it default.

[henryiii]

Yes, “venv-backend” also controls the installer. We didn’t have multiple installers before uv. Technically, you could mix and match, but it keeps the interface simpler. You can always install pip or uv in the other’s environment.

uv’s start up time is less than Python. In fact, it can finish making an empty venv before python -c “” can finish.

[henryiii]

Why twice?

[layday]

uv’s start up time is less than Python. In fact, it can finish making an empty venv before python -c “” can finish.

This difference is somewhere in the region of ~20 ms on my machine FWIW, which is outside the realm of the perceptible. python -m venv --without-pip completes in < 100 ms.

[layday]

Build 1.1 uses external pip if it can automatically

On the topic of build, readers might also be interested in pypa/build#751, adding support for uv.

[pfmoore]

Yes, “venv-backend” also controls the installer. We didn’t have multiple installers before uv. Technically, you could mix and match, but it keeps the interface simpler. You can always install pip or uv in the other’s environment.

Fair enough. The documentation doesn't make this clear, which was the original point I was trying to make, and it suggests that the environment creation gains with uv are because it's simply faster, rather than actually being because it doesn't install pip. Claiming "uv environment creation is faster" is a sore point for me, as I specifically implemented the --python flag for pip so that we could start the (very slow!) process of eliminating the need to install pip everywhere. So having uv "steal" the credit for that speedup is something I probably get over-sensitive about (they get to ignore the backward compatibility issue, which is the only reason the pip/venv ecosystem is taking things so slowly...)

I think that mixing the two aspects might have been a mistake, and having a way to configure install backends (which could be "uv", "external pip" and "internal pip", plus whatever conda options apply) independently of environment creation options would have made it easier for the trade-offs to be presented. In practice, I imagine no-one will care that much about environment creation times, apart from eliminating the cost of installing pip (as @layday said, any remaining cost differences are imperceptible), but they will care about the time vs compatibility trade-offs with installation.

Anyway, as I said I'd be willing to help separate the venv-backend and install-backend behaviours, and make using an external pip more convenient, if that's something that people are interested in. But I don't want to force that on people if there's going to be a lot of pushback on the UI aspects.

[layday]

To add, I don't know how nox creates venvs, but I assume you'd be importing the venv module directly, as opposed to spawning a subprocess for uv, which is bound to be more costly than whatever venv does internally without pip.

[henryiii]

as I specifically implemented the --python flag for pip so that we could start the (very slow!) process of eliminating the need to install pip everywhere.

FYI, @pfmoore, https://fosstodon.org/@henryiii/112039400189486890 / https://twitter.com/HenrySchreiner3/status/1764755669461148138 is entirely from the --python flag for pip. :)

[henryiii]

Nox uses virtualenv by default, though there is an option to use venv. And in both cases, it actually calls the command rather than using the API, so there's some savings possible there.

[pfmoore]

Yeah, thanks - those savings in build are awesome 🙂

To add, I don't know how nox creates venvs, but I assume you'd be importing the venv module directly, as opposed to spawning a subprocess for uv, which is bound to be more costly than whatever venv does internally without pip.

That's an interesting point. I did a benchmark:

    def with_venv(target):
        builder = venv.EnvBuilder()
        builder.create(target)

    def with_uv(target):
        subprocess.run(["uv", "venv", "--quiet", target])

Averaged over 100 runs, venv took 0.007626 seconds, and uv took 0.072377 seconds. So yes, there's a significant speed benefit from using venv (without installing pip) in-process.

That's on Windows with Python 3.12.0. On Lunix (Ubuntu under WSL) venv was 0.001832 sec, uv was 0.564309 sec. I'm going to assume that's something to do with WSL, as otherwise that's a terrible result... Although a docker container with Python 3.12 is just as slow - I don't have a bare Linux machine to test on that.

And in both cases, it actually calls the command rather than using the API, so there's some savings possible there.

Ouch, yes there is. Running venv in a subprocess takes 15 times longer than running it in-process.

Here's my benchmarking script, for what it's worth: https://gist.github.com/pfmoore/a5406e0901609ed08cab8a0719d4e866

[pfmoore]

If we had independent venv and install backends, I'd say that in-process venv would be fastest env creation backend in nearly all cases, with uv being the fastest install backend and pip --python being the most compatible. For cases where having pip installed in the environment is necessary, the existing virtualenv backend (that uses virtualenv's tricks to speed up installing the seed packages) is probably fastest, although in-process venv plus uv for installs, with an explicit uv install pip, might be faster if applicable.

As I said, there's trade-offs 😉

[pfmoore]

Doh. I'm incredibly dense. nox relies on being able to create virtual environments for arbitrary Python versions, so in-process venv creation isn't possible except in extremely limited cases. My apologies, most of what I've said above isn't of any practical use for this project (although IMO it's still interesting in the broader sense of how to create an environment as fast as possible).

[layday]

As am I. I was thinking of how it works in build 🥲

[cjolowicz]

@henryiii

Why twice?

Doesn't pip --python re-run itself on the interpreter in the environment?

[henryiii]

As am I.

Add me to the list. :)

Doesn't pip --python re-run itself on the interpreter in the environment?

Ah, yes, if that's how it works.

The documentation doesn't make this clear ... and it suggests that the environment creation gains with uv are because it's simply faster, rather than actually being because it doesn't install pip

I believe the documentation has a red warning box that says "The uv backend does not install anything by default, including pip, as uv pip is used to install programs instead."

Also, it doesn't mention anything about speed at all. Nothing is claimed to be faster and no advice is given on picking a backend. Even the changelog doesn't mention speed.

[pfmoore]

Also, it doesn't mention anything about speed at all.

Huh, you're right. I guess I simply took that from the initial comment here. My apologies - there's been a lot of hype about how fast uv is, and while it's not entirely wrong, I don't actually think it's the most important thing about uv. The fact that it's an alternative installer making different choices than pip does is more important, IMO.

My track record today is pretty bad, time to stop I think...