ruff is currently built from source in the nix flake, slowing down dev environment creation and requiring a moving rust version

[matrixbot]

This issue has been migrated from #15939.

---

We have a flake.nix file which can be used to create a full development environment for Synapse, including python and native dependencies and things like postgres and redis.

The development environment will automatically keep its python environment up to date with poetry using the versions defined in pyproject.yaml. One of those python package is ruff, which we use as a linter.

ruff ships a rust binary, which is dynamically linked against either glibc or musl. As such, ruff expects glibc or muslc headers to exist at a certain location on your Linux box. Due to the way NixOS works, these do not exist under the expected path, causing any ruff binaries we download from PyPI to fail to execute.

Thus, we have the following workaround in flake.nix, which forces ruff to be compiled on the local system:

https://github.com/matrix-org/synapse/blob/20ae617d1417f8dd52e20b3a20cb01b4c2fd87c9/flake.nix#L115-L122

This works, but is slow (as you need to compile a rust binary) and forces the development environment to maintain a rust version which is high enough to compile ruff (which unfortunately is almost always the latest stable. I found today that while we only require rust 1.60.0 in Synapse, the version of ruff we have specified in our pyproject.toml requires rust 1.70.0.

The ideal solution here is for ruff to provide statically linked binaries to download. This has been discussed in astral-sh/ruff#1699, and they are up for it if it does not compromise performance (or alternatively they could provide a ruff-static PyPI package.

Until then, we can't do much on this side.

[haslersn]

@anoadragon453 what about patching the dynamically linked binary with patchelf?

[anoadragon453]

@haslersn Oh that's a good idea, and not something that we've tried yet.

[anoadragon453]

I've had a play with this and can confirm that patchelf works. However, I want to run it automatically after the poetry venv is built, and ran into a roadblock there: cachix/devenv#1557

Hopefully once that's sorted out we can eliminate the need to build ruff from source altogether, greatly speeding up dev environment building!