Build failures with non-workspace path deps

[andrewbaxter]

Describe the bug
Projects with path deps with relative paths won't build. This is the same issue as here: nix-community/naersk#133 .

That issue has a community workaround, but it doesn't work with crane for some reason (tries to reference path project/Cargo.toml/../. which isn't a valid path I guess) and it'd be great if there were an official solution.

Reproduction
I don't have a flake, but with crate_a and crate_b where crate_a has dependency:

crate_b = { path = "../crate_b" }

and build

crane.buildPackage {
    src = ./crate_a;
}

will produce the error.

[ipetkov]

Hi @andrewbaxter thanks for the report!

When you specify src = ./crate_a; you are instructing Nix to add an entry to the store with the contents of the crate_a directory and then use that as the source for the derivation. That means when the builder runs inside of the sandbox, it can only see the contents of crate_a (recursively), but it cannot access any other directories outside of that. At this point there is nothing crane (or naersk for that matter) can do to circumvent the sandbox.

Even if it could, the Cargo.toml will contain a ../crate_b entry which, inside of the sandbox, will attempt to get resolved as /nix/store/<hash>-source-of-crate_a/../crate_b which will simplify to /nix/store/crate_b which is never going to work out.

You have several options to remedy this:

1. Use some build hooks to copy over the additional sources into the builder's sandbox and then patch all ../crate_b references to point there, as discussed in the linked issue.
2. Change the source to something like src = ./.; (i.e. some directory that is a superset of both crate_a and crate_b) and then change the builder's working directory to be inside of crate_a (or adjust your cargo flags to point it there)
3. Publish the crate to a registry and change the path dependency to a "regular" dependency and let dependency vendoring do its job
4. Change the path dependency to a git dependency and let the dependency vendoring do its job

Hope this helps!

[andrewbaxter]

Thanks, I appreciate the reply!

Publishing isn't an option for various reasons. For one, the purpose of using path dependencies is to avoid publishing (e.g. for local development).

Could you elaborate on thy there's nothing crane could do? Couldn't it do what the workaround does? AFAIK Crane is already rewriting the Cargo.toml to some extent, this doesn't seem like a stretch.

[ipetkov]

AFAIK Crane is already rewriting the Cargo.toml to some extent, this doesn't seem like a stretch.

We "clean" the Cargo.toml when it comes to the buildDepsOnly derivation by stripping out any obviously irrelevant portions (to minimize cache invalidation) but we otherwise do not rewrite anything regarding dependencies.

Could you elaborate on thy there's nothing crane could do?

When it comes to evaluation purity, there's no good way to attempt to operate on files "outside" of the source input that's passed in. Even if there is a way to do that (and say, detect path dependencies outside of the current src parameter), you get into other tricky situations like deciding what contents should or shouldn't be cleaned to avoid unnecessary rebuilds. At the end of the day, it's vastly simpler for the project author to make a choice about how to lay out their dependencies than it is for crane to try to come up with a comprehensive, general build solution while also allowing for any kind of configuration customization and avoiding an incredibly complex API...

I'd recommend checking out the quick-start-workspace example: it shows how you can use lib.fileset to union several different subdirectories (e.g. ./crate_a and ./crate_b) and have everything show up in the builder sandbox without a lot of manual patching