Upstream build.rs attempts to create a directory

[mkatychev]

Git repo reproducig issue: https://github.com/mkatychev/crane_build_rs

Hello, I am currently trying to build a workspace package that depends on a
crate that attempts to do file generation with protobuf:

[package]
name = "my-package"
version.workspace = true
edition.workspace = true

[[bin]]
path = "src/main.rs"
name = "my-app"

[dependencies]
git-package = { git = "https://github.com/mkatychev/crane_build_rs", rev = "02b264f" }

Current problem

Since crane creates derivations of vendored dependencies without triggering
upstream build.rs files in
craneLib.vendorCargoDeps:
my flake returns a build error when git_crate's build.rs tries to call mkdir inside
of /nix/store:

// git_crate/build.rs
fn main() -> std::io::Result<()> {
    std::fs::create_dir("./src/gen")?;
    Ok(())
}

error: failed to run custom build command for `git-package v0.1.0 (https://github.com/mkatychev/crane_build_rs?rev=0>

Caused by:
  process didn't exit successfully: `/private/tmp/nix-build-my-package-deps-0.1.0.drv-0/source/target/release/build/>
  --- stderr
  Error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }

Current workaround

My current workaround involves a preBuild hook that overrides vendoring for
my specific cargo dependency since the vendor phase does not store the
generated files:

# my-workspace/flake.nix
{
  description = "build.rs woo";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    crane = { url = "github:ipetkov/crane"; inputs.nixpkgs.follows = "nixpkgs"; };
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, crane, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # libs
        pkgs = import nixpkgs { inherit system; };
        inherit (pkgs) lib;
        craneLib = crane.lib.${system};

        myPackage = craneLib.buildPackage {
          inherit src cargoArtifacts;
          pname = "my-package";
{
  description = "my-package";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    crane = { url = "github:ipetkov/crane"; inputs.nixpkgs.follows = "nixpkgs"; };
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, nixpkgs, crane, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        # libs
        pkgs = import nixpkgs { inherit system; };
        inherit (pkgs) lib;
        craneLib = crane.lib.${system};

        myPackage = craneLib.buildPackage {
          src = ./.;
          pname = "my-package";

          buildInputs = [ pkgs.protobuf ];
          doCheck = true;
            # workaround
          preBuild = ''
            cargo add git-package --git https://github.com/mkatychev/crane_build_rs \
              --package my-package --rev cfc475c
          '';
          cargoExtraArgs = "--package my-package";
        };

      in
      {
        # $ nix flake check
        checks = {
          inherit myPackage ;
        };

        packages = { default = myPackage; };

      });
}

Is there a better way to handle this issue?
Ideally I'd like to cache a /nix/store/cargo-git/git-package-0.1.0/
derivation with the src/gen directory and files already generated so that
git-package can be cached in craneLib.buildDepsOnly.

I've seen several other workarounds that deal with permissions and build.rs
scripts, but they don't seem to accomodate my issue:
#203 (comment)
#249 (comment)

[mkatychev]

Having doCheck = true in cargoArtifacts still seems to produce the mkdir issue:

# dependency caching
cargoArtifacts = craneLib.buildDepsOnly {
  inherit src;
  pname = "my-workspace";

  buildInputs = [ pkgs.protobuf ];
  doCheck = true;
  preBuild = ''
    export HOME=$(mktemp -d)
  '';
  # 3. disable `cargo test` in the checkPhase
  checkPhaseCargoCommand = "cargoWithProfile check";
};

myPackage = craneLib.buildPackage {
    inherit src cargoArtifacts;
    pname = "my-package";

    buildInputs = [ pkgs.protobuf ];
    doCheck = true;
    preBuild = ''
      export HOME=$(mktemp -d)
    '';
    cargoExtraArgs = "--package my-package";
    checkPhaseCargoCommand = "cargoWithProfile check";
};

[mkatychev]

@ipetkov one of my main issues currently is that derivations are vendored/symlinked then build.rs is called:

crane/lib/vendorMultipleCargoDeps.nix

Lines 63 to 67 in 4bdf559

	linkSources = sources: concatMapStrings
	(name: ''
	ln -s ${escapeShellArg sources.${name}} $out/${escapeShellArg name}
	'')
	(attrNames sources);

Is there any way to do the reverse: cargo vendor, then trigger vendored build.rs and commit derivation?

[mkatychev]

Hi @ipetkov, thanks for the reply.

I have a couple of concerns with $OUT_DIR being used to store generated rust code:

• If multiple dependencies use code generation, it can make reasoning about module structure unintuitive:

  #[path = "/home/path/to/generated_client.rs"]
  pub mod client;

  The #[path ...] tag above is now being provided a non-relative filepath if $OUT_DIR is to be use for all output.
• I've generally found that avoiding creating generated code outside of src makes development very unergonomic due to how rust-analyzer does crate graph generation. I realize this one is a bit tangential but having "goto definition" be functional for generated code makes a big difference in development workflow. Unfortunately the issue linked above requires a non-trivial amount of refactoring (something I tried myself before I stumbled upon that issue) in rust-analyzer to produce a valid dependency graph at runtime.

The current crate graph building constraints of rust-analyzer and URI building of the #[path = ...] tag (which does not currently do well with envvars: #[path = "$OUT_DIR/generated_client.rs"] is not possible) make outputting to src the best choice if one wants to have a reasonable DX:

$ lst some_lib
some_lib
├── Cargo.toml
├── build.rs
└── src
   ├── gen
   │  ├── client.rs
   │  └── messages.rs
   ├── gen.rs
   └── lib.rs

// src/lib.rs
pub mod gen

pub fn new_client() -> gen::client::Client {
    gen::client::Client::new()
}

// src/gen.rs

// src/gen/client.rs
pub mod client
// src/gen/messages.rs
pub mod messages

It still seems to me that allowing omissions to the cargo vendor step would make crane far more flexible while accommodating code generation use cases such as mine that try to play nice with the current module graph limits.

[ipetkov]

@mkatychev the official cargo documentation states (emphasis mine)

Build scripts may save any output files or intermediate artifacts in the directory specified in the OUT_DIR environment variable. Scripts should not modify any files outside of that directory.

I'm actually surprised cargo doesn't extract crate sources to a read-only directory; you wouldn't want a single malicious build script to start editing the files of other crates...

The #[path ...] tag above is now being provided a non-relative filepath if $OUT_DIR is to be use for all output.

One way I've seen this solved is by having the build script tell cargo to set an environment variable which points to the appropriate module (e.g. println!("cargo:rustc-env=FOO_CRATE_CLIENT={}/src/client.rs", std::env::var("OUT_DIR").unwrap()); and then include that output via mod client { include!(env!("FOO_CRATE_CLIENT")); }. Rust analyzer should index those paths just fine

It's also worth noting that $OUT_DIR is unique per build-script, it's not just a single directory

It still seems to me that allowing omissions to the cargo vendor step would make crane far more flexible while accommodating code generation use cases such as mine that try to play nice with the current module graph limits.

I'm not sure I understand what you mean here, what is being omitted?

[mkatychev]

A lot of what you said makes sense, I will try the linked alternatives. Thanks for the references.

I'm not sure I understand what you mean here, what is being omitted?

I meant vendoring all dependencies except ones that do codegen, hence the omission.

[ipetkov]

Ah I see, even if we had some sort of flag to opt-into that behavior, knowing which crate does code gen is not easy (basically the halting problem). If you really wanted to avoid vendoring a crate you could explicitly pass in a Cargo.lock to crane which does not have the offending crate, though it would probably get unweidly

[mkatychev]

I'll keep this discussion open while I'm looking at an active idiomatic workaround for this issue. I can then submit a PR to document the workflow for upstream code generation since I'm not able to find concrete examples for this in the nix space.
Most libraries that deal with this in the nix space either:

• commit the generated code
• or do the include!(env!(... approach you've mentioned

[mkatychev]

Following the pattern in tikv/client-rust worked excellently.

I did write an additional macro that made it easier to deal with codegen that would add suffixes:

// src/gen.rs
// crate::gen;
macro_rules! gen_mod {
    ($package: ident) => {
        #[allow(clippy::all)]
        pub mod $package {
            include!(concat!(
                env!("OUT_DIR"),
                concat!("/", stringify!($package), ".rs")
            ));
        }
    };
    ($package: ident, $suffix: literal) => {
        #[allow(clippy::all)]
        pub mod $package {
            include!(concat!(
                env!("OUT_DIR"),
                concat!("/", stringify!($package), $suffix)
            ));
        }
    };
}

// crate::gen::generated_pkg;
gen_mod!(generated_pkg);
// crate::gen::generated_pkg_with_suffix;
gen_mod!(generated_pkg_with_suffix, "_api.v1.rs");