Cargo uses configuration from $HOME/.cargo if $PWD is under $HOME even if $CARGO_HOME is set

[jonhoo]

Problem

I have an environment variable set using [env] in $HOME/.cargo/config.toml. For a particular project, I use a dedicated $CARGO_HOME with its own config.toml which sets that environment variable to a different value. However, I still observe that the value from my $HOME/.cargo/config.toml is used.

Steps

$ cargo new bad-env-sourcing
     Created binary (application) `bad-env-sourcing` package
$ cd bad-env-sourcing/
$ mkdir cargo-home
$ echo -e "[env.FOO]\nvalue='from-home'" >> $HOME/.cargo/config.toml
$ echo -e "[env.FOO]\nvalue='from-cargo-home'" > cargo-home/config.toml
$ cat src/main.rs
fn main() {
    println!("run: {}", env!("FOO"));
}
$ cat build.rs
fn main() {
    eprintln!("build-build: {}", env!("FOO"));
    eprintln!("build-run: {}", std::env::var("FOO").unwrap());
    println!("cargo:rerun-if-env-changed=FOO");
}
$ env CARGO_HOME=$PWD/cargo-home cargo run
   Compiling bad-env-sourcing v0.1.0 (/local/home/jongje/bad-env-sourcing)
    Finished dev [unoptimized + debuginfo] target(s) in 0.24s
     Running `target/debug/bad-env-sourcing`
run: from-home
$ cat target/debug/**/stderr
build-build: from-home
build-run: from-home

We can confirm that both ~/.cargo/config.toml and $CARGO_HOME/config.toml are loaded by adding an envvar just to the latter and seeing that it's set for build scripts and main code alike:

$ echo -e "[env.BAR]\nvalue='from-cargo-home'" >> cargo-home/config.toml
$ cat src/main.rs
fn main() {
    println!("run: {}", env!("FOO"));
    println!("run BAR: {}", env!("BAR"));
}
$ cat build.rs
fn main() {
    eprintln!("build-build: {}", env!("FOO"));
    eprintln!("build-run: {}", std::env::var("FOO").unwrap());
    println!("cargo:rerun-if-env-changed=FOO");
    eprintln!("build-build BAR: {}", env!("BAR"));
    eprintln!("build-run BAR: {}", std::env::var("BAR").unwrap());
    println!("cargo:rerun-if-env-changed=BAR");
}
$ env CARGO_HOME=$PWD/cargo-home cargo run
   Compiling bad-env-sourcing v0.1.0 (/local/home/jongje/bad-env-sourcing)
    Finished dev [unoptimized + debuginfo] target(s) in 0.53s
     Running `target/debug/bad-env-sourcing`
run: from-home
run BAR: from-cargo-home
$ cat target/debug/**/stderr
build-build: from-home
build-run: from-home
build-build BAR: from-cargo-home
build-run BAR: from-cargo-home

Possible Solution(s)

No response

Notes

When I build and run with a modified version of cargo that prints Config, I get, in part:

[src/bin/cargo/cli.rs:362] &config = Config {
    home_path: Filesystem {
        root: "/home/jongje/bad-env-sourcing/cargo-home",
    },
    ...
    env: {
        ...
        "CARGO_HOME": "/home/jongje/bad-env-sourcing/cargo-home",
        ...
    },
}
[src/cargo/core/compiler/compilation.rs:357] self.config.env_config()? = {
    "FOO": EnvConfigValue {
        inner: Value {
            val: WithOptions {
                value: "from-home",
                force: false,
                relative: false,
            },
            definition: Path(
                "/local/home/jongje/.cargo/config.toml",
            ),
        },
    },
    "BAR": EnvConfigValue {
        inner: Value {
            val: WithOptions {
                value: "from-cargo-home",
                force: false,
                relative: false,
            },
            definition: Path(
                "/home/jongje/bad-env-sourcing/cargo-home/config.toml",
            ),
        },
    },
}

I'm surprised that anything at all from $HOME/.cargo/config.toml is used when $CARGO_HOME is set. I would expect setting the latter to make Cargo forget all about $HOME/.cargo. We should probably make that be the case, and ensure it's not just for [env].

Version

cargo 1.63.0 (fd9c4297c 2022-07-01)
release: 1.63.0
commit-hash: fd9c4297ccbee36d39e9a79067edab0b614edb5a
commit-date: 2022-07-01
host: aarch64-unknown-linux-gnu
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.83.1-DEV (sys:0.4.55+curl-7.83.1 vendored ssl:OpenSSL/1.1.1n)
os: Amazon Linux AMI 2.0.0 [64-bit]

[jonhoo]

Ahh, I think I see what's happening here! Since my project is under $HOME, when Cargo walks up the directory hierarchy it finds $HOME/.cargo/config.toml just like it would find, say, /.cargo/config.toml, and loads that. And since configuration files encountered this way override the config from $CARGO_HOME, it prefers $HOME/.cargo/config.toml.

I... don't know whether this is a bug? It feels extremely unexpected — I suspect most people have their projects under $HOME somewhere. But at the same time Cargo is "doing the right thing". I guess I'll need to set something to stop Cargo from traversing further up the directory hierarchy...

[jonhoo]

Although it doesn't appear for there to be any way to configure Config::set_search_stop_path except if using Cargo as a library 🤔

[jonhoo]

Which I suppose means this ties into #9769, and specifically:

• Fine-grain control of Config.toml discovery #7887
• Add flag to ignore all parent directory configs #7621

[jonhoo]

It's real weird, because this means that setting CARGO_HOME (to somewhere under $HOME) not only doesn't stop using $HOME/.cargo/config.toml, it gives that file higher precedence than it normally would get!

[weihanglo]

I see it like: Once $HOME/.cargo is not $CARGO_HOME anymore, $HOME/.cargo is nothing special and then back to the normal config probing.

As you mentioned, there is a meta issue #9769 and other issues similar to yours, so I am going to close this. Feel free to call out if you think this should stay open. Thank you!

[jonhoo]

Yeah, I agree that this is "working as intended", though I'd still argue it's going to trip users up. I wonder if it's worth adding a warning when this situation is detected?

This also had me thinking — does this mean that if I have a project under $HOME/somewhere, $HOME/.cargo/config.toml actually gets applied twice? Once because it's in $CARGO_HOME and once through the regular directory traversal? Does that, in turn, mean that if I have, say /home/.cargo/config.toml, any settings in there will be overridden by the same setting being applied in $HOME/.cargo/config.toml (during the config directory traversal)?

[weihanglo]

Hmm… It should be skipped and only loaded once. The current behaviour is somehow incorrect IMO. I believe the correct behaviour should be:

• Config from CARGO_HOME is always at the lowest priority.
• If a path probed during directory traversal is equivalent to CARGO_HOME, Cargo skips it.

The current implementation seems not what I thought. If CARGO_HOME is effectively a path of the traversal, its priority stays at where it got found during the traversal, which is incorrect. It never goes down the least prior position.

PR #10736 "Add safe.directories config" solves this issue, though it has yet got merged.

https://github.com/rust-lang/cargo/pull/10736/files#diff-2798557959dfa91ff5779c9aabb6d536dec3219465e53ab7e101570bf2899468R1458-R1501