Enable incremental by default #4817

alexcrichton · 2017-12-13T22:13:28Z

This commit enables incremental compilation by default in Cargo for all
dev-related profiles (aka anything without --release or bench. A
number of new configuration options were also added to tweak how
incremental compilation is exposed and/or used:

A profile.dev.incremental field is added to Cargo.toml to disable
it on a per-project basis (in case of bugs).
A build.incremental field was added in .cargo/config to disable
globally (or enable if we flip this default back off).

Otherwise CARGO_INCREMENTAL can still be used to configure one
particular compilation. The global build.incremental configuration
cannot currently be used to enable it for the release profile.

rust-highfive · 2017-12-13T22:13:31Z

r? @matklad

(rust_highfive has picked a reviewer for you, use r? to override)

alexcrichton · 2017-12-13T22:14:26Z

Note that this is intended to land roughly the same time we rename -Z to -C for the incremental option but I haven't done that quite just yet to ensure that this still passes CI. We'll probably want to hold off on landing until that's ready to go on the rust-lang/rust side of things.

cc @michaelwoerister @nikomatsakis

matklad

Let's add tests for profile and .cargo/config configuration?

A question I'd like to ask is where this option indeed belongs to profile? My understanding here is that it indeed does, because incremental compilation could change the resulting artifact (i.e, LLVM may make different optimization decisions because of incremental).

Also, 🎉 🎊 🎆 🌠 🍒 🎉 🍕 🍍 !!!

matklad · 2017-12-13T22:16:40Z

src/cargo/ops/cargo_rustc/context.rs

@@ -157,21 +158,11 @@ impl<'a, 'cfg> Context<'a, 'cfg> {
        // Enable incremental builds if the user opts in. For now,
        // this is an environment variable until things stabilize a
        // bit more.


Comment can be deleted to avoid causing confusion.

matklad · 2017-12-13T22:19:18Z

src/cargo/ops/cargo_rustc/context.rs

+        //   globally for a system configure whether incremental compilation is
+        //   enabled. Note that setting this to `true` will not actually affect
+        //   all builds though but instead only those profiles which would
+        //   otherwise default to `true`. This can be useful to globally


Note that setting this to true .. only those profiles which would otherwise default to true.

Should any of this say false instead?

Hm sorta, I reworded to hopefully make more sense.

matklad · 2017-12-13T22:27:16Z

Oh, we might want to check how this interacts with cargo check just in case something gets broken.

alexcrichton · 2017-12-14T05:08:07Z

@matklad

Let's add tests for profile and .cargo/config configuration?

Certainly! Should be done now.

A question I'd like to ask is where this option indeed belongs to profile?

Oh yeah incremental definitely affects compilation artifacts so we need to recompile whether it's on or off.

matklad · 2017-12-14T09:34:24Z

src/cargo/ops/cargo_rustc/context.rs

+        //   all builds though. For example a `true` value doesn't enable
+        //   release incremental builds, only dev incremental builds. This can
+        //   be useful to globally disable incremental compilation like
+        //   `CARGO_INCREMENTAL`.


Hm, so the value true for build.incremental does not change anything, because true && default is just default.

So, the only usage for this flag is to disable incremental compilation. So perhaps we can rename the flag to build.disable_incremental and allow only true value for it?

Should this setting override even explicit profile incremental setting? If the primary use-case here is to work-around the possible bugs in incremental compilation, then I think a reliable way to turn off all incremental would be more valuable.

Yeah this was mostly just meant to mirror CARGO_INCREMENTAL as a "global configuration option" to disable it everywhere (I don't think enabling it everywhere makes much sense for release profiles for example).

I personally prefer incremental at least though to disable-incremental in that it's a bit more clear (even if incremental = true doesn't actually do anything).

I think that makes sense as well yeah for .cargo/config to override profiles, I'll tweak that

lilianmoraru · 2017-12-18T08:06:13Z

So, the only usage for this flag is to disable incremental compilation. So perhaps we can rename the flag to build.disable_incremental and allow only true value for it?

From a user's perspective(my personal perspective) that doesn't know what the code does behind, incremental just seems like the more obvious option that would enable or disable incremental compilation and of course, it is more in line with CARGO_INCREMENTAL.

alexcrichton · 2017-12-21T15:18:42Z

Ok now that we've got a -C option upstream, re-r? @matklad

matklad · 2017-12-21T15:20:01Z

@alexcrichton should we add some docs here?

alexcrichton · 2017-12-21T15:23:34Z

Seems plausible yeah, I've added some docs for the config key, env var, and profile option. I don't think we'll want to highlight it too much though (aka explain at length what the defaults are) as hopefully everyone's just going to see faster compiles :)

matklad · 2017-12-21T15:35:08Z

src/cargo/ops/cargo_rustc/context.rs

+        let global_cfg = self.config.get_bool("build.incremental")?.map(|c| c.val);
+        let profile = match unit.profile.incremental {
+            Incremental::Explicit(v) => v,
+            Incremental::Default(default) => default && global_cfg.unwrap_or(default),


Ok, I am staring at this boolean logic for like ten minutes, I can't persuade myself if it is right or wrong, could you recheck it just in case? :)

Here are my doubts:

I believe && global_cfg.unwrap_or(default) condition is not not necessary here. That is, if you remove it, the code would behave exactly the same.

if global cfg is not set, then default && global_cfg.unpwrap_or(default) == default && default == default

if global cfg is true, then default && global_cfg.unwrap_or(default) == default && true == default.

if global cfg is false, than we'll bail out on the next like, without inspecting profile at all.

!self.incremental_env.unwrap_or(global_cfg.unwrap_or(profile))

So, if global_cfg is true, we do incremental regradless of profile setting? This is not what the doc comment says?

Oh oops, good catch!

The default here was intended to affect global_cfg, ensuring that if the default is false and the global cfg says true, we still turn it off. I'll update

matklad · 2017-12-21T17:16:09Z

src/cargo/ops/cargo_rustc/context.rs

+        };
+        if !self.incremental_env.unwrap_or(global_cfg.unwrap_or(profile)) {
+            return Ok(Vec::new())
+        }


This is probably just nitpicking, but I don't find this "nah, let's make global_cfg mutable" approach easy to follow :)

May I suggest the following match instead?

let incremental = match (self.incremental_env, global.cfg, unit.profile.incremental) { (Some(v), _, _) => v, (None, Some(false), _) => false, (None, _, Incremental::Explicit(v)) | (None, _, Incremental::Default(v)) => v, }; if !incremental { return Ok(Vec::new()) }

And looks like we perhaps we can get rid of distinction between Incremental::Explicit and Incremental::Default?

Heh excellent points! Updated now!

matklad · 2017-12-21T18:37:04Z

@bors r!

Excellent! 🚀

matklad · 2017-12-21T18:37:29Z

@bors r+

Robots are too unemotional :(

bors · 2017-12-21T18:37:30Z

📌 Commit 475fa58 has been approved by matklad

bors · 2017-12-21T18:37:36Z

⌛ Testing commit 475fa58115a615acc46966b60b69f4f13e14c05c with merge 203edeaf3cf3329611d7de5f2b71c158ea51d93f...

bors · 2017-12-21T18:52:04Z

💔 Test failed - status-travis

matklad · 2017-12-21T18:54:08Z

Test failure seems legit!

https://travis-ci.org/rust-lang/cargo/jobs/319850166#L1162

This commit enables incremental compilation by default in Cargo for all dev-related profiles (aka anything without `--release` or `bench`. A number of new configuration options were also added to tweak how incremental compilation is exposed and/or used: * A `profile.dev.incremental` field is added to `Cargo.toml` to disable it on a per-project basis (in case of bugs). * A `build.incremental` field was added in `.cargo/config` to disable globally (or enable if we flip this default back off). Otherwise `CARGO_INCREMENTAL` can still be used to configure one particular compilation. The global `build.incremental` configuration cannot currently be used to enable it for the release profile.

alexcrichton · 2017-12-21T18:57:05Z

@bors: r=matklad

bors · 2017-12-21T18:57:06Z

📌 Commit 45cc30b has been approved by matklad

bors · 2017-12-21T18:57:12Z

⌛ Testing commit 45cc30b with merge e10c651...

Enable incremental by default This commit enables incremental compilation by default in Cargo for all dev-related profiles (aka anything without `--release` or `bench`. A number of new configuration options were also added to tweak how incremental compilation is exposed and/or used: * A `profile.dev.incremental` field is added to `Cargo.toml` to disable it on a per-project basis (in case of bugs). * A `build.incremental` field was added in `.cargo/config` to disable globally (or enable if we flip this default back off). Otherwise `CARGO_INCREMENTAL` can still be used to configure one particular compilation. The global `build.incremental` configuration cannot currently be used to enable it for the release profile.

bors · 2017-12-21T19:50:35Z

☀️ Test successful - status-appveyor, status-travis
Approved by: matklad
Pushing e10c651 to master...

aidanhs · 2018-01-09T23:58:51Z

As a point of interest, some (single binary) projects I'm looking at see a ~10% increase in the size of the target directory to store incremental things (~150MB in incremental dir, ~1.1GB overall).

Projects with more binaries will presumably see more impact, and things like crater (which build every single crate individually) see approximately a doubling of target dir size.

michaelwoerister · 2018-01-10T16:33:40Z

Good to know, @aidanhs! A doubling of size is to be expected since we are basically keeping around a second version of the program in the incr. comp. cache. If it were much more than double, then that would be an indicator that cache garbage collection doesn't work as expected.

rust-highfive assigned matklad Dec 13, 2017

matklad reviewed Dec 13, 2017

View reviewed changes

matklad added the relnotes Release-note worthy label Dec 13, 2017

alexcrichton force-pushed the incremental-by-default branch 2 times, most recently from 26cfb4f to ea3b394 Compare December 14, 2017 05:07

matklad reviewed Dec 14, 2017

View reviewed changes

alexcrichton force-pushed the incremental-by-default branch from ea3b394 to e0c822a Compare December 14, 2017 16:55

alexcrichton force-pushed the incremental-by-default branch from e0c822a to 1adb287 Compare December 21, 2017 15:18

alexcrichton force-pushed the incremental-by-default branch 2 times, most recently from b71594e to cf20cca Compare December 21, 2017 15:23

matklad reviewed Dec 21, 2017

View reviewed changes

alexcrichton force-pushed the incremental-by-default branch from cf20cca to f75d234 Compare December 21, 2017 16:50

matklad reviewed Dec 21, 2017

View reviewed changes

alexcrichton force-pushed the incremental-by-default branch from f75d234 to 475fa58 Compare December 21, 2017 18:22

alexcrichton force-pushed the incremental-by-default branch from 475fa58 to 45cc30b Compare December 21, 2017 18:56

bors merged commit 45cc30b into rust-lang:master Dec 21, 2017

SimonSapin mentioned this pull request Jan 4, 2018

Per-target profiles? #4897

Open

wking mentioned this pull request Jan 8, 2018

Delete the old docs, lift up the new #4904

Merged

alexcrichton deleted the incremental-by-default branch January 25, 2018 16:29

kennytm mentioned this pull request Jan 30, 2018

25% regression in serde compile time rust-lang/rust#47628

Closed

phansch mentioned this pull request Feb 20, 2018

Explain how to execute a single UI test rust-lang/rust-clippy#2459

Merged

abonander mentioned this pull request Mar 19, 2018

ICE building panic_unwind (stage 1) in debug mode with incremental compilation rust-lang/rust#49153

Closed

bjorn3 mentioned this pull request Mar 19, 2018

the compiler unexpectedly panicked. this is a bug. rust-lang/rust#49113

Closed

pgerber mentioned this pull request Mar 19, 2018

Travis/Appveyor: disable incremental compilation rusoto/rusoto#1013

Merged

marco-c mentioned this pull request Jun 18, 2018

-Zprofile broken in nightly? rust-lang/rust#50203

Closed

chris-morgan mentioned this pull request Nov 18, 2019

Ignore file lock errors if unsupported, on Windows #7602

Merged

jan-auer mentioned this pull request Jun 17, 2020

ci: Start using github actions for CI getsentry/symbolicator#235

Merged

3 tasks

Mazwak mentioned this pull request Nov 18, 2020

Optimize release ajour/ajour#364

Closed

ehuss added this to the 1.24.0 milestone Feb 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable incremental by default #4817

Enable incremental by default #4817

alexcrichton commented Dec 13, 2017

rust-highfive commented Dec 13, 2017

alexcrichton commented Dec 13, 2017

matklad left a comment

matklad Dec 13, 2017

matklad Dec 13, 2017

alexcrichton Dec 14, 2017

matklad commented Dec 13, 2017

alexcrichton commented Dec 14, 2017

matklad Dec 14, 2017

alexcrichton Dec 14, 2017

lilianmoraru commented Dec 18, 2017

alexcrichton commented Dec 21, 2017

matklad commented Dec 21, 2017

alexcrichton commented Dec 21, 2017

matklad Dec 21, 2017

alexcrichton Dec 21, 2017

matklad Dec 21, 2017

alexcrichton Dec 21, 2017

matklad commented Dec 21, 2017

matklad commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

matklad commented Dec 21, 2017

alexcrichton commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

aidanhs commented Jan 9, 2018

michaelwoerister commented Jan 10, 2018

Enable incremental by default #4817

Enable incremental by default #4817

Conversation

alexcrichton commented Dec 13, 2017

rust-highfive commented Dec 13, 2017

alexcrichton commented Dec 13, 2017

matklad left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad commented Dec 13, 2017

alexcrichton commented Dec 14, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lilianmoraru commented Dec 18, 2017

alexcrichton commented Dec 21, 2017

matklad commented Dec 21, 2017

alexcrichton commented Dec 21, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matklad commented Dec 21, 2017

matklad commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

matklad commented Dec 21, 2017

alexcrichton commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

bors commented Dec 21, 2017

aidanhs commented Jan 9, 2018

michaelwoerister commented Jan 10, 2018