Long build times for exa crate

[mikosdikos]

So, I've finally decided to try out Rust but was rather completely disappointed.

I chose to build exa (the ls replacement) but it took a whopping 61 minutes to build it with Rust 1.5!

The machine is a rather beefy Celeron @400Mhz with 128MB of ram.
Moreover, the resulting stripped binary occupies 2MB of space versus 122kb of ls, the program it purports to replace.

Sorry to break it to you girls but with statistics like these you're not going to replace anything in the computing world unless you can fix this stuff ASAP!

But can it be done?

[White-Oak]

@mikosdikos I wonder, how much time does compilation of ls take on your machine?

[jonas-schievink]

Rustc is extremely memory hungry, are you sure you aren't hitting swap?

I mean, you're running rustc on a 400 MHz machine, so I wouldn't expect much, but an hour seems a bit excessive...

[mikosdikos]

Suprisingly it was using almost no swap - during linking (gold) it was getting full a few times but mostly it stayed between 30-50M. Anything over that was handled by memory compression so there was definitely no slowdown due to low memory.

Let me explain that the system uses about 10M after boot which leaves the full 120M free in console mode of course.

I'll try compiling ls from coreutils to answer the previous question.

[ticki]

@mikosdikos This is an issue that is already being worked on, see

• Incremental recompilation #2369,
• Incremental compilation RFC rfcs#1298,
• [RFC] Introduce a mid-level IR (MIR) in the compiler that will drive borrowck, trans rfcs#1211.

Other related things:

• RFC: Support Incremental Compilation rfcs#594,
• Indicate compilation progress in a form useful to end users #24335.

[ticki]

Also, when recompiling, it is much faster, due to crates being compilation units.

[jonas-schievink]

@ticky This is a clean build, it won't benefit from incremental compilation

[jonas-schievink]

I just noticed that exa has a ton of dependencies. In that case, 60 minutes isn't too far off from what I'd expect on a system like this.

[mikosdikos]

All right, compiling the whole coreutils suite including man page generation took 7m30s and ls alone 25s (a simpler program obviously)

[mikosdikos]

Wait, are you saying that through sheer beginner's luck I actually hit a memory friendly compilation? Bugger me!

[ticki]

This is a clean build, it won't benefit from incremental compilation

Yeah, that's true. On the clean rebuilds: I don't think there is much, we can do. Compared to C, Rust does tons of work, which is not just optimizations. I hope the build time (not rebuild) can benifit from MIR, though.

[nikomatsakis]

I updated the issue title to be a bit more precise. Likely the excessive build times here are due to one of the already open issues, but it's hard to say for sure without more detailed investigation. Certainly we are working on improving compiler performance, yes in the incremental case but also in non-incremental scenarios (though a bit less actively at the moment).

[nodakai]

@mikosdikos Please have a look at https://www.rust-lang.org/faq.html#why-is-rustc-slow

But wait, the FAQ doesn't seem to answer why Rust binaries are generally larger than C/C++ counterparts. We should add it to the FAQ.

http://stackoverflow.com/questions/29008127/why-are-rust-executables-so-huge

[mikosdikos]

Ok, I've tried a few more projects out and it's obvious now I was indeed very lucky memory-wise.

Oh well, at least exa got that right!

[bstrie]

@mikosdikos , our work on MIR is about ready for widespread use, I'd be interested to see a comparison between compiling exa with and without MIR on your machine. To enable MIR, set the environment variable RUSTFLAGS=-Zorbit on the most recent nightly build of Rust.