Reduce compile (and test) time: Split FoxBox into many small crates

[JohanLorenzo]

Initial problem

Changing one single unit test takes about 30s to compile and execute; even if you changed a simple assertion.

Proposed solution

Split FoxBox into many small crates.

Reason: As infered above, FoxBox lives in a single crate. If one line changes (test included), the whole crate is recompiled, no matter if file is a deep dependency or not. A crate is a compilation unit. This means every module is merged within 1 file before being compiled. You can see the generated file by adding -Z unstable-options --pretty expanded to the rustc command line. As a consequence and if I understood correctly, you can only use multi-threading to compile the dependency of a crate, and not the crate itself.

Potential other solution

rustc has gotten a perf regression for about a year: rust-lang/rust#25069 . There has been no activity on this bug for the last 8 months, though.

Investigation details, and discarded suspects

Here below, the default steps are:

1. Compile a test build from scratch and run them (cargo test)
2. In one of the unit tests, add/change an assertion to a dummy one (i.e: assert!(true))
3. time cargo test
4. Repeat the steps a few more time to see the trend

The tests themselves migh take long to be executed

Comment out all the tests but one. That one should be simple, and redo the process detailed above. After 5 tries, the average time decreased from 30s to ±27s.

Stainless macros might take long to generate the tests structure

Take the simple test you had in the previous part. Transform it to use the standard way of define tests. E.g.:

#[test]
fn my_test() {
  assert!(true);
}

As a consequence, the time remains about the same as in the part above.

Unit tests might only use 1 thread

time cargo test -j4 takes the same time as if -j1 was used. After looking up, 1 crate is compiled in 1 thread.

Unit tests might not take long if they were not living along the production code

Fast way: Checkout this branch.

Longer way:

1. Comment out the last unit test
2. Rename src/main.rs to src/lib.rs
3. Create a file called tests/foo.rs
4. Import foxbox as an extern crate and use it in a simple test.
5. Do the regular steps. ⚠️ The build was less stable, I got the compiler crash at the first time. Then if I re-run it, the tests are correctly executed. Some warnings were showed. I believe no crash would happen if the warning were fixed.

For each test defined under tests/, a new binary is create. Then, cargo can use more than 1 thread. I'm not sure if this speeds up the compilation. Like said above, the compiler crashed the first time. When it happens, I remain at something around 24s.