Wasm-builder 3.0

[bkchr]

It's time for a new wasm-builder era :P

This pr adds the following notable changes:

• Instead of building all wasm binaries in one big workspace, we now build each wasm binary in its own project. Is this guy crazy? No! Why have I done this?

  1. It eliminates some problems with wasm-builder are being stuck: wasm-builder stalls build #7375. Before we always had to lock the whole workspace when doing any change to build a wasm binary. This is now gone.
  2. When updating the git reference, it could happen that the build was failing the first time because there was a race condition updating all the project of the wasm workspace: Collator build fails on the first run cumulus#174
  3. Because we don't do 1 anymore we can build all wasm binaries in parallel, this lead to a build time decrease in Polkadot (release build) of around 40-50 seconds on my machine. It may happens that on machines with a small number of cores, the compile time could increase because we now build the dependencies for the wasm binaries multiple times.

• The next big change is the removal of the wasm-builder-runner. This is achieved by using the unstable host_deps feature of cargo. Luckily we are using nightly anyway and being able to use this new feature. This simplifies the build process extremely, because we can include the wasm-builder directly as dependency and aren't forced anymore to do a special dependency handling for the wasm-builder in the build.rs file.

• wasm-builder and wasm-builder-runner are updated to 3.0.0 on crates.io when this pr is approved. While the later one will be an empty crate that just points on the former one saying that people now only need to use this one.

Fixes: #7375
Fixes: paritytech/cumulus#174

[pepyakin]

I hate to say it, but initial testing indicates that the build time increased from 13m 55s (bf78c16) to 14m 36s (63d7c09, the last commit of this PR at the moment). This is after cargo clean. My machine has I9-9880H, 8 cores (16 threads).

[bkchr]

I hate to say it, but initial testing indicates that the build time increased from 13m 55s (bf78c16) to 14m 36s (63d7c09, the last commit of this PR at the moment). This is after cargo clean. My machine has I9-9880H, 8 cores (16 threads).

What have you run? Just a cargo build?

[andresilva]

👍

[gui1117]

As far as I can see, code looks good.

there is still some occurences of SKIP_WASM_BINARY:

./primitives/runtime-interface/test-wasm-deprecated/src/lib.rs:29:/// Wasm binary unwrapped. If built with `SKIP_WASM_BINARY`, the function panics.
./frame/support/test/tests/reserved_keyword.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./frame/support/test/tests/derive_no_bound_ui.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./frame/support/test/tests/decl_module_ui.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./frame/support/test/tests/decl_storage_ui.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./client/executor/runtime-test/src/lib.rs:7:/// Wasm binary unwrapped. If built with `SKIP_WASM_BINARY`, the function panics.

[bkchr]

As far as I can see, code looks good.

there is still some occurences of SKIP_WASM_BINARY:

./primitives/runtime-interface/test-wasm-deprecated/src/lib.rs:29:/// Wasm binary unwrapped. If built with `SKIP_WASM_BINARY`, the function panics.
./frame/support/test/tests/reserved_keyword.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./frame/support/test/tests/derive_no_bound_ui.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./frame/support/test/tests/decl_module_ui.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./frame/support/test/tests/decl_storage_ui.rs:22:	std::env::set_var("SKIP_WASM_BINARY", "1");
./client/executor/runtime-test/src/lib.rs:7:/// Wasm binary unwrapped. If built with `SKIP_WASM_BINARY`, the function panics.

It seems I was drunk 🤦