refactor!: black-box integration tests

[0x009922]

Re-opening of #5087

Build on top of, and should be undrafted after merging of:

• fix(iroha_config): broken trusted peers check #5121
• refactor(iroha_torii): spawn a single server task with only one TcpListener #5122
• build(wasm_samples): pre-compiled wasm_samples #5123

---

Context

iroha_test_network has a few problems:

• It doesn't test Iroha as a black box, but runs directly using internal implementation details.
• Execution flow of peers is messy, hard to debug & control. (e.g. for cases of restarts). Logs from all peers are all over the place, very noisy.
• Messy API: hard to fine tune the network & peers for specific cases; high coupling with defaults; thread::sleeps with pipeline time instead of providing precise lifecycle hooks
• Non-optimal defaults: most of the tests don't need 4 peers with 4s of pipeline time. A single peer with close-to-zero block time works significantly faster.

Black boxing would mean to run Iroha through its CLI as a dedicated process, just as users would do.

Solution

Re-implement iroha_test_network:

• Black-boxing: run irohad as a direct child process. Close Proper black-box integration tests MVP #4500
  • Create temp dir for every randomly generated peer and store there its state, configs, and logs.
  • Configuration through raw TOML - feel your users! Close Refactor config usage in integration tests #4383
  • Can work with any irohad target (e.g. debug, release)
• Allocate ports automatically (fslock-based solution) - no more manual ports setting. Works intra-process (cargo test) and inter-process (cargo nextest Use Nextest #4987).
• Higher-level API suitable for writing expressive tests.
• Optimal defaults: a single peer with very short timings.
• Faster!
• Bonus: test execution can now be terminated immediately with SIGINT (Ctrl + C). It used to be suspended.

Other changes:

• Rewrite many tests, reduce flakiness to the minumum. Turns out some were broken, but passing for other reasons. It was primarily a cause of a messy test network API. Partially address Optimise CI - reduce flakyness, unify workflows #4516
• Remove unstable_network tests: they rely on a direct violation of black boxing - the use of FreezeStatus to make peers faulty. To be re-implemented in some other way.
• Make irohad a closed binary; don't expose Iroha; remove samples from it. Close [suggestion] Refactor Iroha CLI #4136
• Minor changes in iroha_core.

Further steps

• [suggestion] Fully asynchronous client API #3130 is a must
• Make iroha_test_network an executable, use in pytests and in SDKs
• Move tests of business logic from integration to iroha_core itself.

Flaky tests

• restarted_peer_should_have_the_same_asset_amount - possibly due to a bug in Iroha? Read FIXME comment there.
• extra_functional::connected_peers::* sometimes fail due to [BUG] Sumeragi panics with "index out of bounds" message after unregistering a peer #5104

Migration Guide

For devs: you need to have irohad available in $PATH before running tests using iroha_test_network. The easiest way to achieve this is by running:

cargo install --path crates/irohad --locked

---

Review notes

Due to Client still being blocking, there is some ugly code with spawn_blocking.

Checklist

• I've read CONTRIBUTING.md.
• (optional) I've written unit tests for the code changes.
• All review comments have been resolved.
• All CI checks pass.
• Upload test network artifacts

[0x009922]

test_network_runs artifact takes 70mb. Is this okay?

[0x009922]

Self-review