Update Substrate

[nazar-pc]

Upstream changes I found relevant/important/interesting:

• Pub enum runtime to pub struct runtime: Pub enum runtime to pub struct runtime paritytech/substrate#13250
• Deprecate Weight::from_{ref_time, proof_size}: Deprecate Weight::from_{ref_time, proof_size} paritytech/substrate#13475
• Extract syncing protocol from sc-network: Extract syncing protocol from sc-network paritytech/substrate#12828
• sc-slots: Cleanup: sc-slots: Cleanup paritytech/substrate#13590
• consensus: remove caching functionality from block import pipeline: consensus: remove caching functionality from block import pipeline paritytech/substrate#13551
• Remove use of trait Store from all pallets and deprecate it.: Remove use of trait Store from all pallets and deprecate it. paritytech/substrate#13535
• Keystore overhaul: Keystore overhaul paritytech/substrate#13615
• Keystore overhaul (iter 2): Keystore overhaul (iter 2) paritytech/substrate#13634
• Keystore overhaul (final): Keystore overhaul (final) paritytech/substrate#13683
• ‼️ ‼️ ‼️ Deprecate Currency; introduce holds and freezing into fungible traits: Deprecate Currency; introduce holds and freezing into fungible traits paritytech/substrate#12951 ‼️ ‼️ ‼️
• chore: move genesis block builder to chain-spec crate.: chore: move genesis block builder to chain-spec crate. paritytech/substrate#13427

With this release LLVM version shouldn't matter anymore on macOS or other platforms.

A lot of moving parts have changed because of above PRs, I had to re-apply our Substrate tweaks on top of upstream and created https://github.com/subspace/substrate/tree/subspace-v4 branch for that. Changes were caused by networking refactoring, but after applying tweaks one by one it compiled and should work fine (I'm successfully syncing with this branch onto Gemini 3d).

Follow-ups required for:

• Offences report system rework: Offences report system rework paritytech/substrate#13425 (our offences are based on the same code, we can probably improve it in a similar way)
• Swap 'base58' with 'bs58': Swap 'base58' with 'bs58' paritytech/substrate#13739 (straightforward library swap)

Code contributor checklist:

• I have read, understood and followed contributing guide

[nazar-pc]

ORML docs didn't pass tests

[nazar-pc]

@liuchengxu @NingLin-P test failed with:

thread 'tests::test_executor_full_node_catching_up' panicked at 'The test took too long!', domains/client/domain-executor/src/tests.rs:28:1

https://github.com/subspace/subspace/actions/runs/4591054185/jobs/8106976976?pr=1347

Works for me locally though, must be performance sensitive?

[liuchengxu]

Works for me locally as well. It looks more like an interesting glitch somewhere, with some debugging in CI, I found that the ack of slot notification somehow never resolves, https://github.com/subspace/subspace/blob/0043ed4726/test/subspace-test-service/src/mock.rs#L182-L184, still working on it.

[shamil-gadelshin]

I had no local issues with tests.

[nazar-pc]

CI in #1348 (built on top of this) passed for some reason. Let's see if it happens again and if so we can merge it.

[nazar-pc]

Pulled in paritytech/substrate#13804

[ParthDesai]

Worked for me locally as well for eth relay domain.

[nazar-pc]

tests::invalid_execution_proof_should_not_work hangs for me locally sometimes when running with cargo nextest run, but not when running it individually.

Should I ignore it for now so we can make progress here?

[vedhavyas]

Nice work!

I think it is totally OK to skip the failing tests and let the Execution team handle it. If you can create an issue, would be great.

[liuchengxu]

@nazar-pc Let's ignore the failing test for now. I reproduced it locally with a busy machine but haven't got a clue.

@NingLin-P It can be reproduced with busy CPUs. For example, run openssl speed -multi 24 rsa, and then run the test indivisually. You may need to have multiple runs as it sometimes passes even with 100% CPU usage on my machine.

[NingLin-P]

@NingLin-P It can be reproduced with busy CPUs. For example, run openssl speed -multi 24 rsa, and then run the test indivisually. You may need to have multiple runs as it sometimes passes even with 100% CPU usage on my machine.

Thanks for the clue! I will look into it.

[nazar-pc]

Now test_executor_full_node_catching_up timed out, which is annoying. I guess tokio wrapper has some timeout for tests that we might need to increase. Will try to ignore this test too, but this is a really slow dev cycle to run it through CI every time.

Though judging by #1345 (comment) it just gets stuck and increasing timeout will not help.

[rahulksnv]

Thanks. The chain sync changes appear to be mostly code moved around, I will resync autonomys/substrate#46 after this is merged