Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Fix the light client protocol protobuf schema #12732

Merged
merged 4 commits into from
Nov 18, 2022
Merged

Fix the light client protocol protobuf schema #12732

merged 4 commits into from
Nov 18, 2022

Conversation

tomaka
Copy link
Contributor

@tomaka tomaka commented Nov 18, 2022 •
edited
Loading

Fix #12727
Amends #12372

This PR fixes #12372 to do what it was intended to do. Right now the light client handler sends back an empty response in case of error, while what I intended was that it sends a response (most importantly, indicating the type of response) but without a proof in it. This is what this PR does.

This PR also switches the schema to protobuf 2.
In protobuf 2, each field is either optional or required. This is good. It's explicit. You know exactly what you get on the wire. Google, the best™ company in the world, then saw that sometimes we send one or two extra bytes that aren't strictly necessary, and, because they have too many engineers and too much time, decided that they were going to release protobuf 3 where they know better than you. They made fields optional by default, and magically omitted fields from messages when the fields were present but equal to their default value. In other words, a lot of magic.
This confused a lot of people, and Google seemingly realized that sometimes it wasn't a good idea to do so, so they are now adding back an optional keyword to protobuf 3, and because it doesn't mean the same thing as "optional" in protobuf 2, it probably confuses people even more. This keyword could have been used in this PR, unfortunately it was unstable until recently and requires passing an option to older protobuf compilers, which we really don't want to bother with here.
You can read more at https://github.com/protocolbuffers/protobuf/blob/main/docs/field_presence.md if you are into self-harm.

So, long story short, I've switched back to protobuf 2. Libp2p as a whole still uses protobuf 2 as an example. The only reason why we've switched to protobuf 3 as far as I recall is "bigger number is better", and not anything in particular.

@tomaka tomaka added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Nov 18, 2022
@tomaka
Copy link
Contributor Author

tomaka commented Nov 18, 2022 •
edited
Loading

I've taken the liberty to remove the Pair struct from the protobuf schema, as it is unused.

@tomaka tomaka mentioned this pull request Nov 18, 2022
Closed
2 tasks
@tomaka tomaka mentioned this pull request Nov 18, 2022
Closed
@tomaka
Copy link
Contributor Author

tomaka commented Nov 18, 2022

bot merge

@paritytech-processbot paritytech-processbot bot merged commit 7cfaa03 into paritytech:master Nov 18, 2022
@tomaka tomaka deleted the fix-12727 branch November 18, 2022 14:01
@jasl
Copy link
Contributor

jasl commented Nov 19, 2022

ProtoC 3.15 re-add the optional keyward, so basically the behavior is just like proto2, but it's too new that most distribution not bundle that

@tomaka tomaka mentioned this pull request Nov 21, 2022
Merged
mergify bot pushed a commit to paritytech/smoldot that referenced this pull request Nov 22, 2022
@tomaka
Backport changes about the light client protocol (#3018)
7e5ce3f 
Fix #2975

Backports paritytech/substrate#8046
See also paritytech/substrate#12732
@AurevoirXavier AurevoirXavier mentioned this pull request Jan 10, 2023
Closed
@kacperzuk-neti kacperzuk-neti mentioned this pull request Jan 23, 2023
Merged
devdanco added a commit to gasp-xyz/substrate that referenced this pull request Feb 14, 2023
@devdanco @michalkucharczyk
@dharjeezy @bkchr @kianenigma @niklasad1 @ggwpez @athei @dmitry-markin @koute @altonen @nazar-pc @shawntabrizi @agryaznov @davxy @yjhmelody @anonymousGiga @alvicsam @maltekliemann @MrishoLukamba @andresilva @clangenb @Szegoo @ruseinov @gilescope @Doordashcon @acatangiu @joepetrowski @zjb0807 @benluelo @muharem @koushiro @jsidorenko @shekohex @cheme @dependabot @sam0x17 @arkpar @jp-x-g @Ank4n @Lederstrumpf @serban300 @ascjones @gpestana @jasl @sorpaas @dvdplm @gavofyork @4meta5 @tonyalaribe @Artemka374 @yarikbratashchuk @joao-paulo-parity @elv-nate @tomaka @simonsso @librelois @Robbepop @xlc @rcny @mrcnski @KiChjang @ordian @pepoviola @lexnv @eskimor @tgmichel @ltfschoen @Dinonard @apopiak @tomusdrw @mateuszaaa
Mangata dev v0.9.36 (#78)
8d5b532 
* BlockId removal: refactor: ProofProvider (#12519)

* BlockId removal: refactor: ProofProvider

It changes the arguments of methods of `ProofProvider` trait from:
block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* LightClientRequestHandler: excessive BlockIdTo bound removed

* imports cleanup

* formatting

* args tyeps cleanup

* registrar: Avoid freebies in provide_judgement (#12465)

* evaluate repatriate reserved error in pallet identity

* fix benchmarks

* add repatriate reserved error test

* benchmark fix

* undo lock

* include balance to use for benchmarks

* rename test

* Update frame/identity/src/benchmarking.rs

* Update frame/identity/src/benchmarking.rs

Co-authored-by: Bastian Köcher <[email protected]>

* EPM: allow duplicate submissions (#12237)

* allow for duplicate signed submissions

* Fix a bunch of things, seems all good now

* fmt

* Fix

* Update frame/election-provider-multi-phase/src/signed.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Update frame/election-provider-multi-phase/src/signed.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* add migratin

* fmt

* comment typo

* some review comments

* fix bench

Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: Ross Bulat <[email protected]>

* CI check against Rust feature bleed (#12341)

* CI check for rust feature bleed

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Cargo not available

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Handle missing programs

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Check for deps

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add doc

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use correct CI image

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove --offline

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Install cargo-workspaces

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove cargo-workspaces dep

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix try-runtime feature

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix features

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix more features...

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use pipeline-script

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* 🤡

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Make stupid change to test the CI

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* This reverts commit ad2746aa117fa7cb473521113a9bec89aaf26484.

* Use correct branch

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Allow failure

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Make stupid change to test the CI

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Make stupid change to test the CI"

This reverts commit 16ec00e1675c7ec57c988315549ff71e832a3093.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* contracts: Decrease the interation count on slow benchmarks (#12526)

* Decrease amount of benchmark iterations for long slow ones

* ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts

Co-authored-by: command-bot <>

* BlockId removal: refactor: Finalizer (#12528)

* BlockId removal: refactor: Finalizer

It changes the arguments of methods of `Finalizer` trait from:
block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* minor corrections

* failing test corrected

* minor rework

* BlockId removal: refactor: BlockImportOperation+Bknd::finalize_block (#12535)

* BlockId removal: refactor: BlockImportOperation+Bknd::finalize_block

It changes the arguments of methods of `BlockImportOperation` trait
from: block: `BlockId<Block>` to: hash: `&Block::Hash`
`Backend::finalize_block` was also changed.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Review suggestion applied

thx to @davxy

* trigger CI job

* Remove multiple DHTs support from `Discovery` (#12524)

* CI: Enable debug assertions in Wasmer sandbox test (#12540)

* CI: Enable debug assertions in Wasmer sandbox test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix feature dependant import

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Force base weights to be the minimum only when the intercept is negative (#12482)

* Force base weights to be the minimum only when the intercept is negative; emit minimum execution times

* Add an `assert` making sure the intercept is zero when it's supposed to be zero

* Fix template

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_assets

* ".git/.scripts/bench-bot.sh" pallet dev pallet_uniques

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

* Add `DefensiveTruncateFrom` (#12515)

* Add DefensiveTruncateFrom

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Map_err in preimage

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Map_err in beefy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Make test dependant in debug-assertions

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* Refactor service tests in `sc-network` (#12517)

* Refactor service tests in `sc-network`

Create a separate directory for the tests and move common network
instantion related code to `mod.rs` from where it can be used by both
service and chainsync tests.

Use the builder pattern when creating the `TestNetwork` object to reduce
code duplication between the test files.

* Update client/network/src/service/tests/mod.rs

Co-authored-by: Dmitrii Markin <[email protected]>

Co-authored-by: Dmitrii Markin <[email protected]>
Co-authored-by: parity-processbot <>

* Actually fix major sync detection (#12114)

* Actually fix major sync detection

* Introduce `SyncState::Importing` state

* Add target to SyncState enum variants and add `is_major_syncing` method on it

* Remove unnecessary duplicated `best_seen_block` from `SyncState` struct

* Revert "Remove unnecessary duplicated `best_seen_block` from `SyncState` struct"

This reverts commit bb8abd458c939881c049f69d59f3acba47c97c5c.

* Add missing `websocket` feature to `libp2p`

Co-authored-by: parity-processbot <>

* BlockId removal: refactor: Backend::begin_state_operation (#12541)

It changes the arguments of `Backend::begin_state_operation`
from: block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* use headers on templates (#12546)

* Make `NetworkService` callable for `ChainSync` (#12542)

Introduce a middleware called `NetworkServiceProvider` which the
`ChainSync` can use to communicate with `NetworkService`. `ChainSync` is
given a `NetworkServiceHandle` which it uses to call `NetworkServiceProvider`
which then dispatches the calls to `NetworkService` on behalf of `ChainSync`.

This change will allow `ChainSync` to disconnect and report peers and
in the future it'll be possible to send requests and notifications
through the `NetworkServiceProvider`.

`NetworkServiceProvider` is needed only until the `ChainSync` object
has been removed from `Protocol`. After that, a normal `NetworkService`
handle can be passed onto `ChainSync` and these changes can be
deprecated.

Co-authored-by: parity-processbot <>

* Base Kademlia protocol name on genesis hash and fork ID (#12545)

* contracts: Allow indeterministic instructions off-chain (#12469)

* Allow indetermistic instructions off-chain

* Apply suggestions from code review

Co-authored-by: Sasha Gryaznov <[email protected]>

* fmt

Co-authored-by: Sasha Gryaznov <[email protected]>

* Normalize keystore type and its usage across tests (#12553)

* Normalize keystore type and usage across tests

* Extract peer index from array index

* Update template pallet to latest enum syntax (#12552)

* feat: generalize some functions in sp-trie (#12376)

* feat: add to_memory_db to StorageProof

* feat: add iter method and generalize iter_nodes

* fmt

* feat: generalize `encode_compact` like `decode_compact`, add to_compact_proof to StorageProof

* fix to_compact_proof

* improve by suggestions

* improve by suggestions

Co-authored-by: Bastian Köcher <[email protected]>

* Make Multisig Pallet Bounded (#12457)

* Bounded multisig

* just use u32

Co-authored-by: parity-processbot <>

* Fix error during build: failed to run custom build command for sc-network-bitswap (#12494)

* Update `pallet-multisig` benches (#12558)

* Typo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_multisig

* remove functions

* ".git/.scripts/bench-bot.sh" pallet dev pallet_multisig

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Shawn Tabrizi <[email protected]>

* [ci] cargo-check-benches against different base branches (#12557)

* Fix typo (#12571)

* replaced println with log Closes #12338 (#12348)

* replaced println with log Closes #12338

* fixed println

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

* Apply suggestions from code review

* ".git/.scripts/fmt.sh" 1

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: command-bot <>

* Aura: Adds some compatibility mode to support old chains (#12492)

* Aura: Adds some compatibility mode to support old chains

In https://github.com/paritytech/substrate/pull/9132 we changed the way how we get the authorities
from the runtime. Before this mentioned pr we would call `initialize_block` before calling the
authorities runtime function. The problem with this was that when you have a block X that would
switch the authority set, it would already be signed by an authority of the new set. This was wrong,
as a block should only be signed by the current authority set. As this change is a hard fork, this
pr brings back the possibility for users that have a chain running with this old logic to upgrade.

They will need to use:
```
CompatibilityMode::UseInitializeBlock { until: some_block_in_the_future }
```

Using this compatibility mode will make the node behave like the old nodes, aka calling
`initialize_block` before doing the actual runtime call to `authorities`. Then when the given
`until` block is being build/imported the node switches to the new behaviour of not calling
`initialize_block` before. This is a hard fork, so the `until` block should be chosen wisely as a
point where all nodes in the network have upgraded.

* Fixes

* Make docs ready

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <[email protected]>

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <[email protected]>

* Update client/consensus/aura/src/lib.rs

Co-authored-by: André Silva <[email protected]>

* FMT

Co-authored-by: André Silva <[email protected]>

* bump ed25519-zebra; fixes `full_crypto` feature flag in `no_std` (#12576)

* Utility: add more tests for batch/batchAll/forceBatch (#12506)

* Utility: add more tests for batch/batchAll/forceBatch

* remove unnecessary

* batch works with council

* add more tests

* better call examples

* shorter code

* Update frame/utility/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/utility/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/utility/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* update TestBaseCallFilter

* fix

* fix?

* fix

Co-authored-by: Bastian Köcher <[email protected]>

* Treat near-zero intercept values as zero when calculating weights (#12573)

* Treat near-zero intercept values as zero when calculating weights

* Simplify comparison

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update contract weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* [Enhancement] Convert fast-unstake to use StakingInterface, decouplin… (#12424)

* [Enhancement] Convert fast-unstake to use StakingInterface, decoupling it from Staking

* Update primitives/staking/src/lib.rs

Co-authored-by: Kian Paimani <[email protected]>

* Update primitives/staking/src/lib.rs

Co-authored-by: Kian Paimani <[email protected]>

* fix validator comment

* remove todo

* ideas from Kian (#12474)

Co-authored-by: kianenigma <[email protected]>

* Rename StakingInterface -> Staking for nomination-pools

* Staking fixes

* StakingInterface changes

* fix fast-unstake

* fix nomination-pools

* Fix fast-unstake tests

* Fix benches for fast-unstake

* fix is_unbonding

* fix nomination pools

* fix node code

* add mock comments

* fix imports

* remove todo

* more fixes

* more fixes

* bench fixes

* more fixes

* more fixes

* import fix

* more fixes

* more bench fix

* refix

* refix

* Update primitives/staking/src/lib.rs

Co-authored-by: Kian Paimani <[email protected]>

* is_unbonding returns a result

* fix

* review fixes

* more review fixes

* more fixes

* more fixes

* Update frame/fast-unstake/src/benchmarking.rs

Co-authored-by: Squirrel <[email protected]>

* remove redundant CurrencyBalance from nom-pools

* remove CB

* rephrase

* Apply suggestions from code review

* Update frame/nomination-pools/src/tests.rs

* finish damn renamed

* clippy fix

* fix

Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: kianenigma <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Squirrel <[email protected]>

* nomination-pools: allow pool-ids to be reused (#12407)

* initial

* logic check

* do_create

* cargo fmt

* fixes

* ensure_signed

* update

* cargo fmt

* remove unused import

* WIP: Replace `wasm-gc` with `wasm-opt` (#12280)

* Use wasm-opt on runtime

* Optimize for size

* Simplify fn compact_wasm_file

* Run a lighter pass for non production builds

* Disable optimizations and keep name section

* Update wasm-opt

* Remove dward sections

* Update wasm-opt

* Update wasm-opt

* Use minimum_nominator_bond instead of nominator_bond (#12585)

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* BlockId removal: refactor: Backend::append_justification (#12551)

* BlockId removal: refactor: Backend::append_justification

It changes the arguments of `Backend::append_justification`
from: block: `BlockId<Block>` to: hash: `&Block::Hash`

This PR is part of `BlockId::Number` refactoring analysis (paritytech/substrate#11292)

* Error message improved

Co-authored-by: Adrian Catangiu <[email protected]>

* single error message in beefy::finalize

* println removed

Co-authored-by: Adrian Catangiu <[email protected]>

* fix some typos (#12584)

* Added test for Client::block (#12590)

* client/beefy: fix incorrect BEEFY justifications import test (#12593)

Signed-off-by: Adrian Catangiu <[email protected]>

* BlockId removal: refactor: Backend::body (#12587)

It changes the arguments of `Backend::body` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

Co-authored-by: parity-processbot <>

* fix: construct_runtime multiple features (#12594)

* fix: construct_runtime multiple features

* Update frame/support/procedural/src/construct_runtime/mod.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Fix fungible unbalanced trait (#12569)

* Fix fungible unbalanced trait

* Add simple decrease_balance test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix decrease_balance_at_most

* Fix decrease_balance_at_most in fungibles

* Rename free_balanceto balance_on_free

* Use reducible_balance instead of balance_on_free

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* [ci] allow fail skip-if-draft job (#12604)

* BlockId removal: refactor: Backend::justifications (#12602)

* BlockId removal: refactor: Backend::justifications

It changes the arguments of `Backend::justifications` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* trigger CI job

* trigger CI job

* bug fix

* match -> if

Co-authored-by: Adrian Catangiu <[email protected]>

Co-authored-by: Adrian Catangiu <[email protected]>

* use associated iterator types for InspectEnumerable (#12389)

* use associated iterator types for InspectEnumerable

* Update frame/uniques/src/impl_nonfungibles.rs

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>

* Add map and try_map methods (#12581)

Co-authored-by: parity-processbot <>

* stabilize 4 storage host funcs (#12611)

* Collective benchmark respects DefaultVote configuration (#12612)

* Collective benchmark respects DefaultVote configuration

* ".git/.scripts/bench-bot.sh" pallet dev pallet_collective

Co-authored-by: command-bot <>

* BlockId removal: refactor: Backend::block_indexed_body (#12609)

* BlockId removal: refactor: Backend::block_indexed_body

It changes the arguments of `Backend::block_indexed_body` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* trigger CI job

* Introduce DefensiveMin and DefensiveMax (#12554)

* traits for defensive min and defensive max

* defensive min and strict min with tests

* defensive max and strict max with tests

* include docs

* implement partial ord on defensive min and max

* Update frame/support/src/traits/misc.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* wrap lines

* Fix traits

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/support/src/traits/misc.rs

* Update frame/support/src/traits/misc.rs

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* pallet-sudo: add `CheckOnlySudoAccount` signed extension (#12496)

* pallet-sudo: add `CheckSudoKey` signed extension

Signed-off-by: koushiro <[email protected]>

* Rename CheckSudoKey => CheckOnlySudo

Signed-off-by: koushiro <[email protected]>

* Rename extension name and Add some docs

* Apply review suggestions

* Update frame/sudo/src/extension.rs

Co-authored-by: Jegor Sidorenko <[email protected]>

* Update frame/sudo/src/extension.rs

Signed-off-by: koushiro <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Jegor Sidorenko <[email protected]>

* Move Throughput into `sc-sysinfo` (#12368)

* move Throughput to sc-sysinfo

* replace u64

* fix in tests

* change Throughput

* refactored Throughput

* fixes

* moved tests & fixes

* custom serializer

* note

* fix serializer

* forgot to remove

* deserialize

* functioning deserialization :)

* try to make clipply happy

* Serialize as function

* test HwBench

* rename

* fix serialization

* deserialize as function

* unused import

* move serialize/deserialize

* don't serialize none

* remove nonsense

* remove nonsense comment :P

* fixes

* remove all the todos

* return enum

* fixes

* fix nit

* improve docs & readability

* Update client/sysinfo/src/sysinfo.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* fix all the nits

* rename

* fix

* Update client/sysinfo/src/sysinfo.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* remove unit from serialization

* Update utils/frame/benchmarking-cli/src/machine/hardware.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Bump `k256` from `0.10.4` to `0.11.4` (#12085)

* Bump `k256` from `0.10.4` to `0.11.4`

* Update Cargo.lock

* Update

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Guard some invalid node for proof decoding. (#12417)

* Guard some invalid node for proof decoding.

* only forbid bitmap with no children.

* change format

* scale error.

* small test

Co-authored-by: parity-processbot <>

* Bump regex from 1.5.5 to 1.6.0 (#12117)

Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.6.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parity-processbot <>

* Make `--db` case insensitive again (#12630)

This was broken in the switch to Clap v4.

* txpool: enactment state forced update (#12632)

* txpool: enactment state forced update

When `tree_route` computation fails, we still need to update the
`enactment_state` to be aligned with last known finalized/best block.

We do not execute enactment phase of maintain procedure, but we do
update the state.

* error -> debug

* test added

* Add pallet dev mode (#12536)

* stub for construct_dev_runtime!

* revert

* stub for dev_mode proc macro

* preliminary docs for pallet::dev_mode (attribute) proc macro

* add dev_mode to pallet_macros module

* add docs item for dev_mode to frame_support

* parsing of #[pallet(dev_mode)]

* strip out dev_mode stub since it will be an arg for pallet instead

* make pallet Def struct aware of dev mode

* WIP

* revert changes to call.rs

* pass dev_mode to pallet parsing code

* auto-specify default weights when in dev mode if not specified

* add proof / expect for syn::parse in dev mode weight processing

* set all storages to unbounded when in dev mode

* just use 0

Co-authored-by: Shawn Tabrizi <[email protected]>

* add invalid pallet arg test

* add passing dev mode pallet test

* add test confirming that dev mode features only work in dev mode

* cargo fmt + clean up

* bump CI

* fix pallet ui test

* add docs for dev mode

* add warning about using dev mode in production circumstances

* remove comment about no other attributes being supported

* fix unneeded assignment

* make warning more explicit

* more explicit warning about using dev mode in production

* simpler assignment for dev_mode boolean

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* add note about MEL requirement

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* add comment specifying why weights can be omitted in example

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* tweak wording of comments

* bump ci

Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* BlockId removal: &Hash to Hash (#12626)

It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Do not update peer information if ancestor search is in progress (#12631)

* Do not update peer information if ancestor search is in progress

If block announcement is received from a peer while ancestor search
for that same peer is still in progress, do not update the peer's best
hash and best number as that causes the ancestor search to yield
different information from what was expected and can cause, for example,
a fork of lower height not be be downloaded.

* Block until peers are in sync

* Pipeline with ci image with rust 1.65 (#12628)

* Pipeline with ci image with rust 1.65

* fix tests

* use image with sha

* `sp_trie::Recorder`: Fix recording the same key for different tries (#12636)

With `StateVersion::V1` values over a certain size are not inlined and being put into the backend
with their own hash. When accessing a value in the trie with a recorder, we check if the value is maybe already
recorded and thus, we can check the cache. To check if a value is already recorded, we use the key
of the value to differentiate them. The problem is when there are multiple tries, like multiple
child tries that all have different values under the same key. Before this pull request we didn't
have differentiated for which trie we already had recorded a (key, value) pair. This is now done by also taking
the storage root into account in the recorder to differentiate the different (key, value) pair in
the tries.

* Fix UI tests (#12642)

* `payment_queryInfo`: Make it work with `WeightV2` (#12633)

* `payment_queryInfo`: Make it work with `WeighV2`

The runtime api for querying the payment info depends on the `Weight` type and broke for old
runtimes that still use the `WeighV1`. This pull requests fixes this by:

1. Bumping the version of the runtime api.
2. Making the node side code use the correct runtime api function depending on the version of the
runtime api.
3. Make the RPC always return `WeighV1`.

Users of the api should switch to `state_call` and decide based on the version of the runtime api
which `Weight` type is being returned.

* Fix tests

* Review comment

* State-db refactoring (#12239)

* Prune discarded blocks immediately

* state-db refactoring part 1

* Some renames

* Get rid of pending state

* Revert "Prune discarded blocks immediately"

This reverts commit 790f54038b52ff379a573ed0806f38d09af098ec.

* Cleanup

* Make clippy happy

* Minor changes

* Remove duplicate units (#12634)

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add batching to fast-unstake pallet (#12394)

* implement a brand new batch with all tests passing.

* fix benchmarks as well

* make benchmarks more or less work

* fix migration

* add some testing

* Update frame/fast-unstake/src/benchmarking.rs

Co-authored-by: Roman Useinov <[email protected]>

* review comments

* some fixes

* fix review comments

* fix build

* fmt

* fix benchmarks

* fmt

* update

Co-authored-by: Roman Useinov <[email protected]>

* [ci] Use ci-linux:production image in ci (#12648)

* New Weights for All Pallets (#12325)

* new weights for everything

* fix

* fmt

* new batch

* fmt

* new batch

* Update run_all_benchmarks.sh

* add headers

* update weights

* Update lib.rs

* block and extrinsic weight

* Remove partial key size limit from trie codec (#12566)

* remove size limit from trie codec

* test previous upper limit is not enforced anymore

* fmt

* restore test

* Keep the same name (#12616)

Co-authored-by: x <[email protected]>

* Do not finalize parent twice (#12653)

If the parent block is alread finalized, we don't need to do this again.

* update paritydb and remove dev deps on rocksdb (#12641)

* update paritydb and remove dev deps on rocksdb

* feature rocksdb for node testing

* feature decl in node-bench

* revert change to rocksdb inclusion logic

* Update bin/node/bench/Cargo.toml

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>

* Epoch-Changes tree pruning was lagging by one epoch (#12567)

* Remove all not required nodes from the epoch-changes tree

Some outdated nodes were left there because of the predicate

* Test to excercise the fix

* Add a fork on genesis to the test

* Fix typo in comments

* Bound Election and Staking by MaxActiveValidators (#12436)

* bounding election provider with kian

* multi phase implement bounded election provider

* election provider blanket implementation

* staking compiles

* fix test for election provider support

* fmt

* fixing epmp tests, does not compile yet

* fix epmp tests

* fix staking tests

* fmt

* fix runtime tests

* fmt

* remove outdated wip tags

* add enum error

* sort and truncate supports

* comment

* error when unsupported number of election winners

* compiling wip after kian's suggestions

* fix TODOs

* remove,fix tags

* ensure validator count does not exceed maxwinners

* clean up

* some more clean up and todos

* handle too many winners

* rename parameter for mock

* todo

* add sort and truncate rule if there are too many winners

* fmt

* fail, not swallow emergency result bound not met

* remove too many winners resolution as it can be guaranteed to be bounded

* fix benchmark

* give MaxWinners more contextual name

* make ready solution generic over T

* kian feedback

* fix stuff

* Kian's way of solvign this

* comment fix

* fix compile

* remove use of BoundedExecution

* fmt

* comment out failing integrity test

* cap validator count increment to max winners

* dont panic

* add test for bad data provider

* Update frame/staking/src/pallet/impls.rs

Co-authored-by: Kian Paimani <[email protected]>

* fix namespace conflict and add test for onchain max winners less than desired targets

* defensive unwrap

* early convert to bounded vec

* fix syntax

* fmt

* fix doc

* fix rustdoc

* fmt

* fix maxwinner count for benchmarking

* add instant election for noelection

* fmt

* fix compile

* pr feedbacks

* always error at validator count exceeding max winners

* add useful error message

* pr comments

* import fix

* add checked_desired_targets

* fmt

* fmt

* fix rust doc

Co-authored-by: parity-processbot <>
Co-authored-by: kianenigma <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>

* Update some dependencies to prune duplicated crates with different version (#12560)

* sc-client-babe/sp-arithmetic-fuzzer: update num-bigint and num-rational to v0.4

* update lru 0.7.5 ==> v0.8.1

* pallet-example-offchain-worker: update lite-json v0.1.3 ==> v0.2.0

* update hyper 0.14.16 ==> 0.14.20, num-fromat 0.4.0 ==> 0.4.3

* pallet-mmr: update ckb-merkle-mountain-range v0.3.2 ==> v0.5.2

* update handlebars v4.2.2 ==> v4.3.5

* `runtime_cache_size` must always be at least 1

Signed-off-by: koushiro <[email protected]>

* default cache size with .min(1)

Signed-off-by: koushiro <[email protected]>

* update hyper 0.14.20 ==> 0.14.22

Signed-off-by: koushiro <[email protected]>

* update lru 0.8.0 ==> 0.8.1

Signed-off-by: koushiro <[email protected]>

* Apply suggestions from code review

* Apply suggestions from code review

* Fix Cargo.lock

Signed-off-by: koushiro <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Consolidate and deduplicate MMR API methods (#12530)

* histor. batch proof: make best block arg optional

* correct testing range

* make generate_batch_proof stub for historical

* merge generate_{historical_}batch_proof functions

* merge generate_{batch_}proof functions

* merge verify_{batch_}proof functions

* merge verify_{batch_}proof_stateless functions

* remove {Leaf}Proof

Not utilized by API anymore, so superfluous.
Removal consistent with prior changes to just use "batch" proof API.

* rename BatchProof->Proof

no need to qualify if only one universal proof type.

* cleanup

* expose verify_proof rpc api

* document verify_proof

* expose verify_proof_stateless rpc api

* add optional BlockHash to mmr_root rpc api

* fixup! expose verify_proof rpc api

* fix documentation phrasing

Co-authored-by: Adrian Catangiu <[email protected]>

* documentation grammar

Co-authored-by: Adrian Catangiu <[email protected]>

* define mmr error msgs together with error enum

Co-authored-by: Serban Iorga <[email protected]>

* fixup! define mmr error msgs together with error enum

* map decoding errors to CallError::InvalidParams

Co-authored-by: Serban Iorga <[email protected]>

* fixup! map decoding errors to CallError::InvalidParams

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Serban Iorga <[email protected]>

* Bump ss58-registry from 1.29.0 to 1.34.0 (#12659)

Bumps [ss58-registry](https://github.com/paritytech/ss58-registry) from 1.29.0 to 1.34.0.
- [Release notes](https://github.com/paritytech/ss58-registry/releases)
- [Changelog](https://github.com/paritytech/ss58-registry/blob/main/CHANGELOG.md)
- [Commits](https://github.com/paritytech/ss58-registry/compare/v1.29.0...v1.34.0)

---
updated-dependencies:
- dependency-name: ss58-registry
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add `CreateOrigin` to Assets Pallet (#12586)

* add CreateOrigin to Assets pallet

* fix asset-tx-payment test

* use AccountId > u64 in test

* Update frame/assets/src/benchmarking.rs

Co-authored-by: Jegor Sidorenko <[email protected]>

Co-authored-by: Jegor Sidorenko <[email protected]>

* `sp-runtime`: make `parity-util-mem` dependency optional (#12657)

* `sp-runtime`: make `parity-util-mem` dependency optional

* Use default-features = false for sp-runtime in sp-keyring

* Remove parity-util-mem from sp-core

* Cargo.lock

* Restore default-features for keyring dependency

* GrandpaJustification: Feature gate `Debug` (#12664)

The grandpa crate is deriving `Debug` only when the `std` feature is enabled. `RuntimeDebug` can be
forced to derive `Debug` also in `no_std` and that doesn't work together. So, we should feature gate
`Debug` on `no_std`.

* More testing and fuzzing and docs for pools (#12624)

* move pools fuzzing to hongfuzz

* merge more small fixes

* fix all tests

* Update frame/nomination-pools/fuzzer/src/call.rs

Co-authored-by: Gonçalo Pestana <[email protected]>

* remove transactional

* fmt

* fix CI

* fmt

* fix build again

* fix CI

Co-authored-by: Gonçalo Pestana <[email protected]>

* Remove `sp_tasks::spawn` API and related code + host functions (#12639)

* Remove `sp_tasks::spawn` API and related code

* Remove `RuntimeTasks::{spawn, join}` host functions

* remove unused

* Remove a few more tests that I forgot to remove

Co-authored-by: Shawn Tabrizi <[email protected]>

* Contracts pallet: Bump Runtime API (#12677)

* Fix typo (#12680)

* Move `WeightCounter` to `sp-weights` (#12603)

* Move WeightCounter to sp_weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Rename to WeightMeter and test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix pallet-scheduler for new usage

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update primitives/weights/src/weight_meter.rs

Co-authored-by: David <[email protected]>

* More tests for can_accrue

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Clippy

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Remove defensive_accrue and fixup consumed_ratio

I dont think there is a good use-case for defensive_accrue
without saturation. Only in tests maybe, will remove for now
until we have a use-case.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: David <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* Allow other pallets to check asset ids. (#12666)

* Make it easier for other pallets to check asset ids.

* Avoid boxing

* cargo fmt

* derive type info for some grandpa types (#12683)

* Safe TreeRoute constructor (#12691)

* Safe TreeRoute constructor
* Remove test duplicate
* Better tree route error info

* New `root_testing` pallet (#12451)

* Move fill_block to RootOffences

* docs

* new pallet

* new line

* fix

* Update frame/root-testing/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/root-testing/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update bin/node/runtime/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/root-testing/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/root-testing/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update frame/root-testing/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* fixes

* problem solved

* revert

* fix dependency

* hopefully making the CI happy

* ...

* dummy call

* remove dummy

* fix warning

Co-authored-by: Bastian Köcher <[email protected]>

* [ci] Add DAG for build-rustdoc and check-dependent-project (#12687)

* [ci] Debug ci runner

* try gha

* allow mac jobs fail

* add dags

* install protoc

* fix protobuf name

* fix dags

* remove allow fail for mac jobs

* remove gha

* adjust cargo-check-macos

* Collective: Benchmark with greater `MaxProposals` (#12454)

* Collective: Benchmark with greated

* fix

* remove bs

* id_to_remark_data

* fix

* remove hardcoded

* clean up

* simplify

* questionable renaming

* better variable name

* better solution

* no need for large length

* better solution

* Update frame/collective/src/benchmarking.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* fix

* test

* remove test

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* [ci] fix buildah for publishing docker (#12703)

* Make public is_passing and ReferendumStatus (#12667)

* init

* clean

* remove manual getter for ReferendumStatus in favor of changing pub crate to pub for ReferendumStatus DecidingStatus Deposit types

* rm status getters because fields are pub now

* Asset Pallet: Support repeated destroys to safely destroy large assets (#12310)

* Support repeated destroys to safely destroy large assets

* require freezing accounts before destroying

* support only deleting asset as final stage when there's no assets left

* pre: introduce the RemoveKeyLimit config parameter

* debug_ensure empty account in the right if block

* update to having separate max values for accounts and approvals

* add tests and use RemoveKeyLimit constant

* add useful comments to the extrinsics, and calculate returned weight

* add benchmarking for start_destroy and finish destroy

* push failing benchmark logic

* add benchmark tests for new functions

* update weights via local benchmarks

* remove extra weight file

* Update frame/assets/src/lib.rs

Co-authored-by: joe petrowski <[email protected]>

* Update frame/assets/src/types.rs

Co-authored-by: joe petrowski <[email protected]>

* Update frame/assets/src/lib.rs

Co-authored-by: joe petrowski <[email protected]>

* effect some changes from codereview

* use NotFrozen error

* remove origin checks, as anyone can complete destruction after owner has begun the process; Add live check for other extrinsics

* fix comments about Origin behaviour

* add AssetStatus docs

* modularize logic to allow calling logic in on_idle and on_initialize hooks

* introduce simple migration for assets details

* reintroduce logging in the migrations

* move deposit_Event out of the mutate block

* Update frame/assets/src/functions.rs

Co-authored-by: Muharem Ismailov <[email protected]>

* Update frame/assets/src/migration.rs

Co-authored-by: Muharem Ismailov <[email protected]>

* move AssetNotLive checkout out of the mutate blocks

* rename RemoveKeysLimit to RemoveItemsLimit

* update docs

* fix event name in benchmark

* fix cargo fmt.

* fix lint in benchmarking

* Empty commit to trigger CI

* Update frame/assets/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/assets/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/assets/src/functions.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/assets/src/functions.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/assets/src/functions.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/assets/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update frame/assets/src/functions.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* effect change suggested during code review

* move limit to a single location

* Update frame/assets/src/functions.rs

Co-authored-by: joe petrowski <[email protected]>

* rename events

* fix weight typo, using rocksdb instead of T::DbWeight. Pending generating weights

* switch to using dead_account.len()

* rename event in the benchmarks

* empty to retrigger CI

* trigger CI to check cumulus dependency

* trigger CI for dependent cumulus

* Update frame/assets/src/migration.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* move is-frozen to the assetStatus enum (#12547)

* add pre and post migration hooks

* update do_transfer logic to add new assert for more correct error messages

* trigger CI

* switch checking AssetStatus from checking Destroying state to checking live state

* fix error type in tests from Frozen to AssetNotLive

* trigger CI

* change ensure check for fn reducible_balance()

* change the error type to Error:<T,I>::IncorrectStatus to be clearer

* Trigger CI

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Muharem Ismailov <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* `seal_reentrant_count` returns contract reentrant count (#12695)

* Add logic, test, broken benchmark

* account_entrance_count

* Addressing comments

* Address @agryaznov's comments

* Add test for account_entrance_count, fix ci

* Cargo fmt

* Fix tests

* Fix tests

* Remove delegated call from test, address comments

* Minor fixes and indentation in wat files

* Update test for account_entrance_count

* Update reentrant_count_call test

* Delegate call test

* Cargo +nightly fmt

* Address comments

* Update reentrant_count_works test

* Apply weights diff

* Add fixture descriptions

* Update comments as suggested

* Update reentrant_count_call test to use seal_address

* add missing code

* cargo fmt

* account_entrance_count -> account_reentrance_count

* fix tests

* fmt

* normalize signatures

Co-authored-by: yarikbratashchuk <[email protected]>

* Assets Pallet: reintroduce fungibles::Destroy trait  (#12708)

* update docs formatting

* reintroduce the destroy trait

* copy changes from original PR

* remove witness

* Trigger CI

* Trigger CI

* release `sp-core 7.0.0` and `sp-runtime 7.0.0` (#12599)

* chore(release): sp-core v7.0.0

* chore(release): sp-runtime v7.0.0

* fix bad merge

* Release `sp-keyring` and `pallet-contracts-primitives` `7.0.0` (#12716)

* Bump sp-keyring

* Bump pallet-contracts-primitives

* Cargo.lock

* Fix `cargo check` for `pallet-contracts-proc-macro` (#12706)

* fix `cargo check` for pallet-contracts-proc-macro

* add test for cargo-check of pallet-contracts-proc-macro

* remove cargo-check-contracts-proc-macro

https://github.com/paritytech/substrate/pull/12706/files#r1022783937

* [ci] Improve pipeline stopper (#12717)

* [ci] Improve pipeline stopper

* break test-linux-stable 1/3

* break test-linux-stable 2/3

* break test-linux-stable 3/3

* break cargo-check-benches 1/2

* break cargo-check-benches 2/2

* fix benches

* sc-chainspec: Switch to `assimilate_storage` (#12720)

Before it was using `build_storage` and `assimilate_storage` was returning an error. However, there
was no real reason for `assimilate_storage` to return an error. This pr implements
`assimilate_storage` and uses the default `build_storage` of the trait.

* [Cleanup] Remove obsolete event from fast-unstake (#12725)

Trivial, just removing unused code.

* [Fix] Deposit for fast-unstake has to be define as pallet::constant (#12729)

Fixes https://github.com/paritytech/substrate/issues/12618

* Add event testing example to pallet template (#12722)

Add an example of how to test for events into the example pallet. Right now, the information is pretty hard to find without looking into pallet tests or finding some particular posts on the stackoverflow.

* Remove the `wasmtime` feature flag (#12684)

* Remove the `wasmtime` feature flag

* rustfmt

* Fix the light client protocol protobuf schema (#12732)

* Fix the light client protocol protobuf schema

* Add another test

* Remove unused protobuf struct

* Ok you have to use the nightly rustfmt apparently

* Update template to remove clippy warnings (#12670)

* Update template to remove clippy warnings

* ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery

* Update templates from child project

This should remove clippy warnings on generated files

* Update after review

* Update frame-weight-template.hbs

Commit suggestion

* ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery

* Rerun linter on linked project

Updates from child project

* ".git/.scripts/bench-bot.sh" pallet dev pallet_lottery

Co-authored-by: command-bot <>

* Check all crates (#12709)

* check all crates individually

It's relevant to check workspace crates individually because otherwise their compilation problems
due to feature misconfigurations won't be caught, as exemplified by
https://github.com/paritytech/substrate/issues/12705

* adapt to lack of multiple macos runners

https://github.com/paritytech/substrate/pull/12709#discussion_r1022868752

* fix cancel-pipeline-cargo-check-each-crate-macos

* fix cargo-check-each-crate-macos again

* time command execution

* fix YAML anchors

* add explanation for rounding division

* ensure the minimum of one crate per group

* collect artifacts for pipeline stopper

* revert hardcoded crates_per_group

* re-add crates_per_group=1

* client/beefy: persist voter state (#12712)

* client/beefy: prepare worker for persisting state

* client/beefy: persist voter state

* client/beefy: initialize persistent state

* client/beefy: try to vote from the very beginning

Now that voter is initialized from persistent state, it makes
sense that it can attempt voting right away. This also helps
the genesis case when we consider block `One` as mandatory.

* client/beefy: add tests for voter state db
* client/beefy: persist voter state as soon as initialized
* client/beefy: make sure min-block-delta is at least 1
* client/beefy: persist state after voting

Persist state after handling self vote to avoid double voting in case
of voter restarts.

* client/beefy: persist state after handling mandatory block vote

For mandatory blocks we want to make sure we're not losing votes
in case of crashes or restarts, since voter will not make further
progress without finalizing them.

* frame/beefy: use GENESIS_AUTHORITY_SET_ID on pallet genesis

* client/beefy: initialize voter at either genesis or last finalized
To guarantee unbroken chain of mandatory blocks justifications, voter
will always resume from either last BEEFY-justified block or
`pallet-beefy` genesis, whichever is more recent.

Initialization walks back the chain from latest GRANDPA finalized
block looking for one of the above. Along the way, it also records
and enqueues for processing any BEEFY mandatory blocks that have
been already GRANDPA finalized but not BEEFY finalized.

* client/beefy: decouple voter init from aux db state load
* client/beefy: fix voter init tests
* remove debug prints
* gadget future must be type ()
* fix init from last justification

Signed-off-by: Adrian Catangiu <[email protected]>

* [Fix] Get target count from TargetList instead of storage (#12748)

Co-authored-by: parity-processbot <>

* Move block/state/warpc sync requests/responses to `ChainSync` (#12739)

* Move block/state/warpc sync requests/responses to `ChainSync`

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Apply review suggestions

* cargo-fmt + doc fix

* Fix tests

Co-authored-by: Bastian Köcher <[email protected]>

* perf: generate_initial_session_keys: load runtime only if its relevant (#12651)

* perf: generate_initial_session_keys: load runtime only if its relevant

* apply review suggestion

* Update primitives/session/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* Prevent epochs pruning while finalizing blocks on epoch 0 (#12758)

* Prevent epochs pruning while on epoch 0

* return error instead of expect in `feasibility_check` (#12745)

* Update lib.rs

* make defensive

* fmt

* fix batching migration

* fix

* fix

Co-authored-by: parity-processbot <>

* BEEFY: optimize voter event loop for fewer 'active' wakeups (#12760)

* client/beefy: remove high-freq network events from main loop

Network events are many and very frequent, remove the net-event-stream
from the main voter loop and drastically reduce BEEFY voter task
'wakeups'.

Instead have the `GossipValidator` track known peers as it already
has callbacks for that coming from `GossipEngine`.

Signed-off-by: acatangiu <[email protected]>

* Sort crates before splitting them into groups (+ some improvements) (#12755)

* sort crates before splitting them into groups

this is useful so that crates always get routed to a specific group for a given version of the source code, which means that jobs for each batch can be reliably retried individually

* more verbose output

* misc improvements

* put uniq after sort

uniq filters by adjacent lines

* shellcheck

* rm useless backlashes

* handle edge case of no crates detected

* contracts: Replace `sp-sandbox` and `wasmi-validation` by newest wasmi (#12501)

* Replace sp-sandbox and wasmi-validation by just wasmi

* ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts

* Re-check original code on re-instrumentation

* Fix clippy

* ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts

* Apply suggestions from code review

Co-authored-by: Robin Freyler <[email protected]>

* Replace wasmi by ::wasmi

* Bump wasmi to 0.20

* Add explanation for `unreachable`

* Change proof

* Fixup master merge

* ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts

* Fixup naming inconsistencies introduced by reentrancy PR

* Fix `scan_imports` docs

* Apply suggestions from code review

Co-authored-by: Sasha Gryaznov <[email protected]>

* Fixup suggestions

* Remove unnecessary &mut

* Fix test

* ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts

* Fix benchmark merge fail

* ".git/.scripts/bench-bot.sh" pallet dev pallet_contracts

* Fix docs as suggested by code review

* Improve docs for `CodeRejected`

* Apply suggestions from code review

Co-authored-by: Sasha Gryaznov <[email protected]>

* Fix logic bug when setting `deterministic_only`

* Don't panic when module fails to compile

* Apply suggestions from code review

Co-authored-by: Robin Freyler <[email protected]>

Co-authored-by: command-bot <>
Co-authored-by: Robin Freyler <[email protected]>
Co-authored-by: Sasha Gryaznov <[email protected]>

* update DefaultNoBound derive macro (#12723)

fix derive for empty enums

Update derive & ui tests

clean up

Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

rename variable

formatting & clippy

formatting

Co-authored-by: parity-processbot <>

* Fix rustdoc (#12777)

* Fix table formatting

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix sp-runtime-interface table

Using HTML now since multi-line tables are not a thing and fmt
destroys them.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* More rustdoc fixes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix tags

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* More fixes...

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use Bastis patch

Co-authored-by: Bastian Köcher <[email protected]>
Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add more backticks

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* change ci image

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: alvicsam <[email protected]>

* Allow Alliance Fellows to Give Up Voting Rights (#12730)

* allow fellows to abdicate voting rights

* rename founders to founding fellows, give equal power

* Drop FoundingFellow role and veto call (#12762)

* drop FoundingFellow role

* drop veto call

* Storage migration to remove founder role (#12766)

* storage migration to remove founder role

* skip migration if no members

* truncate the final fellows set if overflows

* change log - action order

* MemberAbdicated -> FellowAbdicated

Co-authored-by: Muharem Ismailov <[email protected]>

* Add total nb to trie migration rpc (#12770)

* Add total nb to trie migration rpc

* fix and format

* Use struct instead of tuple

* fixes

Co-authored-by: parity-processbot <>

* add EnsureWithSuccess (#12775)

* add EnsureWithSuccess

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* add docs

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>

* Explicitly unset RUSTC_WRAPPER=sccache environment variable (#12771)

* CI: Explicitly unset RUSTC_WRAPPER=sccache environment variable

* Try with `rusty-cachier` disabled

* Re-enable `rusty-cachier` and try with the staging image

* Bring back `production` image

* Sort crates before splitting them into groups (+ some improvements) (#12755)

* sort crates before splitting them into groups

this is useful so that crates always get routed to a specific group for a given version of the source code, which means that jobs for each batch can be reliably retried individually

* more verbose output

* misc improvements

* put uniq after sort

uniq filters by adjacent lines

* shellcheck

* rm useless backlashes

* handle edge case of no crates detected

* Revert "Sort crates before splitting them into groups (+ some improvements) (#12755)"

This reverts commit fde839183a12a2bd51efc7143ebcddeed81ea6fa.

Co-authored-by: João Paulo Silva de Souza <[email protected]>

* contracts: Don't put unstable functions in special module (#12781)

* Don't put unstable functions in special module

* Apply suggestions from code review

Co-authored-by: Sasha Gryaznov <[email protected]>

* cargo fmt

Co-authored-by: Sasha Gryaznov <[email protected]>

* ed25519_verify: Support using dalek for historical blocks (#12661)

* ed25519_verify: Support using dalek for historical blocks

The switch from `ed25519-dalek` to `ed25519-zebra` was actually a breaking change. `ed25519-zebra`
is more permissive. To support historical blocks when syncing a chain this pull request introduces
an externalities extension `UseDalekExt`. This extension is just used as a signaling mechanism to
`ed25519_verify` to use `ed25519-dalek` when it is present. Together with `ExtensionBeforeBlock` it
can be used to setup a node in way to sync historical blocks that require `ed25519-dalek`, because
they included a transaction that verified differently as when using `ed25519-zebra`.

This feature can be enabled in the following way. In the chain service file, directly after the
client is created, the following code should be added:

```
use sc_client_api::ExecutorProvider;
client.execution_extensions().set_extensions_factory(
	sc_client_api::execution_extensions::ExtensionBeforeBlock::<Block, sp_io::UseDalekExt>::new(BLOCK_NUMBER_UNTIL_DALEK_SHOULD_BE_USED)
);
```

* Fix doc

* More fixes

* Update client/api/src/execution_extensions.rs

Co-authored-by: André Silva <[email protected]>

* Fix merge and warning

* Fix docs

Co-authored-by: André Silva <[email protected]>

* client/beefy: fix on-demand justifications sync for old blocks (#12767)

* client/beefy: fix on-demand justif sync for old blocks

When receiving BEEFY justifications for old blocks the state might
be pruned for them, in which case justification verification fails
because BEEFY validator set cannot be retrieved from runtime state.

Fix this by having the voter give the validator set to the
`OnDemandJustificationsEngine` as request information. On receiving
a BEEFY justification for requested block, the provided validator
set will be used to validate the justification.

Signed-off-by: acatangiu <[email protected]>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* impl review suggestions

* client/beefy: fail initialization if state unavailable

* beefy: remove spammy log

Signed-off-by: acatangiu <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>

* Remove Default, HasCompact, and TypeInfo trait bounds on AssetId (#12740)

* Remove Default, HasCompact, and TypeInfo trait bounds on AssetId

* don't use default in benchmarking

* add helper trait

* add helper to assets tx payment test

* docs fixes

* i'm confused

* aha, cargo

* move affected dispatchable calls into new indices

* Helper -> BenchmarkHelper

* benchmark use of helper

* actually, don't break every call interface

* use into on AssetIdParameter

* Remove From from AssetIdParameter and use it in BenchmarkHelper

* include from

Co-authored-by: parity-processbot <>

* pallet-mmr: move offchain logic to client-side gadget (#12753)

* Move MMR utils methods from pallet to primitives

Signed-off-by: Serban Iorga <[email protected]>

* Add method to MmrApi

* Move forks expanding logic from babe to primitives

* Implement MMR gadget

* Remove prunning logic from the MMR pallet

* Code review changes: 1st iteration

* Replace MaybeCanonEngine with CanonEngineBuilder

* fix mmr_leaves_count() for kitchen sink demo

* Update client/merkle-mountain-range/src/canon_engine.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* Code review changes: 2nd iteration

* fix INDEXING_PREFIX

* impl review comments

* add documentation and minor rename

Signed-off-by: Serban Iorga <[email protected]>
Co-authored-by: Adrian Catangiu <[email protected]>

* Require rust-features check (#12796)

* Typo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Move rust feature check to docker and require not to fail

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add .docker-env

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Move test-rust-features check back to kubernetes

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* MMR: move RPC code from frame/ to client/ (#12805)

* mmr: move MMR RPC from frame/ to client/

Signed-off-by: Adrian Catangiu <[email protected]>

* client/mmr: adjust logging levels to avoid spam

* cargo fmt

* remove unused imports

Signed-off-by: Adrian Catangiu <[email protected]>

* chore: remove unused traits for wasm interface (#12792)

* sc-transaction-handler: Fix potential crashes on exit (#12807)

This fixes some potential crashes in the stream handling in `sc-transaction-handler`.

* Don't announce blocks in `sync_to_tip_when_we_sync_together_with_multiple_peers` (#12783)

* Fix syncing test

* cargo fmt

* Fix test

* contracts: Replace cargo feature `unstable-interface` with config (#12787)

* Replace cargo feature with config

* Update frame/contracts/proc-macro/src/lib.rs

Co-authored-by: Sasha Gryaznov <[email protected]>

Co-authored-by: Sasha Gryaznov <[email protected]>

* Bounties use SpendOrigin (#12808)

* Bounties use SpendOrigin

* Fix tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* tests: increase spend limits

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fix benchmarks

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix child-bounties tests

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* ".git/.scripts/bench-bot.sh" pallet dev pallet_bounties

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

* Reduce provisioner work (#12749)

* Move create_inherent_data call to use side

* Make provide_inherent_data async

* Fix tests

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* Log errors

* Fix test

* Fix test

* fix

* Deduplicate test code

* fix

* flag

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Revert "Deduplicate test code"

This reverts commit ba46adbe089329c78cd69ccdb08e27ed67bd77cf.

* Fix test

* remove commented out code

* minor to start CI run

* start CI

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Marcin S. <[email protected]>

* Apply PR suggestions

* Apply PR suggestions

* Update client/consensus/slots/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* minor

* kickoff CI

* PR suggestions

* Compute remaining duration instead of using slot_info.duration

* Don't rely on sub implementat…
ark0f pushed a commit to gear-tech/substrate that referenced this pull request Feb 27, 2023
@tomaka @ark0f
Fix the light client protocol protobuf schema (paritytech#12732)
503ff5b 
* Fix the light client protocol protobuf schema

* Add another test

* Remove unused protobuf struct

* Ok you have to use the nightly rustfmt apparently
@tomaka tomaka mentioned this pull request Jun 19, 2023
Draft
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@dmitry-markin dmitry-markin dmitry-markin approved these changes

@altonen altonen altonen approved these changes

Assignees
No one assigned
Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Substrate sometimes sends back an empty response to light client requests
4 participants
@tomaka @jasl @dmitry-markin @altonen