Gui shawntabrizi runtime upgrade (#5118)

* adding unleash to ci (#5020)

* adding unleash to ci

* fixing formatting

* with a dot please

* alpha.3 now

* do not publish testing helpers

* remove old test-helpers cruft

* fix cargo.lock

* with alpha 4

* do not publish runtime-interface-test either

* disable more test crates from publishing

* switch to alpha.5

* replace tempdir with tempfile

* update lru

* switch to bytes 0.5

* release script fixes

* switch on and to latest alpha

* BUT THE SPACES

* Fix: CI failing for some CLI tests (#5043)

* Initial commit

Forked at: 41bb219
Parent branch: origin/master

* Increase killing grace period of CLI tests and display more info

* Use --dev everywhere possible

* Put pruning mode to its own params struct

* Add pruning params to export-blocks command

* Added missing file

* Removed not-dev mode in tests

* Add pruning mode to the revert command

* Decrease killing grace period again

* Move back unsafe_pruning to import_params

* Applied proposed changes

* aura: remove unused tx pool (#5046)

* aura: remove unused transaction pool parameter

* node-template: remove transaction pool from aura

* aura: fix tests

* Extend rust nightly detection in `wasm-builder` (#5021)

Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.

* Add steps setting to benchmarking CLI (#5033)

* Add steps setting to CLI, use max value to hit worst case.

* Bump impl_version.

* Apply review suggestion.

* Remove indices from node-template (#5025)

* Remove indices from node-template

* Use identity lookup instead

* Bump impl

* clean cargo.toml

* Fix documentation for "BlockBuilder::push_trusted" (#5051)

* fix doc

* rephrase

* do not check unleash on every PR, only master and tags (#5054)

* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <[email protected]>

* prepare version to alpha.1 (#5055)

bump version to -alpha.1

* Sync: validate block responses for required data (#5052)

* Less verbose state-db logging

* Validate block responses for block bodies

* Update client/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <[email protected]>

* Added validation test

* Disconnect on missing header as well

* Typo

Co-Authored-By: André Silva <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: André Silva <[email protected]>

* Make these chainspecs fields private (#5031)

* Fix dockerfile (#5059)

* Adds documentation for `wipe` and `commit` (#5053)

* Adds documentation for `wipe` and `commit`

This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.

* Update primitives/externalities/src/lib.rs

Co-Authored-By: joe petrowski <[email protected]>

Co-authored-by: joe petrowski <[email protected]>

* Fix the issue with `trybuild`'s `ui` tests (#4992)

* repro ui bug

* fix the tests

* test with the new image

* test without CARGO_HOME

* test without fixes

* test again

* fix trybuild old versions

* bump CArgo.lock

* fix trybuild newest versions

* bump Cargo.lock

* trying on the latest image

* bump Cargo.lock

* run with the old image

* ci will be green on the image from 2020-02-19 [skip ci]

* bump Cargo.lock

* Activate publishing of draft releases... (#5062)

* Activate publishing of draft releases...

... And fix the message sending (missing parameter).

* publish_draft_release.sh now checks latest...

... release on github rather than just a tag

* Fix/div by zero (#5041)

* Handle gas_price being zero separately

* Bump spec_version

* Add a unit & integration tests for gas price = 0

* set missing metadata fields, prepping alpha.2 (#5067)

* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2

* Fix revalidation not revalidating multiple times (#5065)

* removes use of sc_client::Client from sc_finality_grandpa (#5030)

* removes use of sc_client::Client from sc_finality_grandpa

* code formatting

* code formatting

* removes use of sc_client::Client from sc_finality_grandpa

* Remove deprecated host functions (#5038)

Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.

* removes use of sc_client::Client from sc_basic_authorship (#5050)

* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path

* pallet-transaction-payment clean up (#5070)

* Formatting clean up

* Introduce separate setters for the fees.

* *: Rename prometheus-exporter crate to substrate-prometheus-end… (#5076)

This patch renames the crate for the following two reasons:

1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".

2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.

* rename `browser-utils` to `substrate-browser-utils` (#5079)

* prepping for Alpha.3 (#5080)

* Bump to alpha.3

* update gitlab-ci

* Propagate DispatchError for benchmarks. (#5075)

* Propagate DispatchError for benchmarks.

* Apply review suggestions.

* Use RuntimeString.

* fix expect

Co-Authored-By: Bastian Köcher <[email protected]>

Co-authored-by: Bastian Köcher <[email protected]>

* Add options to overwrite range bounds in benchmark command. (#5072)

* Add --mins --maxs to benchmark command.

* Apply review suggestions.

* Update yamux to version 0.4.4. (#5086)

* Remove more instances of futures01 (#4633)

* Start removing last few instances of futures01

* Use to_poll on wasm

* Revert "Use to_poll on wasm"

This reverts commit 1c61728.

* Fix fg test

* Upgrade network test futures

* Update offchain hyper version

* Update service test

* bump tokio to 0.2.10

* Removed some unneeded tokios

* fixes

* fix run_until_all_full

* Make service test debuggable

* Update client/offchain/src/api/http.rs

Co-Authored-By: Demi Obenour <[email protected]>

* Add service_test to test-int output

* nitpicking

* Finally fix test

* Give up and revert client/serviec/test

* Revert gitlab ci too

Co-authored-by: Demi Obenour <[email protected]>

* Make export blocks default to json on stdout (#5090)

* Make export blocks default to json on stdout

* Multiline instead of single line to stay under 100 cols

* Change --json flag to --binary, defaulting to json

* Offence reporting returns a result (#5082)

* Offence reporting returns a result

* Bump spec_version

* Use unwrap instead of assertions

* Fix more review grumbles

* Update to libp2p 0.16.2 (#5088)

* Remove request ID from the new protocol (#5049)

* Make sure we remove a peer on disconnect in gossip (#5104)

* Make sure we remove peers on disconnect in gossip state machine

* Clear up the code

* Add a comment

* Expose `state-db` memory info (#5110)

This exposes memory statistics from the state-db.

* Change extrinsic_count to extrinsic_index in pallet-utility (#5044)

Co-authored-by: Benjamin Kampmann <[email protected]>

* Add more metrics to prometheus (#5034)

* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate

* Update to SCALE 1.2.0 (#5113)

This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.

* Lazy payouts (#4474)

* TODOs

* Remove superfluous:

* partial implementation

* full implementation

* fix preferences

* update comments

* upgrade test WIP

* fix more tests

* fix cutoff

* fix saturation

* comment

* upgrade mock

* upgrade test

* WIP migration

* WIP migration

* remove slot stake stuff

* fix merge

* migration of ledger

* remove equalize from test

* add test

* fix

* update doc

* fix compilation

* improve test readibility

* improve doc

* fix most todo

* fix migration and test

* remove println

* WIP

* add test and spec

* weight

* update doc

* safer end_era

* fix exposure of conversion

* Revert "safer end_era"

This reverts commit 72ff737.

* fix useless put

* exposure clipped

* doc

* fix payout with clipped

* fix node runtime

* add doc

* pluggable and generalized staking module

* remove print

* update doc

* refactor

* improve documentation and implementation

* fix test

* Fix test

* fix test

* fix test

* fix remove lowest stake from exposure, not biggest.

* nomination index arguments in nominator_payout

* add test

* try to fix offence

* apply slashed and bond eras until active era

* doc

* update spec version

* add test upgrade from previous test environment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <[email protected]>

* nominators upgrade has been cleaned

* dynamic history depth implementation

* make current_era - history_depth included

* Change equality check to start era to less than or equal

* Use era specific validator prefs

* Add print statement and comment about start era if <

* fix next_reward overflow

* make more check for bad era claim for zero cost

* small refactor

* code refactor + fix use of deprecated storage

* fix wasm build

* add comment

* Fix tests

* remove outdated comment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <[email protected]>

* gather active era information into one storage

Co-authored-by: thiolliere <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* impl on_runtime_upgrade

Co-authored-by: Benjamin Kampmann <[email protected]>
Co-authored-by: Cecile Tonglet <[email protected]>
Co-authored-by: André Silva <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Marcio Diaz <[email protected]>
Co-authored-by: Nikolay Volf <[email protected]>
Co-authored-by: s3krit <[email protected]>
Co-authored-by: Arkadiy Paronyan <[email protected]>
Co-authored-by: Pierre Krieger <[email protected]>
Co-authored-by: Chevdor <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Denis Pisarev <[email protected]>
Co-authored-by: Eric <[email protected]>
Co-authored-by: Seun Lanlege <[email protected]>
Co-authored-by: Sergei Pepyakin <[email protected]>
Co-authored-by: Max Inden <[email protected]>
Co-authored-by: Ashley <[email protected]>
Co-authored-by: Toralf Wittner <[email protected]>
Co-authored-by: Demi Obenour <[email protected]>
Co-authored-by: pscott <[email protected]>
Co-authored-by: Fedor Sakharov <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: thiolliere <[email protected]>

frame/executive/src/lib.rs

@@ -186,6 +186,13 @@ where
 			digest,
 			frame_system::InitKind::Full,
 		);
+		let last_runtime_upgrade = <frame_system::Module<System>>::last_runtime_upgrade();
+		if last_runtime_upgrade.map(|n| n == *block_number).unwrap_or(false) {
+			<AllModules as OnRuntimeUpgrade>::on_runtime_upgrade();
+			<frame_system::Module<System>>::register_extra_weight_unchecked(
+				<AllModules as WeighBlock<System::BlockNumber>>::on_runtime_upgrade()
+			);
+		}
 		<AllModules as OnInitialize<System::BlockNumber>>::on_initialize(*block_number);
 		<frame_system::Module<System>>::register_extra_weight_unchecked(
 			<AllModules as WeighBlock<System::BlockNumber>>::on_initialize(*block_number)

frame/staking/src/lib.rs

@@ -927,10 +927,13 @@ decl_module! {
 
 		fn deposit_event() = default;
 
-		fn on_initialize() {
+		fn on_runtime_upgrade() {
 			Self::ensure_storage_upgraded();
 		}
 
+		fn on_initialize() {
+		}
+
 		fn on_finalize() {
 			// Set the start of the first era.
 			if let Some(mut active_era) = Self::active_era() {