Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Add buckets on lower end of distribution to network bridge latency metrics #6359

Merged
merged 6 commits into from
Dec 2, 2022
Merged

Add buckets on lower end of distribution to network bridge latency metrics #6359

merged 6 commits into from
Dec 2, 2022

Conversation

bredamatt
Copy link
Contributor

@bredamatt bredamatt commented Nov 28, 2022 •
edited
Loading

Similar to a previous PR. Adds a couple of more buckets to the network bridge latency metrics. As noticeable below, all observations are currently below 50ms, and although trivial, this PR adds two more buckets for the lower end of the distribution which could be somewhat useful for future network improvements.

Screenshot 2022-11-28 at 15 01 54

@bredamatt bredamatt added A0-please_review Pull request needs code review. C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit. labels Nov 28, 2022
@bredamatt bredamatt self-assigned this Nov 28, 2022
@bredamatt bredamatt added the B0-silent Changes should not be mentioned in any release notes label Nov 28, 2022
@sandreim
Copy link
Contributor

I would imagine most of the samples are < 5ms. Did we try this in Zombienet to see what are the actual values that we want to study their distribution ?

@vstakhov
Copy link
Contributor

I'd suggest to remove some high latency buckets to compensate additional more precise low end buckets. It also makes perfect sense counting that the graphs clearly demonstrate that the high end buckets are not populated.

@bredamatt
Copy link
Contributor Author

bredamatt commented Nov 29, 2022 •
edited
Loading

I would imagine most of the samples are < 5ms. Did we try this in Zombienet to see what are the actual values that we want to study their distribution ?

@sandreim Just tried this now, but these results are quite misleading and not representative as real network values. It will always be a lot faster on one host. Also, it appears that every bucket is incremented per observation, whilst the sum is very very small, so it is hard to say how many observations actually fall within each bucket just by looking at the output like this. Am I right in thinking that whenever bucket with le=0.05 is incremented, then all buckets with le > 0.05 are also incremented? Nevertheless, it does not really answer how low the values go I think.

Here is the output:

polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.05"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.1"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.2"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.3"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.4"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.5"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.6"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.75"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.9"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="1"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="1.2"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="1.5"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="1.75"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="+Inf"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="circulate_statement",chain="rococo_local_testnet"} 0.00000045100000000000005
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="circulate_statement",chain="rococo_local_testnet"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.05"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.1"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.2"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.3"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.4"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.5"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.6"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.75"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.9"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="1"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="1.2"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="1.5"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="1.75"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="+Inf"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="handle_incoming_message",chain="rococo_local_testnet"} 0.000000617
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="handle_incoming_message",chain="rococo_local_testnet"} 25
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.05"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.1"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.2"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.3"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.4"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.5"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.6"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.75"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.9"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="1"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="1.2"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="1.5"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="1.75"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="+Inf"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="new_gossip_topology",chain="rococo_local_testnet"} 0.00000020500000000000002
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="new_gossip_topology",chain="rococo_local_testnet"} 10
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.05"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.1"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.2"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.3"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.4"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.5"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.6"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.75"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.9"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="1"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="1.2"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="1.5"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="1.75"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="+Inf"} 143
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="peer_view_change",chain="rococo_local_testnet"} 0.0000034530000000000023
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="peer_view_change",chain="rococo_local_testnet"} 143

@sandreim
Copy link
Contributor

I would imagine most of the samples are < 5ms. Did we try this in Zombienet to see what are the actual values that we want to study their distribution ?

@sandreim Just tried this now, but these results are quite misleading and not representative as real network values. It will always be a lot faster on one host. Also, it appears that every bucket is incremented per observation, whilst the sum is very very small, so it is hard to say how many observations actually fall within each bucket just by looking at the output like this. Am I right in thinking that whenever bucket with le=0.05 is incremented, then all buckets with le > 0.05 are also incremented? Nevertheless, it does not really answer how low the values go I think.

All samples are in 0.05 in this output. Can you try this with your new buckets ? The idea was to calibrate the buckets based on the observations

@sandreim
Copy link
Contributor

you can do sum/count to get the average value which is helpful

@bredamatt
Copy link
Contributor Author

bredamatt commented Nov 29, 2022 •
edited
Loading

As predicted by @sandreim even lower buckets don't really improve this visualisation. I will experiment with the values and add a new commit to see if I find some values that give better insight.

Here is the output from even smaller buckets than what is introduced by this PR (0.0025, 0.005, 0.1, 0.025, 0.05):

polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.0025"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.005"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.01"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.025"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.05"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.1"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.2"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.3"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.4"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.5"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.6"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="0.75"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="circulate_statement",chain="rococo_local_testnet",le="+Inf"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="circulate_statement",chain="rococo_local_testnet"} 0
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="circulate_statement",chain="rococo_local_testnet"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.0025"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.005"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.01"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.025"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.05"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.1"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.2"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.3"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.4"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.5"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.6"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="0.75"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="handle_incoming_message",chain="rococo_local_testnet",le="+Inf"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="handle_incoming_message",chain="rococo_local_testnet"} 0
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="handle_incoming_message",chain="rococo_local_testnet"} 1
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.0025"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.005"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.01"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.025"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.05"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.1"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.2"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.3"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.4"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.5"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.6"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="0.75"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="new_gossip_topology",chain="rococo_local_testnet",le="+Inf"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="new_gossip_topology",chain="rococo_local_testnet"} 0.000000041
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="new_gossip_topology",chain="rococo_local_testnet"} 3
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.0025"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.005"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.01"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.025"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.05"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.1"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.2"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.3"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.4"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.5"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.6"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="0.75"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_bucket{message_type="peer_view_change",chain="rococo_local_testnet",le="+Inf"} 36
polkadot_parachain_statement_distribution_network_bridge_update_v1_sum{message_type="peer_view_change",chain="rococo_local_testnet"} 0.0000008210000000000002
polkadot_parachain_statement_distribution_network_bridge_update_v1_count{message_type="peer_view_change",chain="rococo_local_testnet"} 36

bredamatt
bredamatt commented Nov 29, 2022
View reviewed changes
node/network/statement-distribution/src/metrics.rs Outdated
@@ -18,7 +18,7 @@ use polkadot_node_subsystem_util::metrics::{self, prometheus};

/// Buckets more suitable for checking the typical latency values
const HISTOGRAM_LATENCY_BUCKETS: &[f64] =
&[0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.75, 0.9, 1.0, 1.2, 1.5, 1.75];
&[0.000025, 0.00005, 0.000075, 0.0001, 0.0003125, 0.000625, 0.00125, 0.0025, 0.005, 0.01, 0.025, 0.05, 0.1];
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Locally these buckets don't make a big difference, but I am interested in seeing if they do in a more realistic scenario.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some realistic values I would look at some historical data from Versi. Even if we don't have these buckets, the sum and count of samples allow us to compute some average value which could be used to estimated some good buckets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

@bredamatt bredamatt changed the title Add two buckets on lower end of distribution to network bridge latency metrics Add buckets on lower end of distribution to network bridge latency metrics Nov 29, 2022
vstakhov
vstakhov approved these changes Dec 2, 2022
View reviewed changes
Copy link
Contributor

@vstakhov vstakhov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks matching with the data we have (I'm not 100% sure about 25 usec bucket though). It would also be nice to fix the fmt failure.

@sandreim
Copy link
Contributor

sandreim commented Dec 2, 2022

bot merge

@paritytech-processbot
Copy link

Error: Statuses failed for 73f2bf6

@sandreim sandreim merged commit 3a918d4 into master Dec 2, 2022
@sandreim sandreim deleted the bredamattia/network-bridge-metrics-granularity branch December 2, 2022 16:28
ordian added a commit that referenced this pull request Dec 6, 2022
@ordian
Merge branch 'master' into ao-5535-followup
5f1758f 
* master:
  Bump (#6404)
  kusama whitelist pallet preimage dep upgrade (#6392)
  Make submission deposit reasonable (#6402)
  Companion of Substrate PR 12837 (#6385)
  crowdloan: Fix migration. (#6397)
  Bump Substrate (#6394)
  allow root with gov2 origins (#6390)
  Tweak some NIS params (#6393)
  OpenGov improvements for Kusama (#6372)
  Introduce NIS functionality into Kusama/Rococo (#6352)
  Set polkadot version in one place (#6095)
  Upgrade tokio to 1.22.0 (#6262)
  update deprecated alias `--all` (#6383)
  Reduce provisioner work (#6328)
  Add buckets on lower end of distribution to network bridge latency metrics (#6359)
  Kusama: approve/reject treasury prop by treasurer (#6354)
devdanco added a commit to gasp-xyz/polkadot that referenced this pull request Feb 14, 2023
@devdanco @kianenigma
@niklasad1 @bkchr @coderobe @tifecool @ordian @mrcnski @shawntabrizi @ggwpez @nuke-web3 @ruseinov @athei @eskimor @pepoviola @sandreim @michalkucharczyk @Szegoo @senseless @xlc @alvicsam @KiChjang @Ank4n @koushiro @Lederstrumpf @moh-eulith @gilescope @cheme @BradleyOlson64 @tdimitrov @kukabi @koute @joepetrowski @altonen @lexnv @tugytur @skunert @bredamatt @serban300 @acatangiu @muharem @amab8901 @dmitry-markin @gavofyork @joao-paulo-parity @rcny
Mangata dev v0.9.36 (#20)
04f7ef1 
* Companion for EPM duplicate submissions (paritytech#6115)

* make it work

* add migration

* fix

* Update utils/staking-miner/src/opts.rs

Co-authored-by: Niklas Adolfsson <[email protected]>

* Update utils/staking-miner/src/monitor.rs

* small tweaks

* Update utils/staking-miner/src/opts.rs

Co-authored-by: Bastian Köcher <[email protected]>

* fmt

* fix print'

* fmt

* update lockfile for {"substrate"}

Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>

* try and fix build (paritytech#6170)

* sync versions with current release (0.9.31) (paritytech#6176)

* Bump spec_version to 9310

* bump transaction_version (0.9.31) (paritytech#6171)

* Bump transaction_version for polkadot

* Bump transaction_version for kusama

* Bump transaction_version for rococo

* Bump transaction_version for westend

* Bump transaction_version for polkadot

* Bump transaction_version for kusama

* Bump transaction_version for rococo

* Bump transaction_version for westend

* Bump crate versions (0.9.31)

* Use a more typesafe approach for managing indexed data (paritytech#6150)

* Fix for issue #2403

* Nightly fmt

* Quick documentation fixes

* Default Implementation

* iter() function integrated

* Implemented iter functionalities

* Fmt

* small change

* updates node-network

* updates in dispute-coordinator

* Updates

* benchmarking fix

* minor fix

* test fixes in runtime api

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* Removal of [index], shorting of FromIterator, Renaming of GroupValidators to ValidatorGroups

* Removal of ops import

* documentation fixes for spell check

* implementation of generic type

* Refactoring

* Test and documentation fixes

* minor test fix

* minor test fix

* minor test fix

* Update node/network/statement-distribution/src/lib.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* removed IterMut

* Update node/core/dispute-coordinator/src/import.rs

Co-authored-by: Andronik <[email protected]>

* Update node/core/dispute-coordinator/src/initialized.rs

Co-authored-by: Andronik <[email protected]>

* Update primitives/src/v2/mod.rs

Co-authored-by: Andronik <[email protected]>

* fmt

* IterMut

* documentation update

Co-authored-by: Andronik <[email protected]>

* minor adjustments and new TypeIndex trait

* spelling fix

* TypeIndex fix

Co-authored-by: Andronik <[email protected]>

* Add missing prerequisite in README for implementers guide (paritytech#6181)

* Companion for #12457 (Bounded Multisig) (paritytech#6172)

* u16 -> u32

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Make some fixes to logging in PVF subsystem (paritytech#6180)

* Log exit status code for workers

* Make log for execute job conclusion match prepare job conclusion

Trace log for conclusion of prepare job:

```rs
gum::debug!(
	target: LOG_TARGET,
	validation_code_hash = ?artifact_id.code_hash,
	?worker,
	?rip,
	"prepare worker concluded",
);
```

Co-authored-by: parity-processbot <>

* Co #12558: Update `pallet-multisig` benches (paritytech#6188)

* Typo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add multisig weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update multisig weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* Update impl guide README (paritytech#6197)

* update impl guide readme

* Update README.md

* [Companion] StakingInterface adjustments (paritytech#6199)

* [Companion] StakingInterface adjustments

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Companion for update `wasm-opt` (paritytech#6209)

* Update cc

* Update regex

* Update thiserror

* Update anyhow

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Make ValidateUnsigned available on all chains for paras. (paritytech#6214)

Co-authored-by: eskimor <[email protected]>

* fix upgrade node test, use latest as base image (paritytech#6215)

* Check if approval voting db is empty on startup (paritytech#6219)

* Add approval voting db sanity check

Signed-off-by: Andrei Sandu <[email protected]>

* add newline back

Signed-off-by: Andrei Sandu <[email protected]>

* no need for overlay to read

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* PVF timeouts follow-up (paritytech#6151)

* Rename timeout consts and timeout parameter; bump leniency

* Update implementor's guide with info about PVFs

* Make glossary a bit easier to read

* Add a note to LENIENT_PREPARATION_TIMEOUT

* Remove PVF-specific section from glossary

* Fix some typos

* BlockId removal: refactor: BlockBackend::block_body (paritytech#6223)

* BlockId removal: refactor: BlockBackend::block_body

It changes the arguments of `BlockBackend::block_body` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Replace parachain/parathread boolean by enum (paritytech#6198)

* Replace parachain/parathread boolean by enum

* Address PR comments

* Update dependencies

* ParaType -> ParaKind

* Swap enum field order to avoid migration

* Rename paratype field to parakind

* Manual en-/decocing of Parakind

* Manual TypeInfo for ParaKind

* rename field back to parachain

* minor

* Update runtime/parachains/src/paras/mod.rs

Co-authored-by: Andrei Sandu <[email protected]>

* Manual serde Serialize and Deserialize for ParaKind

* cargo fmt

* Update runtime/parachains/src/paras/mod.rs

Co-authored-by: Andronik <[email protected]>

* Add test for serde_json encoding/decoding

* Move serde_json dep to dev-deps

Co-authored-by: Andrei Sandu <[email protected]>
Co-authored-by: Andronik <[email protected]>

* BlockId removal: refactor: Backend::justifications (paritytech#6229)

* BlockId removal: refactor: Backend::justifications

It changes the arguments of `Backend::justifications` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* formatting

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* BlockId removal: refactor: Backend::block_indexed_body (paritytech#6233)

* BlockId removal: refactor: Backend::block_indexed_body

It changes the arguments of `Backend::block_indexed_body` method from: `BlockId<Block>` to: `&Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Xcm-Simulator Docs (paritytech#6178)

* Xcm-Simulator Docs

* spelling

* examples

* better docs

Co-authored-by: parity-processbot <>

* Add stake.plus bootnodes for westend, kusama, polkadot (paritytech#6224)

* add stake.plus bootnodes

* add stake.plus bootnodes for westend, kusama and polkadot

Co-authored-by: senseless <[email protected]>
Co-authored-by: parity-processbot <>

* clean up executed runtime migrations (paritytech#6206)

* kusama: clean up executed migrations

* polkadot: clean up executed migrations

* westend: clean up executed migrations

* Co #12085: Update `k256` (paritytech#6238)

* Typo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Cargo.lock

* update lockfile for {"substrate"}

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>

* cleanup deps (paritytech#6242)

* Pipeline with ci image with rust 1.65 (paritytech#6243)

* Pipeline with ci image with rust 1.65

* fix a warning

* return production image

Co-authored-by: Andronik <[email protected]>

* BlockId removal: &Hash to Hash (paritytech#6246)

* BlockId removal: &Hash to Hash

It changes &Block::Hash argument to Block::Hash.

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* missing file corrected

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Increase max rewardable nominators (paritytech#6230)

* Increase max rewardable nominators

* update kusama as well

* Update polkadot inflation to take into account auctions (paritytech#5872)

* Update polkadot inflation to take into account auctions

* a possible solution -- but needs a rather untrivial data seeding

* some additional comments

* Use LOWEST_PUBLIC_ID as a guide to filter out system/common good para ids

* Fixes

* move tests

* fix

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Add a `last change` footer to the implementers guide (paritytech#6216)

* Add a `last change` footer to the implementers guide

Some of the newcomers were noticing outdated pages in the implementer's guide.
This idea came up as a heuristic for how up-to-date an individual page is.

* Update `build-implementers-guide` CI job

* Retry failed PVF prepare jobs (paritytech#6213)

* staking miner: remove needless queue check (paritytech#6221)

* staking miner: remove needless queue check

If the queue is full and the "mined solution" is better than solutions in the queue according to the "strategy"
then the solution with worse score will be kicked out from the queue.

Thus, the check can be removed completly.

* fix compile warns

* companion for fast unstake batching (paritytech#6253)

* update

* add migrations

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Make rolling session more resilient in case of long finality stalls (paritytech#6106)

* Impl dynamic window size. Keep sessions for unfinalized chain

Signed-off-by: Andrei Sandu <[email protected]>

* feedback

Signed-off-by: Andrei Sandu <[email protected]>

* Stretch also in contructor plus  tests

Signed-off-by: Andrei Sandu <[email protected]>

* review feedback

Signed-off-by: Andrei Sandu <[email protected]>

* fix approval-voting tests

Signed-off-by: Andrei Sandu <[email protected]>

* grunting: dispute coordinator tests

Signed-off-by: Andrei Sandu <[email protected]>

* add session window column

Signed-off-by: Andrei Sandu <[email protected]>

* integrate approval vote and fix tests

Signed-off-by: Andrei Sandu <[email protected]>

* fix rolling session tests

Signed-off-by: Andrei Sandu <[email protected]>

* Small refactor

Signed-off-by: Andrei Sandu <[email protected]>

* WIP, tests failing

Signed-off-by: Andrei Sandu <[email protected]>

* Fix approval voting tests

Signed-off-by: Andrei Sandu <[email protected]>

* fix dispute-coordinator tests

Signed-off-by: Andrei Sandu <[email protected]>

* remove uneeded param

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* fix loose ends

Signed-off-by: Andrei Sandu <[email protected]>

* allow failure and tests for it

Signed-off-by: Andrei Sandu <[email protected]>

* fix comment

Signed-off-by: Andrei Sandu <[email protected]>

* comment fix

Signed-off-by: Andrei Sandu <[email protected]>

* style fix

Signed-off-by: Andrei Sandu <[email protected]>

* new col doesn't need to be ordered

Signed-off-by: Andrei Sandu <[email protected]>

* fmt and spellcheck

Signed-off-by: Andrei Sandu <[email protected]>

* db persist tests

Signed-off-by: Andrei Sandu <[email protected]>

* Add v2 config and cols

Signed-off-by: Andrei Sandu <[email protected]>

* DB upgrade WIP

Signed-off-by: Andrei Sandu <[email protected]>

* Fix comments

Signed-off-by: Andrei Sandu <[email protected]>

* add todo

Signed-off-by: Andrei Sandu <[email protected]>

* update to parity-db to "0.4.2"

Signed-off-by: Andrei Sandu <[email protected]>

* migration complete

Signed-off-by: Andrei Sandu <[email protected]>

* One session window size

Signed-off-by: Andrei Sandu <[email protected]>

* fix merge damage

Signed-off-by: Andrei Sandu <[email protected]>

* fix build errors

Signed-off-by: Andrei Sandu <[email protected]>

* fmt

Signed-off-by: Andrei Sandu <[email protected]>

* comment fix

Signed-off-by: Andrei Sandu <[email protected]>

* fix build

Signed-off-by: Andrei Sandu <[email protected]>

* make error more explicit

Signed-off-by: Andrei Sandu <[email protected]>

* add comment

Signed-off-by: Andrei Sandu <[email protected]>

* refactor conflict merge

Signed-off-by: Andrei Sandu <[email protected]>

* rename col_data

Signed-off-by: Andrei Sandu <[email protected]>

* add doc comment

Signed-off-by: Andrei Sandu <[email protected]>

* fix build

Signed-off-by: Andrei Sandu <[email protected]>

* migration: move all cols to v2

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* Retry failed PVF execution (AmbiguousWorkerDeath) (paritytech#6235)

* Fix a couple of typos

* Retry failed PVF execution

PVF execution that fails due to AmbiguousWorkerDeath should be retried once.
This should reduce the occurrence of failures due to transient conditions.

Closes paritytech#6195

* Address a couple of nits

* Write tests; refactor (add `validate_candidate_with_retry`)

* Update node/core/candidate-validation/src/lib.rs

Co-authored-by: Andronik <[email protected]>

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Andronik <[email protected]>

* [Companion] Bound Election and Staking by MaxActiveValidators (paritytech#6157)

* add maximum winners to multi phase election provider

* fallback to noelection

* fmt

* missing values

* convert boundedvec to inner before sort

* dont clone

* pr feedback

* update lockfile for {"substrate"}

* run onchain election on westend benchmark

Co-authored-by: parity-processbot <>

* Companion for substrate#12560 (paritytech#6226)

* Companion for substrate#12560

Signed-off-by: koushiro <[email protected]>

* update num-format v0.4.0 ==> v0.4.3

* Fix

* update lockfile for {"substrate"}

Signed-off-by: koushiro <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>

* Companion for substrate#12530: Consolidate and deduplicate MMR API methods (paritytech#6167)

* histor. batch proof: make best block arg optional

* make generate_batch_proof stub for historical

* merge generate_{historical_}batch_proof functions

* merge generate_{batch_}proof functions

* merge verify_{batch_}proof functions

* merge verify_{batch_}proof_stateless functions

* rename BatchProof->Proof

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Add a few staking params to fast-runtime build (paritytech#5424)

Co-authored-by: Squirrel <[email protected]>

* State trie migration rococo runtime changes. (paritytech#6127)

* add state-trie-migration (warn key need to be changed)

* rococo root

* restore master benchs (weights from substrate are used).

* use ord_parameter macro.

* do not upgrade runtime version yet

* apply review changes

* to test ci

* Revert "to test ci"

This reverts commit 5df6c5c.

* test ci

* Revert "test ci"

This reverts commit 0747761.

Co-authored-by: parity-processbot <>

* Brad implementers guide revisions 2 (paritytech#6239)

* Add disputes subsystems fix

* Updated dispute approval vote import reasoning

* Improved wording of my changes

* Resolving issues brought up in comments

* Update disputes prioritisation in `dispute-coordinator` (paritytech#6130)

* Scraper processes CandidateBacked events

* Change definition of best-effort

* Fix `dispute-coordinator` tests

* Unit test for dispute filtering

* Clarification comment

* Add tests

* Fix logic

If a dispute is not backed, not included and not confirmed we
don't participate but we do import votes.

* Add metrics for refrained participations

* Revert "Add tests"

This reverts commit 7b8391a.

* Revert "Unit test for dispute filtering"

This reverts commit 92ba5fe.

* fix dispute-coordinator tests

* Fix scraping

* new tests

* Small fixes in guide

* Apply suggestions from code review

Co-authored-by: Andrei Sandu <[email protected]>

* Fix some comments and remove a pointless test

* Code review feedback

* Clarification comment in tests

* Some tests

* Reference counted `CandidateHash` in scraper

* Proper handling for Backed and Included candidates in scraper

Backed candidates which are not included should be kept for a
predetermined window of finalized blocks. E.g. if a candidate is backed
but not included in block 2, and the window size is 2, the same
candidate should be cleaned after block 4 is finalized.

Add reference counting for candidates in scraper. A candidate can be
added on multiple block heights so we have to make sure we don't clean
it prematurely from the scraper.

Add tests.

* Update comments in tests

* Guide update

* Fix cleanup logic for `backed_candidates_by_block_number`

* Simplify cleanup

* Make spellcheck happy

* Update tests

* Extract candidate backing logic in separate struct

* Code review feedback

* Treat  backed and included candidates in the same fashion

* Update some comments

* Small improvements in test

* spell check

* Fix some more comments

* clean -> prune

* Code review feedback

* Reword comment

* spelling

Co-authored-by: Andrei Sandu <[email protected]>

* approval-voting: remove redundant validation check (paritytech#6266)

* approval-voting: remove a redundant check

* candidate-validation: remove unreachable check

* remove fill_block (paritytech#6200)

Co-authored-by: parity-processbot <>

* fix a compilation warning (paritytech#6279)

Fixes paritytech#6277.

* Only report concluded if there is an actual dispute. (paritytech#6270)

* Only report concluded if there is an actual dispute.

Hence no "non"-disputes will be added to disputes anymore.

* Fix redundant check.

* Test for no onesided disputes.

Co-authored-by: eskimor <[email protected]>

* [ci] fix buildah image (paritytech#6281)

* Revert special casing of Kusama for grandpa rounds. (paritytech#6217)

Co-authored-by: eskimor <[email protected]>

* Fixes "for loop over an `Option`" warnings (paritytech#6291)

Was seeing these warnings when running `cargo check --all`:

```
warning: for loop over an `Option`. This is more readably written as an `if let` statement
    --> node/core/approval-voting/src/lib.rs:1147:21
     |
1147 |             for activated in update.activated {
     |                              ^^^^^^^^^^^^^^^^
     |
     = note: `#[warn(for_loops_over_fallibles)]` on by default
help: to check pattern in a loop use `while let`
     |
1147 |             while let Some(activated) = update.activated {
     |             ~~~~~~~~~~~~~~~         ~~~
help: consider using `if let` to clear intent
     |
1147 |             if let Some(activated) = update.activated {
     |             ~~~~~~~~~~~~         ~~~
```

My guess is that `activated` used to be a SmallVec or similar, as is
`deactivated`. It was changed to an `Option`, the `for` still compiled (it's
technically correct, just weird), and the compiler didn't catch it until now.

* companion for #12599 (paritytech#6290)

* companion for #12599

* update Cargo.lock

* use cargo path instead of diener

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* [ci] Improve pipeline stopper (paritytech#6300)

* [ci] Improve pipeline stopper

* break test-linux-stable

* fix test-linux-stable

* Provisioner should ignore unconfirmed disputes (paritytech#6294)

* Fix typos

* Filter unconfirmed disputes in provisioner -  random_selection

* Rework dispute coordinator to return `DisputeStatus` with
`ActiveDisputes` message.
* Rework the random_selection implementation of `select_disptues` in
  `provisioner` to return only confirmed disputes.

* Filter unconfirmed disputes in provisioner - prioritized_selection

* Add test for unconfirmed disputes handling

* Fix `dispute-distribution` tests

* Add Helikon boot nodes for Polkadot, Kusama and Westend. (paritytech#6240)

* Dedup subsystem name (paritytech#6305)

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* Change best effort queue behaviour in `dispute-coordinator` (paritytech#6275)

* Change best effort queue behaviour in `dispute-coordinator`

Use the same type of queue (`BTreeMap<CandidateComparator,
ParticipationRequest>`) for best effort and priority in
`dispute-coordinator`.

Rework `CandidateComparator` to handle unavailable parent
block numbers.

Best effort queue will order disputes the same way as priority does - by
parent's block height. Disputes on candidates for which the parent's
block number can't be obtained will be treated with the lowest priority.

* Fix tests: Handle `ChainApiMessage::BlockNumber` in `handle_sync_queries`

* Some tests are deadlocking on sending messages via overseer so change `SingleItemSink`to `mpsc::Sender` with a buffer of 1

* Fix a race in test after adding a buffered queue for overseer messages

* Fix the rest of the tests

* Guide update - best-effort queue

* Guide update: clarification about spam votes

* Fix tests in `availability-distribution`

* Update comments

* Add `make_buffered_subsystem_context` in `subsystem-test-helpers`

* Code review feedback

* Code review feedback

* Code review feedback

* Don't add best effort candidate if it is already in priority queue

* Remove an old comment

* Fix insert in best_effort

* [ci] fix build implementers guide (paritytech#6306)

* [ci] fix build implementers guide

* add comment

* rm git fetch from publish-docs

* Remove the `wasmtime` feature flag (companion for substrate#12684) (paritytech#6268)

* Remove the `wasmtime` feature flag

* Update `substrate` to the newest `master`

* Update `substrate` to the newest `master`

* Add `starts_with` to v0 and v1 MultiLocation (paritytech#6311)

* add `starts_with` to v0 and v1 MultiLocation

* add tests

* fmt

* Extend lower bound of `manage_lease_period_start` from `runtime_common::slots` (paritytech#6318)

* Update async-trait version to v0.1.58 (paritytech#6319)

* Update async-trait version

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Add PVF module documentation (paritytech#6293)

* Add PVF module documentation

TODO (once the PRs land):

- [ ] Document executor parametrization.

- [ ] Document CPU time measurement of timeouts.

* Update node/core/pvf/src/lib.rs

Co-authored-by: Andrei Sandu <[email protected]>

* Clarify meaning of PVF acronym

* Move PVF doc to implementer's guide

* Clean up implementer's guide a bit

* Add page for PVF types

* pvf: Better separation between crate docs and implementer's guide

* ci: Add "prevalidating" to the dictionary

* ig: Remove types/chain.md

The types contained therein did not exist and the file was not referenced
anywhere.

Co-authored-by: Andrei Sandu <[email protected]>

* Rate limit improvements (paritytech#6315)

* We actually don't need to rate limit redundant requests.

Those redundant requests should not actually happen, but still.

* Add some logging.

* Also log message when the receiving side hit the rate limit.

* Update node/network/dispute-distribution/src/sender/mod.rs

Co-authored-by: Alexandru Vasile <[email protected]>

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Alexandru Vasile <[email protected]>

* [ci] fix build-implementers-guide (paritytech#6335)

* [ci] fix build-implementers-guide

* fix user

* Added Amforc bootnodes for Polkadot and Kusama (paritytech#6077)

* add wss and update dns

* fix wrong node id

* ig: Fix description of execution retry delay (paritytech#6342)

The delay is 3s and not 1s. I removed the reference to a specific number of
seconds as it may be too specific for a high-level description.

* Add support for outbound only configs on request/response protocols (paritytech#6343)

* Add option ot add outbound_only configurations

* Improve comment

* cargo update -p sp-io (paritytech#6351)

* Add more granularity to prometheus histogram buckets (paritytech#6348)

* Add buckets below 5ms

* Add more specific histogram buckets

* Add more specific buckets

* cargo fmt

* Provide some more granular metrics for polkadot_pvf_execution_time (paritytech#6346)

* [ci] fix implementer guide job (paritytech#6357)

* [DNM] debug implementer guide job

* remove git depth

* change git strategy

* add git depth

* try k8s runner

* fix k8s template

* test job

* fix test

* fix

* return pipeline back

* enable disabled deploy-parity-testnet

* dispute-coordinator: fix earliest session checks for pruning and import (paritytech#6358)

* RollingSession: add fn contains

Signed-off-by: Andrei Sandu <[email protected]>

* handle_import_statements fix ancient dispute check

Signed-off-by: Andrei Sandu <[email protected]>

* work with earliest session instead of latest

Signed-off-by: Andrei Sandu <[email protected]>

* update comment

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* remove executed migrations (0.9.33) (paritytech#6364)

* Companion for: pallet-mmr: move offchain logic to client-side gadget (paritytech#6321)

* Spawn MMR gadget when offchain indexing is enabled

* companion PR code review changes: 1st iteration

* Code review changes: 2nd iteration

* update lockfile for {"substrate"}

Co-authored-by: acatangiu <[email protected]>
Co-authored-by: parity-processbot <>

* Remove `parity-util-mem` from `runtime-api` cache (paritytech#6367)

* Add Collectives as Trusted Teleporter (paritytech#6326)

* add collectives as trusted teleporter

* fix statemint-specific doc

* sync versions with current release (0.9.33) (paritytech#6363)

* westend: update transaction version

* polkadot: update transaction version

* kusama: update transaction version

* Bump spec_version to 9330

* bump versions to 0.9.33

* Clippyfy (paritytech#6341)

* Add clippy config and remove .cargo from gitignore

* first fixes

* Clippyfied

* Add clippy CI job

* comment out rusty-cachier

* minor

* fix ci

* remove DAG from check-dependent-project

* add DAG to clippy

Co-authored-by: alvicsam <[email protected]>

* Companion for: MMR: move RPC code from frame/ to client/ (paritytech#6369)

* rpc: mmr rpc crate name change

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* support opengov calls in proxy definitions (paritytech#6366)

* Use CPU clock timeout for PVF jobs (paritytech#6282)

* Put in skeleton logic for CPU-time-preparation

Still needed:
- Flesh out logic
- Refactor some spots
- Tests

* Continue filling in logic for prepare worker CPU time changes

* Fix compiler errors

* Update lenience factor

* Fix some clippy lints for PVF module

* Fix compilation errors

* Address some review comments

* Add logging

* Add another log

* Address some review comments; change Mutex to AtomicBool

* Refactor handling response bytes

* Add CPU clock timeout logic for execute jobs

* Properly handle AtomicBool flag

* Use `Ordering::Relaxed`

* Refactor thread coordination logic

* Fix bug

* Add some timing information to execute tests

* Add section about the mitigation to the IG

* minor: Change more `Ordering`s to `Relaxed`

* candidate-validation: Fix build errors

* guide: remove refences to outdated secondary checkers (paritytech#6309)

* guide: remove refences to outdated secondary checkers

* Update roadmap/implementers-guide/src/glossary.md

* guide: remove refences to Fisherman

* revert changes to roadmap/parachains.md

* Kusama: approve/reject treasury prop by treasurer (paritytech#6354)

* Add buckets on lower end of distribution to network bridge latency metrics (paritytech#6359)

* Add two more buckets on lower end of distribution

* add even smaller buckets

* cargo fmt

* Reduce provisioner work (paritytech#6328)

* Store values needed to create inherent data when needed instead of creating them early on

* Point deps to substrate branch

* Arc the client

* Cargo update

* Fix main cargo files

* Undo cargo file changes

* Add overseer dep to inherents

* Update deps

* Simplify code

* Update benchmark

* Update node/client/src/benchmarking.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Revert "Update node/core/parachains-inherent/src/lib.rs"

This reverts commit 8b9555d.

* Revert "Update node/core/parachains-inherent/src/lib.rs"

This reverts commit 816c92d.

* cargo update -p sp-io

* fmt

Co-authored-by: Bastian Köcher <[email protected]>

* update deprecated alias `--all` (paritytech#6383)

`--all` is a deprecated alias for `--workspace` (https://doc.rust-lang.org/cargo/commands/cargo-test.html)

* Upgrade tokio to 1.22.0 (paritytech#6262)

Co-authored-by: Sebastian Kunert <[email protected]>

* Set polkadot version in one place (paritytech#6095)

* rust 1.64 enables workspace properties

* add edition, repository and authors.

* of course, update the version in one place.

Co-authored-by: Andronik <[email protected]>

* Introduce NIS functionality into Kusama/Rococo (paritytech#6352)

* Integrate NIS into Kusama/Rococo

* Missing files

* Fix

* Fix

* Formatting

* Add Kusama weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add Rococo weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump

* Bump

* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_nis

* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_nis

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

* OpenGov improvements for Kusama (paritytech#6372)

* Tweaks to optimise gov2

* Use new inactive funds

* Introduce migrations

* Fixes

* Fixes

* Fixes

* Some constant updates for Fellowship

* Further tweaks

* Lower floor for whitelisted

* Tweak some NIS params (paritytech#6393)

* Reduce base period, increase queue count

* Reduce base period, increase queue count

* allow root with gov2 origins (paritytech#6390)

* Bump Substrate (paritytech#6394)

* crowdloan: Fix migration. (paritytech#6397)

The migration would not have been run because of the `current_version ==
1` check.

* Companion of Substrate PR 12837 (paritytech#6385)

* rename some crates for publishing to crates.io

* s/remote-ext/frame-remote-externalities

* cargo update

* Make submission deposit reasonable (paritytech#6402)

* kusama whitelist pallet preimage dep upgrade (paritytech#6392)

* Bump (paritytech#6404)

* Companion for paritytech/substrate#12795 (paritytech#6374)

* Begin removing `parity-util-mem`; remove `collect_memory_stats`

* Update some dependencies that were using `parity-util-mem`

* Remove `trie-memory-tracker` feature

* Update Cargo.lock

* Update `kvdb-shared-tests`

* Add back jemalloc

* Add missing license header

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
Co-authored-by: Andronik <[email protected]>

* Let the PVF host kill the worker on timeout (paritytech#6381)

* Let the PVF host kill the worker on timeout

* Fix comment

* Fix inaccurate comments; add missing return statement

* Fix a comment

* Fix comment

* Companion for paritytech/substrate#12788 (paritytech#6360)

* Companion for paritytech/substrate#12788

* migrations

* rustfmt

* update lockfile for {"substrate"}

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Make sure to preserve backing votes (paritytech#6382)

* Guide updates

* Consider more dead forks.

* Ensure backing votes don't get overridden.

* Fix spelling.

* Fix comments.

* Update node/primitives/src/lib.rs

Co-authored-by: Tsvetomir Dimitrov <[email protected]>

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>

* Refactoring to condense disputes tests (paritytech#6395)

* Refactoring to condense disputes tests

* Removing unhelpful comment lines

* Addressing Tsveto's suggestions

* Fixing formatting nit

* [ci] Adjust check-runtime-migration job (paritytech#6107)

* [WIP][ci] Add node to check-runtime-migration job

* add image

* remove sscache from before_script

* add nodes

* restart pipeline

* restart pipeline2

* disable other jobs

* debug

* remove debug

* add ports

* restart pipeline

* restart pipeline

* restart pipeline

* try kusama first

* run polkadot 1st

* disable some jobs

* try command from command bot

* cargo run

* test passing variables

* run without condition

* adjust kusama and westend

* fix

* return jobs

* fix small nits

* split check-runtime-migration

Co-authored-by: parity-processbot <>

* update weights (sync with 0.9.33) (paritytech#6362)

* update weights (0.9.33) (paritytech#6299)

* kusama: update weights

* polkadot: update weights

* rococo: update weights

* westend: update weights

* fix deps

* Resolve merge

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Reset Kusama whitelist weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Feature gate test `benchmark_storage_works` (paritytech#6376)

* Feature gate storage bench test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* typo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* Companion for paritytech/substrate#12868 (paritytech#6406)

* Replace WEIGHT_PER_* with WEIGHT_REF_TIME_PER_*

* cargo fmt

* Update substrate

* OpenGov: Tweak parameters further (paritytech#6416)

* Tweak parameters further

* Further tweaks to deposits

* companion for #12663 jsonrpsee v0.16 (paritytech#6339)

* companion for #12663 jsonrpsee v0.16.1

* update substrate

* merge master

* Update rpc/Cargo.toml

Co-authored-by: Alexandru Vasile <[email protected]>

* Update utils/staking-miner/Cargo.toml

Co-authored-by: Alexandru Vasile <[email protected]>

* update lockfile for {"substrate"}

Co-authored-by: Alexandru Vasile <[email protected]>
Co-authored-by: parity-processbot <>

* CI: Remove the excessive variable declaration (paritytech#6426)

* companion slash chilling update (paritytech#6424)

* update weights

* goddamit

* update weights

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* swap error responses (paritytech#6421)

* [ci] fix check-transaction-versions (paritytech#6425)

* [ci] fix check-transaction-versions

* allow fail the job

* approval-distribution: batched approval/assignment sending (paritytech#6401)

* Imple batched send

Signed-off-by: Andrei Sandu <[email protected]>

* Add batch tests

Signed-off-by: Andrei Sandu <[email protected]>

* pub MAX_NOTIFICATION_SIZE

Signed-off-by: Andrei Sandu <[email protected]>

* spell check

Signed-off-by: Andrei Sandu <[email protected]>

* spellcheck ...

Signed-off-by: Andrei Sandu <[email protected]>

* Fix comment

Signed-off-by: Andrei Sandu <[email protected]>

* o.O

Signed-off-by: Andrei Sandu <[email protected]>

* 2 constants

Signed-off-by: Andrei Sandu <[email protected]>

* Ensure batch size is at least 1 element

Signed-off-by: Andrei Sandu <[email protected]>

* feedback impl

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* add westend bootnode (paritytech#6434)

* [ci] add job switcher (paritytech#6433)

* [ci] add job switcher

* add before_script to docker and k8s runners

* upd runners

* sccache :(

* companion for try-runtime revamp (paritytech#6187)

* update to reflect latest try-runtime stuff

* update to latest version

* fix

* fix miner

* update

* update

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Fix wrong rate limit + add a few logs. (paritytech#6440)

* Add trace log

* More tracing.

* Fix redundant rate limit.

* Add trace log.

* Improve logging.

Co-authored-by: eskimor <[email protected]>

* Bump workspace unified version

* Bump spec_version to 9360

* fix diener output

* bump substrate

* clean up migrations

* bump version (0.9.36)

* sync transaction_version with 0.9.35 (compatible)

* sync transaction_version with 0.9.35 (compatible)

* update weights (0.9.36) (paritytech#6450)

* polkadot: update weights (0.9.36)

* kusama: update weights (0.9.36)

* rococo: update weights (0.9.36)

* westend: update weights (0.9.36)

* delete unnecessary files

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Signed-off-by: Andrei Sandu <[email protected]>
Signed-off-by: koushiro <[email protected]>
Co-authored-by: Kian Paimani <[email protected]>
Co-authored-by: Niklas Adolfsson <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Mara Robin B <[email protected]>
Co-authored-by: Boluwatife Bakre <[email protected]>
Co-authored-by: Andronik <[email protected]>
Co-authored-by: Marcin S <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Dan Shields <[email protected]>
Co-authored-by: Roman Useinov <[email protected]>
Co-authored-by: Alexander Theißen <[email protected]>
Co-authored-by: eskimor <[email protected]>
Co-authored-by: eskimor <[email protected]>
Co-authored-by: Javier Viola <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>
Co-authored-by: Michal Kucharczyk <[email protected]>
Co-authored-by: alexgparity <[email protected]>
Co-authored-by: Sergej Sakac <[email protected]>
Co-authored-by: Tom <[email protected]>
Co-authored-by: senseless <[email protected]>
Co-authored-by: Xiliang Chen <[email protected]>
Co-authored-by: Alexander Samusev <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Ankan <[email protected]>
Co-authored-by: Qinxuan Chen <[email protected]>
Co-authored-by: Robert Hambrock <[email protected]>
Co-authored-by: moh-eulith <[email protected]>
Co-authored-by: Squirrel <[email protected]>
Co-authored-by: cheme <[email protected]>
Co-authored-by: Bradley Olson <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>
Co-authored-by: Kutsal Kaan Bilgin <[email protected]>
Co-authored-by: Koute <[email protected]>
Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Aaro Altonen <[email protected]>
Co-authored-by: Alexandru Vasile <[email protected]>
Co-authored-by: tugy <[email protected]>
Co-authored-by: Sebastian Kunert <[email protected]>
Co-authored-by: Mattia L.V. Bradascio <[email protected]>
Co-authored-by: Serban Iorga <[email protected]>
Co-authored-by: acatangiu <[email protected]>
Co-authored-by: alvicsam <[email protected]>
Co-authored-by: Muharem Ismailov <[email protected]>
Co-authored-by: amab8901 <[email protected]>
Co-authored-by: Dmitry Markin <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>
Co-authored-by: João Paulo Silva de Souza <[email protected]>
Co-authored-by: Vlad <[email protected]>
devdanco added a commit to gasp-xyz/polkadot that referenced this pull request May 25, 2023
@devdanco @ordian
@muharem @bredamatt @bkchr @amab8901 @dmitry-markin @skunert @gilescope @gavofyork @ggwpez @joepetrowski @joao-paulo-parity @mrcnski @eskimor @tdimitrov @BradleyOlson64 @alvicsam @coderobe @KiChjang @niklasad1 @lexnv @rcny @kianenigma @sandreim @tugytur @drahnr @michalkucharczyk @acatangiu @sergejparity @dependabot @Ank4n @serban300 @melekes @hirschenberger @s0me0ne-unkn0wn @chevdor @cheme @Maharacha @apopiak @stanly-johnson @nanocryk @bkontur @rphmeier @vgeddes @shawntabrizi @sorpaas @koute @Lederstrumpf @Szegoo @pmensik @arkpar @notlesh @pepoviola @andresilva @DaviRain-Su @rzadp @miloskriz @altonen @tonyalaribe @cocoyoon @girazoki @samelamin @pmikolajczyk41 @gpestana @mrshiposha @paulormart @daredevil3435 @louismerlin @mateo-moon @davxy @antonva @jiguantong @vivekvpandya @vstakhov @xlc
Mangata dev v0.9.40 (#23)
c484ce0 
* guide: remove refences to outdated secondary checkers (#6309)

* guide: remove refences to outdated secondary checkers

* Update roadmap/implementers-guide/src/glossary.md

* guide: remove refences to Fisherman

* revert changes to roadmap/parachains.md

* Kusama: approve/reject treasury prop by treasurer (#6354)

* Add buckets on lower end of distribution to network bridge latency metrics (#6359)

* Add two more buckets on lower end of distribution

* add even smaller buckets

* cargo fmt

* Reduce provisioner work (#6328)

* Store values needed to create inherent data when needed instead of creating them early on

* Point deps to substrate branch

* Arc the client

* Cargo update

* Fix main cargo files

* Undo cargo file changes

* Add overseer dep to inherents

* Update deps

* Simplify code

* Update benchmark

* Update node/client/src/benchmarking.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update node/core/parachains-inherent/src/lib.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Revert "Update node/core/parachains-inherent/src/lib.rs"

This reverts commit 8b9555dc2451acfabab173d259e00da2728b7aa2.

* Revert "Update node/core/parachains-inherent/src/lib.rs"

This reverts commit 816c92d0e001e71f677d0acbcf22bdc3f511bc56.

* cargo update -p sp-io

* fmt

Co-authored-by: Bastian Köcher <[email protected]>

* update deprecated alias `--all` (#6383)

`--all` is a deprecated alias for `--workspace` (https://doc.rust-lang.org/cargo/commands/cargo-test.html)

* Upgrade tokio to 1.22.0 (#6262)

Co-authored-by: Sebastian Kunert <[email protected]>

* Set polkadot version in one place (#6095)

* rust 1.64 enables workspace properties

* add edition, repository and authors.

* of course, update the version in one place.

Co-authored-by: Andronik <[email protected]>

* Introduce NIS functionality into Kusama/Rococo (#6352)

* Integrate NIS into Kusama/Rococo

* Missing files

* Fix

* Fix

* Formatting

* Add Kusama weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add Rococo weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump

* Bump

* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_nis

* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_nis

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>

* OpenGov improvements for Kusama (#6372)

* Tweaks to optimise gov2

* Use new inactive funds

* Introduce migrations

* Fixes

* Fixes

* Fixes

* Some constant updates for Fellowship

* Further tweaks

* Lower floor for whitelisted

* Tweak some NIS params (#6393)

* Reduce base period, increase queue count

* Reduce base period, increase queue count

* allow root with gov2 origins (#6390)

* Bump Substrate (#6394)

* crowdloan: Fix migration. (#6397)

The migration would not have been run because of the `current_version ==
1` check.

* Companion of Substrate PR 12837 (#6385)

* rename some crates for publishing to crates.io

* s/remote-ext/frame-remote-externalities

* cargo update

* Make submission deposit reasonable (#6402)

* kusama whitelist pallet preimage dep upgrade (#6392)

* Bump (#6404)

* Companion for paritytech/substrate#12795 (#6374)

* Begin removing `parity-util-mem`; remove `collect_memory_stats`

* Update some dependencies that were using `parity-util-mem`

* Remove `trie-memory-tracker` feature

* Update Cargo.lock

* Update `kvdb-shared-tests`

* Add back jemalloc

* Add missing license header

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>
Co-authored-by: Andronik <[email protected]>

* Let the PVF host kill the worker on timeout (#6381)

* Let the PVF host kill the worker on timeout

* Fix comment

* Fix inaccurate comments; add missing return statement

* Fix a comment

* Fix comment

* Companion for paritytech/substrate#12788 (#6360)

* Companion for paritytech/substrate#12788

* migrations

* rustfmt

* update lockfile for {"substrate"}

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Make sure to preserve backing votes (#6382)

* Guide updates

* Consider more dead forks.

* Ensure backing votes don't get overridden.

* Fix spelling.

* Fix comments.

* Update node/primitives/src/lib.rs

Co-authored-by: Tsvetomir Dimitrov <[email protected]>

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>

* Refactoring to condense disputes tests (#6395)

* Refactoring to condense disputes tests

* Removing unhelpful comment lines

* Addressing Tsveto's suggestions

* Fixing formatting nit

* [ci] Adjust check-runtime-migration job (#6107)

* [WIP][ci] Add node to check-runtime-migration job

* add image

* remove sscache from before_script

* add nodes

* restart pipeline

* restart pipeline2

* disable other jobs

* debug

* remove debug

* add ports

* restart pipeline

* restart pipeline

* restart pipeline

* try kusama first

* run polkadot 1st

* disable some jobs

* try command from command bot

* cargo run

* test passing variables

* run without condition

* adjust kusama and westend

* fix

* return jobs

* fix small nits

* split check-runtime-migration

Co-authored-by: parity-processbot <>

* update weights (sync with 0.9.33) (#6362)

* update weights (0.9.33) (#6299)

* kusama: update weights

* polkadot: update weights

* rococo: update weights

* westend: update weights

* fix deps

* Resolve merge

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Reset Kusama whitelist weights

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Feature gate test `benchmark_storage_works` (#6376)

* Feature gate storage bench test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* typo

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* Companion for paritytech/substrate#12868 (#6406)

* Replace WEIGHT_PER_* with WEIGHT_REF_TIME_PER_*

* cargo fmt

* Update substrate

* OpenGov: Tweak parameters further (#6416)

* Tweak parameters further

* Further tweaks to deposits

* companion for #12663 jsonrpsee v0.16 (#6339)

* companion for #12663 jsonrpsee v0.16.1

* update substrate

* merge master

* Update rpc/Cargo.toml

Co-authored-by: Alexandru Vasile <[email protected]>

* Update utils/staking-miner/Cargo.toml

Co-authored-by: Alexandru Vasile <[email protected]>

* update lockfile for {"substrate"}

Co-authored-by: Alexandru Vasile <[email protected]>
Co-authored-by: parity-processbot <>

* CI: Remove the excessive variable declaration (#6426)

* companion slash chilling update (#6424)

* update weights

* goddamit

* update weights

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* swap error responses (#6421)

* [ci] fix check-transaction-versions (#6425)

* [ci] fix check-transaction-versions

* allow fail the job

* approval-distribution: batched approval/assignment sending (#6401)

* Imple batched send

Signed-off-by: Andrei Sandu <[email protected]>

* Add batch tests

Signed-off-by: Andrei Sandu <[email protected]>

* pub MAX_NOTIFICATION_SIZE

Signed-off-by: Andrei Sandu <[email protected]>

* spell check

Signed-off-by: Andrei Sandu <[email protected]>

* spellcheck ...

Signed-off-by: Andrei Sandu <[email protected]>

* Fix comment

Signed-off-by: Andrei Sandu <[email protected]>

* o.O

Signed-off-by: Andrei Sandu <[email protected]>

* 2 constants

Signed-off-by: Andrei Sandu <[email protected]>

* Ensure batch size is at least 1 element

Signed-off-by: Andrei Sandu <[email protected]>

* feedback impl

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* add westend bootnode (#6434)

* [ci] add job switcher (#6433)

* [ci] add job switcher

* add before_script to docker and k8s runners

* upd runners

* sccache :(

* companion for try-runtime revamp (#6187)

* update to reflect latest try-runtime stuff

* update to latest version

* fix

* fix miner

* update

* update

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Fix wrong rate limit + add a few logs. (#6440)

* Add trace log

* More tracing.

* Fix redundant rate limit.

* Add trace log.

* Improve logging.

Co-authored-by: eskimor <[email protected]>

* Make sure errors get logged (#6451)

instead of silently dropped.

Co-authored-by: eskimor <[email protected]>

* Adding some more logs (#6455)

* Adding some more logs

More logs are always better! Joke aside, these logs help to debug
certain issues.

* Apply suggestions from code review

* [bump] orchestra v0.0.4 (#6413)

* update orchestra to v0.0.3

* fix locks

* update orchestra to v0.0.4

* update lock

* Use explicit call indices (#6449)

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use explicit call indices

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Update Substrate (#6457)

* BlockId removal: refactor: HeaderBackend::header (#6418)

* BlockId removal: refactor: HeaderBackend::header

It changes the arguments of:
- `HeaderBackend::header`,
- `Client::header`

methods from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* missed fixes

* BlockId removal: refactor: HeaderBackend::expect_header

It changes the arguments of `HeaderBackend::expect_header` method from: `BlockId<Block>` to: `Block::Hash`

* update lockfile for {"substrate"}

* misspell fixed

Co-authored-by: parity-processbot <>

* PVF preparation: do not conflate errors (#6384)

* PVF preparation: do not conflate errors

+ Adds some more granularity to the prepare errors.
+ Better distinguish whether errors occur on the host side or the worker.
+ Do not kill the worker if the error happened on the host side.
+ Do not retry preparation if the error was `Panic`.
+ Removes unnecessary indirection with `Selected` type.

* Add missing docs, resolve TODOs

* Address review comments and remove TODOs

* Fix error in CI

* Undo unnecessary change

* Update couple of comments

* Don't return error for stream shutdown

* Update node/core/pvf/src/worker_common.rs

* [ci] New try-runtime command (#6445)

* [ci] New try-runtime command

* restart pipeline

* build features

* fix --features flag

* rm --execution flag

* remove --no-spec-check-panic flag

* change target dir

* debug ccargo target dir

* return target

* try different location for runtime wasm

* build only the runtime crate

* check all generated .wasm files

* adjust paths again

* remove debug command

Co-authored-by: joao-paulo-parity <[email protected]>

* Kusama origins as xcm multi_location (#6273)

* Kusamsa origins as xcm multilocation

* Fellows origin index

* origins to xcm plurality body

* cleanup

* fix cargo spellcheck

* Apply suggestions from code review

Co-authored-by: joe petrowski <[email protected]>

* include Fellows into scope

* include Fellows into scope

Co-authored-by: joe petrowski <[email protected]>
Co-authored-by: Gavin Wood <[email protected]>

* BlockId removal: refactor: HeaderBackend::status (#6459)

* BlockId removal: refactor: HeaderBackend::status

It changes the arguments of `HeaderBackend::status` method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* Update node/core/chain-api/src/tests.rs

Co-authored-by: Adrian Catangiu <[email protected]>

* unused import removed

* update lockfile for {"substrate"}

Co-authored-by: Adrian Catangiu <[email protected]>
Co-authored-by: parity-processbot <>

* cargo update -p libp2p-yamux (#6464)

* fix dependabot labels (#6468)

* Fix runtime-migration label detection (#6469)

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump secp256k1 from 0.24.0 to 0.24.2 (#6411)

Bumps [secp256k1](https://github.com/rust-bitcoin/rust-secp256k1) from 0.24.0 to 0.24.2.
- [Release notes](https://github.com/rust-bitcoin/rust-secp256k1/releases)
- [Changelog](https://github.com/rust-bitcoin/rust-secp256k1/blob/secp256k1-0.24.2/CHANGELOG.md)
- [Commits](https://github.com/rust-bitcoin/rust-secp256k1/compare/secp256k1-0.24.0...secp256k1-0.24.2)

---
updated-dependencies:
- dependency-name: secp256k1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parity-processbot <>

* [Companion] Allow StakingAdmin to set min_commission (#6444)

* rename staking origin

* fix comments

* rename origin

* give fake weight before re-benchmarking

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_staking

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_staking

* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_staking

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Added comment describing satisfied invariant (#6460)

* Runtime diagnostics for leaked messages in unbounded channels (part 2) (#6481)

* Bump `backtrace` to v0.3.67

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Update disputes subsytems section. (#6329)

* Update disputes subsytems section.

* Update roadmap/implementers-guide/src/node/disputes/README.md

Co-authored-by: Andrei Sandu <[email protected]>

* Update roadmap/implementers-guide/src/node/disputes/README.md

Co-authored-by: Andrei Sandu <[email protected]>

* Update roadmap/implementers-guide/src/node/disputes/README.md

Co-authored-by: ordian <[email protected]>

* Update roadmap/implementers-guide/src/node/disputes/README.md

Co-authored-by: ordian <[email protected]>

* Review feedback.

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Andrei Sandu <[email protected]>
Co-authored-by: ordian <[email protected]>

* Update dispute participation on active leaves update (#6303)

* Passed candidate events from scraper to participation

* First draft PR 5875

* Added support for timestamp in changes

* Some necessary refactoring

* Removed SessionIndex from unconfirmed_disputes key

* Removed duplicate logic in import statements

* Replaced queue_participation call with re-prio

* Simplifying refactor. Backed were already handled

* Removed unneeded spam slots logic

* Implementers guide edits

* Undid the spam slots refactor

* Added comments and implementers guide edit

* Added test for participation upon backing

* Round of fixes + ran fmt

* Round of changes + fmt

* Error handling draft

* Changed errors to bubble up from reprioritization

* Starting to construct new test

* Clarifying participation function rename

* Reprio test draft

* Very rough bump to priority queue test draft

* Improving logging

* Most concise reproduction of error on third import

* Add `handle_approval_vote_request`

* Removing reprioritization on included event test

* Removing unneeded test config

* cargo fmt

* Test works

* Fixing final nits

* Tweaks to test Tsveto figured out

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Tsvetomir Dimitrov <[email protected]>

* BlockId removal: refactor: BlockBackend::block|block_status (#6477)

* BlockId removal: refactor: BlockBackend::block|block_status

It changes the arguments of:
-  `BlockBackend::block`
-  `BlockBackend::block_status`

method from: `BlockId<Block>` to: `Block::Hash`

This PR is part of BlockId::Number refactoring analysis (paritytech/substrate#11292)

* update lockfile for {"substrate"}

* ".git/.scripts/fmt.sh"

Co-authored-by: parity-processbot <>

* Deploy scheduler agenda cleanup migration (#6465)

* Add migration

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Add comments

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Migrate Staking pallet to v13 (#6365)

* add staking v13 migration

* add migration to polkadot

* only migrate staking

* un-remove other migrations

* fmt

* retrigger ci

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Co #13045: Selectable on-runtime-upgrade checks (#6498)

* Use UpgradeCheckSelector

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>

* Add `try-runtime` to Rococo runtime (#6501)

* Add `try-runtime` to Rococo runtime

* Enable try-runtime in all pallets

* Update runtime/rococo/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Add dev-dependencies for the try-runtime test

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Fix `polkadot-runtime-constants` std build (#6503)

* Fix polkadot-runtime-constants std build

* Fix kusama, rococo and westend constants std build

* Code review fixes

* Issue 6274: keeping all backing votes in provisioner vote set (#6494)

* Fixing filter to keep all backing votes

* Comment and implementers guide edit

* Formatting

* Using fallthrough

* Fmt

* Log PVF retries (#6504)

* av-store: write meta for unknown finalized blocks (#6452)

* av-store: write meta for unknown finalized blocks

* fix test

* upgrade libp2p to 0.50.0 (#6500)

* update asynchronous-codec

* update futures-rustls

* update rustls

* update ipnet

* update once_cell

* update futures

* update futures and its deps

* update lockfile for {"substrate"}

* Bump zombienet test timeout from 30 to 60 s

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Revert "Bump zombienet test timeout from 30 to 60 s"

This reverts commit 657738dfffe3234195a3b5b491f08e4660ca7bad.

* Update Substrate

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>

* Update serde because substrate->trybuild needs a newer version (#6505)

companion PR for https://github.com/paritytech/substrate/pull/13073

* [ci] Remove check-transaction-versions job (#6509)

Co-authored-by: parity-processbot <>

* Unlimited Proof size per block (#6085)

* Unlimited Proof size per block

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fix

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use `polkadot-node-metrics` where possible (#6484)

See https://github.com/paritytech/polkadot/issues/6015. (Doesn't close yet,
there might be more we can do.)

* disputes pallet: Remove spam slots (#6345)

* disputes pallet: Filter disputes with votes less than supermajority threshold

* Remove `max_spam_slots` usages

* Remove `SpamSlots`

* Remove `SpamSlotChange`

* Remove `Error<T>::PotentialSpam` and stale comments

* `create_disputes_with_no_spam` -> `create_disputes`

* Make tests compile - wip commit

* Rework `test_dispute_timeout`. Rename `update_spam_slots` to `filter_dispute_set`

* Remove `dispute_statement_becoming_onesided_due_to_spamslots_is_accepted` and `filter_correctly_accounts_spam_slots` -> they bring no value with removed spam slots

* Fix `test_provide_multi_dispute_success_and_other`

* Remove an old comment

* Remove spam slots from tests - clean todo comments

* Remove test - `test_decrement_spam`

* todo comments

* Update TODO comments

* Extract `test_unconfirmed_are_ignored` as separate test case

* Remove dead code

* Fix `test_unconfirmed_are_ignored`

* Remove dead code in `filter_dispute_data`

* Fix weights (related to commit "Remove `SpamSlots`")

* Disputes migration - first try

* Remove `dispute_max_spam_slots` + storage migration

* Fix `HostConfig` migration tests

* Deprecate `SpamSlots`

* Code review feedback

* add weight for storage version update
* fix bound for clear()

* Fix weights in disputes migration

* Revert "Deprecate `SpamSlots`"

This reverts commit 8c4d967c7b061abd76ba8b551223918c0b9e6370.

* Make mod migration public

* Remove `SpamSlots` from disputes pallet and use `storage_alias` in the migration

* Fix call to `clear()` for `SpamSlots` in migration

* Update migration and add a `try-runtime` test

* Add `pre_upgrade` `try-runtime` test

* Fix some test names in `HostConfiguration` migration

* Link spamslots migration in all runtimes

* Add `test_unconfirmed_disputes_cause_block_import_error`

* Update guide

- Remove `SpamSlots` related information from roadmap/implementers-guide/src/runtime/disputes.md
- Add 'Disputes filtering' to Runtime section of the Implementor's guide

* Update runtime/parachains/src/configuration/migration.rs

Co-authored-by: Marcin S. <[email protected]>

* Code review feedback - update logs

* Code review feedback: fix weights

* Update runtime/parachains/src/disputes.rs

Co-authored-by: s0me0ne-unkn0wn <[email protected]>

* Additional logs in disputes migration

* Fix merge conflicts

* Add version checks in try-runtime tests

* Fix a compilation warning`

Co-authored-by: Marcin S. <[email protected]>
Co-authored-by: s0me0ne-unkn0wn <[email protected]>

* Remove unused code (#6525)

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* Fix flaky test in `dispute-coordinator` (#6524)

https://github.com/paritytech/polkadot/pull/6494 updates disputes
participation priority on Active Leaves update. This operation might
trigger participation in some cases and as a result some of the message
ordering is not as nice as it used to be.

As a side effect of this `resume_dispute_without_local_statement` was
failing occasionally. The solution is not to expect that `BlockNumber`,
`CandidateEvents`, `FetchOnChainVotes` and `Ancestors` messages are
executed after `FinalizedBlockNumber` and in any specific order.

This should be okay as the code is in helper function and doesn't affect
the actual test behaviour.

Fixes https://github.com/paritytech/polkadot/issues/6514

* Bump tokio from 1.22.0 to 1.24.1 (#6523)

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.22.0 to 1.24.1.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.22.0...tokio-1.24.1)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Guide changes 3 (#6520)

* Guide changes

* Adding clarification

* Replace async-std with tokio in PVF subsystem (#6419)

* Replace async-std with tokio in PVF subsystem

* Rework workers to use `select!` instead of a mutex

The improvement in code readability is more important than the thread overhead.

* Remove unnecessary `fuse`

* Add explanation for `expect()`

* Update node/core/pvf/src/worker_common.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Update node/core/pvf/src/worker_common.rs

Co-authored-by: Bastian Köcher <[email protected]>

* Address some review comments

* Shutdown tokio runtime

* Run cargo fmt

* Add a small note about retries

* Fix up merge

* Rework `cpu_time_monitor_loop` to return when other thread finishes

* Add error string to PrepareError::IoErr variant

* Log when artifacts fail to prepare

* Fix `cpu_time_monitor_loop`; fix test

* Fix text

* Fix a couple of potential minor data races.

First data race was due to logging in the CPU monitor thread even if the
job (other thread) finished. It can technically finish before or after the log.

Maybe best would be to move this log to the `select!`s, where we are guaranteed
to have chosen the timed-out branch, although there would be a bit of
duplication.

Also, it was possible for this thread to complete before we executed
`finished_tx.send` in the other thread, which would trigger an error as the
receiver has already been dropped. And right now, such a spurious error from
`send` would be returned even if the job otherwise succeeded.

* Update Cargo.lock

Co-authored-by: Bastian Köcher <[email protected]>

* Fix cycle dispute-coordinator <-> dispute-distribution  (#6489)

* First iteration of message sender.

* dyn Fn variant (no cloning)

* Full implementation + Clone, without allocs on `Send`

* Further clarifications/cleanup.

* MessageSender -> NestingSender

* Doc update/clarification.

* dispute-coordinator: Send disputes on startup.

+ Some fixes, cleanup.

* Fix whitespace.

* Dispute distribution fixes, cleanup.

* Cargo.lock

* Fix spaces.

* More format fixes.

What is cargo fmt doing actually?

* More fmt fixes.

* Fix nesting sender.

* Fixes.

* Whitespace

* Enable logging.

* Guide update.

* Fmt fixes, typos.

* Remove unused function.

* Simplifications, doc fixes.

* Update roadmap/implementers-guide/src/node/disputes/dispute-coordinator.md

Co-authored-by: Marcin S. <[email protected]>

* Fmt + doc example fix.

Co-authored-by: eskimor <[email protected]>
Co-authored-by: Marcin S. <[email protected]>

* Update the GHA srtool-actions to the latest version (#6532)

* Minor fixes (#6533)

* Minor fixes

* Fix compile errors

* Trigger update-node-template when a final release tag is created (#6522)

* trigger update-node-template when final release tag is created

ref https://github.com/paritytech/release-engineering/issues/142

* fix comment

* Update scripts/ci/gitlab/pipeline/publish.yml

Co-authored-by: Alexander Samusev <[email protected]>

* fix regular expression

Co-authored-by: Alexander Samusev <[email protected]>

* ci: mark update-weights non-interruptible (#6539)

* Bump JasonEtco/create-an-issue from 2.6.0 to 2.9.1 (#6473)

Bumps [JasonEtco/create-an-issue](https://github.com/JasonEtco/create-an-issue) from 2.6.0 to 2.9.1.
- [Release notes](https://github.com/JasonEtco/create-an-issue/releases)
- [Commits](https://github.com/JasonEtco/create-an-issue/compare/9e6213aec58987fa7d2f4deb8b256b99e63107a2...e27dddc79c92bc6e4562f268fffa5ed752639abd)

---
updated-dependencies:
- dependency-name: JasonEtco/create-an-issue
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: parity-processbot <>

* Improve inactive fund tracking (#6471)

* Improve inactive fund tracking

* Docs

* Introduce the re-migration code

* Bump

* Update runtime/kusama/src/lib.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update runtime/common/src/crowdloan/migration.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update runtime/common/src/crowdloan/migration.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Grumbles

* Apply comments suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: muharem <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: parity-processbot <>

* Re-export current primitives in crate root (#6487)

* Re-export current primitives in crate root

* Add missing exports

* restart CI

* Add a paragraph about slashing in runtime disputes section from the guide (#6531)

* Add a paragraph about slashing in runtime disputes section from the guide

* Add definition for inconclusive

* Westend state trie to version 1 (#6336)

* Switch to state V1 and add state-trie-migration pallet with dummy manual
account.

* Initialize limit on runtime upgrade.

* add prelude

* sp_std prelude only for no_std

* Disable filter for signed migration

* revert hex dep

* use NeverEnsureOrigin

* fix

* correct fix

* check init state in try-runtime

Co-authored-by: parity-processbot <>

* handle substrate-node-template and substrate-parachain-template (#6541)

* Updated Dwellir bootnodes. Now using wss. (#6534)

* pvf: Fix missing execution request when retrying preparation (#6537)

* pvf: Add checks for result sender when retrying preparation in tests

* pvf: Fix missing execution request when retrying preparation

* Update comment

* pre-checking: Reject failed PVFs (#6492)

* pre-checking: Reject failed PVFs

* paras: immediately reject any PVF that cannot reach a supermajority

* Make the `quorum` reject condition a bit more clear semantically

* Add comment

* Update implementer's guide

* Update a link

Not related to the rest of the PR, but I randomly noticed and fixed this.

* Update runtime/parachains/src/paras/tests.rs

Co-authored-by: s0me0ne-unkn0wn <[email protected]>

* Remove unneeded loop

* Log PVF retries using `info!`

* Change retry logs to `warn!` and add preparation failure log

* Log PVF execution failure

* Clarify why we reject failed PVFs

* Fix PVF reject runtime benchmarks

Co-authored-by: s0me0ne-unkn0wn <[email protected]>

* Fix set-output deprecation (#6554)

set-output is being deprecated by GH.
More information at https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/

* add erasure-coding benches (#6308)

* add erasure-code benches

* revert Cargo.lock changes

* revert Cargo.lock changes

* CI: Code mark to request a pipeline failure (#6562)

* XCM v3 (#4097)

* cargo fmt

* Create benchmarks for XCM instructions introduced in v3 (#4564)

* Create benchmarks for BurnAsset and ExpectAsset

* Add benchmarks for ExpectOrigin and ExpectError

* Add benchmarks for QueryPallet and ExpectPallet

* Add benchmarks for ReportTransactStatus and ClearTransactStatus

* cargo fmt

* Use AllPalletsWithSystem in mocks

* Update XCM generic benchmarks for westend

* Remove default impls for some XCM weight functions

* Fix compilation error

* Add weight_args helper attribute

* Remove manually written XcmWeightInfo

* Parse trailing comma

* Revert "Add weight_args helper attribute"

This reverts commit 3b7c47a6182e1b9227036c38b406d494c3fcf6fd.

* Fixes

* Fixes

* XCM v3: Introduce querier field into `QueryReponse` (#4732)

* Introduce querier field into QueryReponse

* Convert &Option<MultiLocation> to Option<&MultiLocation>

&Option<T> is almost always never quite useful, most of the time it
still gets converted to an Option<&T> via `as_ref`, so we should simply
make functions that accept Option<&T> instead.

* Fix tests

* cargo fmt

* Fix benchmarks

* Appease spellchecker

* Fix test

* Fix tests

* Fix test

* Fix mock

* Fixes

* Fix tests

* Add test for response queriers

* Update xcm/pallet-xcm/src/lib.rs

* Test for non-existence of querier

Co-authored-by: Keith Yeung <[email protected]>

* Fixes

* Fixes

* Add `starts_with` function to `MultiLocation` and `Junctions` (#4835)

* add matches_prefix function to MultiLocation and Junctions

* rename matches_prefix to starts_with

* remove unnecessary main in doc comment

Co-authored-by: Bastian Köcher <[email protected]>

* Make use of starts_with in match_and_split

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* XCM v3: Bridge infrastructure (#4681)

* XCM bridge infrastructure

* Missing bit of cherry-pick

* Revamped XCM proc macros; new NetworkIds

* Fixes

* Formatting

* ExportMessage instruction and config type

* Add MessageExporter definitions

* Formatting

* Missing files

* Fixes

* Initial bridging config API

* Allow for two-stage XCM execution

* Update xcm/src/v3/mod.rs

Co-authored-by: Keith Yeung <[email protected]>

* XCM crate building again

* Initial bridging primitive

* Docs

* Docs

* More work

* More work

* Merge branch 'gav-xcm-v3' into gav-xcm-v3-bridging

* Make build

* WithComputedOrigin and SovereignPaidRemoteExporter

* Remove TODOs

* Slim bridge API and tests.

* Fixes

* More work

* First bridge test passing

* Formatting

* Another test

* Next round of bridging tests

* Repot tests

* Cleanups

* Paid bridging

* Formatting

* Tests

* Spelling

* Formatting

* Fees and refactoring

* Fixes

* Formatting

* Refactor SendXcm to become two-phase

* Fix tests

* Refactoring of SendXcm and ExportXcm complete

* Formatting

* Rename CannotReachDestination -> NotApplicable

* Remove XCM v0

* Minor grumbles

* Formatting

* Formatting

* Fixes

* Fixes

* Cleanup XCM config

* Fee handling

* Fixes

* Formatting

* Fixes

* Bump

Co-authored-by: Keith Yeung <[email protected]>

* Bump Substrate

* XCM v3: `ExchangeAsset` and Remote-locking (#4945)

* Asset Exchange and Locks

* Make sure XCM typers impl MaxEncodedLen

* Basic implementation for locks

* Bump Substrate

* Missing files

* Use new API

* Introduce  instruction

* Big refactor

* Docs

* Remove deprecated struct

* Remove deprecated struct

* Repot XCM builder tests

* ExchangeAsset test

* Exchange tests

* Locking tests

* Locking tests

* Fixes and tests

* Fixes

* Formatting

* Spelling

* Add simulator test for remote locking

* Fix tests

* Bump

* XCM v3: Support for non-fungibles (#4950)

* NFT support and a test

* New files.

* Integration tests for sending NFTs

* Formatting

* Broken Cargo features

* Use 2021 edition

* Fixes

* Formatting

* Formatting

* Update xcm/xcm-builder/src/asset_conversion.rs

Co-authored-by: Keith Yeung <[email protected]>

* Update xcm/xcm-builder/src/nonfungibles_adapter.rs

Co-authored-by: Keith Yeung <[email protected]>

* Update xcm/xcm-executor/src/lib.rs

Co-authored-by: Keith Yeung <[email protected]>

* Fixes

* Fixes

* Fixes

* Formatting

* Fixes

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Keith Yeung <[email protected]>

* XCM v3: Context & ID hash (#4756)

* send_xcm returns message hash

* cargo fmt

* Create topic register and instructions

* Fix weights

* Use tabs

* Sketch out XcmContext

* Fix doc test

* Add the XCM context as a parameter to executor trait fns

* Fixes

* Add XcmContext parameter

* Revert adding context as an arg to SendXcm trait methods

* Revert adding context argument to ConvertOrigin trait methods

* cargo fmt

* Do not change the API of XcmExecutor::execute

* Fixes

* Fixes

* Fixes

* Fixes

* Remove convenience method

* Fixes

* Fixes

* cargo fmt

* Fixes

* Add benchmarks for XCM topic instructions

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

* Remove context argument on FilterAssetLocation

* Fixes

* Remove unused import

* Fixes

* Fixes

* Fixes

* Accept XCM hash parameter in ExecuteXcm trait methods

* cargo fmt

* Properly enable sp-io/std

* Fixes

* default-features = false

* Fixes

* Fixes

* Fixes

* Make XcmContext optional in withdraw_asset

* Fixes

* Fixes

* Fixes

* Modify tests to check for the correct XCM hash

* Small refactor

* cargo fmt

* Check for expected hash in xcm-builder unit tests

* Add doc comment for the optionality of the XCM context in withdraw_asset

* Update xcm/src/v3/traits.rs

* Update xcm/src/v3/traits.rs

* Store XcmContext and avoid rebuilding

* Use ref for XcmContext

* Formatting

* Fix incorrect hash CC @KiChjang

* Refactor and make clear fake hashes

* Fixes

* Fixes

* Fixes

* Fix broken hashing

* Docs

* Fixes

* Fixes

* Fixes

* Formatting

* Fixes

* Fixes

* Fixes

* Remove unknowable hash

* Formatting

* Use message hash for greater identifiability

* Formatting

* Fixes

* Formatting

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Parity Bot <[email protected]>

* Fixes

* Fixes

* Fixes

* Fixes

* Formatting

* Fixes

* Formatting

* Fixes

* Fixes

* Formatting

* Formatting

* Remove horrible names

* Bump

* Remove InvertLocation trait (#5092)

* Remove InvertLocation trait

* Remove unneeded functions

* Formatting

* Fixes

* Remove XCMv1 (#5094)

* Remove XCMv1

* Remove XCMv1

* Formatting

* Fixes

* Fixes

* Formatting

* derive serialize/deserialize for xcm primitives (#5036)

* derive serialize/deserialize for xcm primitives

* derive serialize/deserialize for xcm primitives

* update v3

* update v2

Co-authored-by: Gav Wood <[email protected]>

* Update lock

* Fixes

* Add benchmarks for the ExchangeAsset instruction

* `AliasOrigin` instruction stub (#5122)

* AliasOrigin instruction stub

* Fixes

* Fixes

* Update substrate

* Fixes

* Ensure same array length before using copy_from_slice

* Fixes

* Add benchmarks for the UniversalOrigin instruction

* Remove unused import

* Remove unused import

* Add benchmarks for SetFeesMode instruction

* Add benchmarks for asset (un)locking instructions

* Leave AliasOrigin unbenchmarked

* Fixes after merge

* cargo fmt

* Fixes

* Fixes

* Set TrustedReserves to None on both Kusama and Westend

* Remove extraneous reserve_asset_deposited benchmark

* Fix universal_origin benchmark

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

* Don't rely on skipped benchmark functions

* Fixes

* cargo run --quiet --profile=production  --features=runtime-benchmarks -- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_generic.rs

* Fix unused variables

* Fixes

* Spelling

* Fixes

* Fix codec index of VersionedXcm

* Allows to customize how calls are dispatched from XCM (#5657)

* CallDispatcher trait

* fmt

* unused import

* fix test-runtime

* remove JustDispatch type

* fix typo in test-runtime

* missing CallDispatcher

* more missing CallDispatcher

* Update comment `NoteAssetLocked` -> `NoteUnlockable`

* Fixes

* Fixes

* Adjust MultiAssets weights based on new wild card variants

* Fixes

* Fixes

* Fixes

* Fixes

* Fixes

* Some late fixes for XCMv3 (#5237)

* Maximise chances that trapped assets can be reclaimed

* Do origin check as part of ExportMessage for security

* Formatting

* Fixes

* Cleanup export XCM APIs

* Formatting

* Update xcm/src/v3/junctions.rs

* UnpaidExecution instruction and associated barrier.

* Tighten barriers (ClearOrigin/QueryResponse)

* Allow only 1 ClearOrigin instruction in AllowTopLevelPaidExecutionFrom

* Bi-directional teleport accounting

* Revert other fix

* Build fixes]

* Tests build

* Benchmark fixes

Co-authored-by: Keith Yeung <[email protected]>

* Update Substrate

* Re-export `pub` stuff from universal_exports.rs + removed unecessary clone (#6145)

* Re-export `pub` stuff from universal_exports.rs

* Removed unnecessary clone

* Use 2D weights in XCM v3 (#6134)

* Depend upon sp-core instead of sp-runtime

* Make sp-io a dev-dependency

* Use 2D weights in XCM v3

* cargo fmt

* Add XCM pallet migration to runtimes

* Use from_parts

* cargo fmt

* Fixes

* cargo fmt

* Remove XCMWeight import

* Fixes

* Fixes

* Fixes

* Fixes

* Use translate in migration

* Increase max upward message size in tests

* Fix doc test

* Remove most uses of from_ref_time

* cargo fmt

* Fixes

* Fixes

* Add extrinsic benchmarking to XCM pallet

* cargo fmt

* Fixes

* Use old syntax

* cargo fmt

* Fixes

* Remove hardcoded weights

* Add XCM pallet to benchmarks

* Use successful origin

* Fix weird type parameter compilation issue

* Fixes

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm

* Use benchmarked XCM pallet weights

* Fixes

* Fixes

* Use override instead of skip

* Fixes

* Fixes

* Fixes

* Fixes

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm

* Fixes

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm

Co-authored-by: command-bot <>

* Replace Weight::MAX with 100b weight units

* Add test to ensure all_gte in barriers is correct

* Update xcm/src/v3/junction.rs

Co-authored-by: asynchronous rob <[email protected]>

* Add more weight tests

* cargo fmt

* Create thread_local in XCM executor to limit recursion depth (#6304)

* Create thread_local in XCM executor to limit recursion depth

* Add unit test for recursion limit

* Fix statefulness in tests

* Remove panic

* Use defer and environmental macro

* Fix the implementation

* Use nicer interface

* Change ThisNetwork to AnyNetwork

* Move recursion check up to top level

* cargo fmt

* Update comment

Co-authored-by: Bastian Köcher <[email protected]>

* Add upper limit on the number of overweight messages in the queue (#6298)

* Add upper limit on the number of ovwerweight messages in the queue

* Add newline

* Introduce whitelist for Transact and limit UMP processing to 10 messages per block (#6280)

* Add SafeCallFilter to XcmConfig

* Limit UMP to receive 10 messages every block

* Place 10 message limit on processing instead of receiving

* Always increment the message_processed count whenever a message is processed

* Add as_derivative to the Transact whitelist

* cargo fmt

* Fixes

* Update xcm/xcm-builder/src/universal_exports.rs

Co-authored-by: Branislav Kontur <[email protected]>

* Fixes

* Fixes

* Remove topic register and instead use the topic field in XcmContext

* Derive some common traits for DispatchBlobError

* Fixes

* cargo fmt

* Fixes

* Fixes

* Fix comments

* Fixes

* Introduce WithOriginFilter and apply it as the CallDispatcher for runtimes

* Fixes

* Appease clippy and fixes

* Fixes

* Fix more clippy issues

* Fixes

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm

* Add benchmark function for ExportMessage

* Fix comment

* Add upper limit to DownwardMessageQueues size

* Add max size check for queue in can_queue_downward_message

* Fixes

* Make Transact runtime call configurable

* Return Weight::MAX when there is no successful send XCM origin

* Update substrate

* Fixes

* Fixes

* Remove ExportMessage benchmark

* Remove assertion on Transact instruction benchmark

* Make reachable destination configurable in XCM pallet benchmarks

* Fixes

* Fixes

* Remove cfg attribute in fuzzer

* Fixes

* Remove cfg attribute for XCM pallet in test runtime

* Fixes

* Use ReachableDest where possible

* Fixes

* Add benchmark for UnpaidExecution

* Update substrate

* Ensure benchmark functions pass filters

* Add runtime-benchmarks feature to fuzzer

* Ensure FixedRateOfFungible accounts for proof size weights

* cargo fmt

* Whitelist remark_with_event when runtime-benchmarks feature is enabled

* Use remark_with_event for Transact benchmarks

* Fix Cargo.lock

* Allow up to 3 DescendOrigin instructions before UnpaidExecution

* cargo fmt

* Edit code comment

* Check check_origin for unpaid execution privilege

* Fixes

* Small nits for xcm-v3 (#6408)

* Add possibility to skip benchmark for export_message

* ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic

* Revert

* ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic

* Add HaulBlobError to `fn haul_blob`

* ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic

Co-authored-by: command-bot <>

* Revert changes to UnpaidExecution

* Change AllowUnpaidExecutionFrom to be explicit

* Fix log text

* cargo fmt

* Add benchmarks for XCM pallet version migration (#6448)

* Add benchmarks for XCM pallet version migration

* cargo fmt

* Fixes

* Fixes

* Fixes

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm

* Fix benchmarks

* Fix benchmarks

* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm

* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm

Co-authored-by: command-bot <>

* Merge remote-tracking branch 'origin/master' into gav-xcm-v3

* Fixes

* Fix comments (#6470)

* Specify Ethereum networks by their chain id (#6286)

Co-authored-by: Squirrel <[email protected]>

* Use  for Kusama

* Use WithComputedOrigin for Polkadot, Rococo and Westend

* Update lock

* Fix warning

* Update xcm/pallet-xcm/src/tests.rs

Co-authored-by: Squirrel <[email protected]>

* Update runtime/parachains/src/ump/migration.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Update xcm/pallet-xcm/src/migration.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Fixes

* cargo fmt

* Typo

* Update xcm/src/v3/mod.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Docs

* Docs

* Docs

* Docs

* Docs

* Update xcm/src/v3/multiasset.rs

Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Add tests for MultiAssets::from_sorted_and_deduplicated

* Fail gracefully when same instance NFTs are detected during push

* Update Substrate to fix benchmarks

* Apply suggestions from code review

* Update runtime/kusama/src/xcm_config.rs

* Rename arguments

* Attempt to fix benchmark

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev runtime_parachains::ump

* Use actual weights for UMP pallet in Polkadot

* ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::ump

* ".git/.scripts/commands/bench/bench.sh" runtime westend-dev runtime_parachains::ump

* ".git/.scripts/commands/bench/bench.sh" runtime rococo-dev runtime_parachains::ump

Co-authored-by: Keith Yeung <[email protected]>
Co-authored-by: Alexander Popiak <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Parity Bot <[email protected]>
Co-authored-by: stanly-johnson <[email protected]>
Co-authored-by: nanocryk <[email protected]>
Co-authored-by: Branislav Kontur <[email protected]>
Co-authored-by: asynchronous rob <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Vincent Geddes <[email protected]>
Co-authored-by: Squirrel <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Shawn Tabrizi <[email protected]>

* Add runtime-benchmarks feature to test client and test service (#6566)

* Exclude CI related dictionary files from the CI team approval (#6563)

* Enable treasury.spend by Root origin for Polkadot network before Gov2 (#6490)

* Enable treasury.spend by Root origin

* EnsureRoot -> EnsureRootWithSuccess

* Co #12928: NIS should retain funds in reserve (#6569)

* Add new weight functions

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Configure NIS pallet ReserveId

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Bump

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Gav <[email protected]>

* Allow AuctionAdmin to use Scheduler (#6573)

* Companion for substrate#13154 (#6570)

* Remove lifetime from `KeyIterator`

* Remove the lifetime from the method too

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Added `HaulBlobError` to pub re-exports (#6579)

* Issue 4804: Notify chain selection of concluded disputes directly (#6512)

* Setting up new ChainSelectionMessage

* Partial first pass

* Got dispute conclusion data to provisioner

* Finished first draft for 4804 code

* A bit of polish and code comments

* cargo fmt

* Implementers guide and code comments

* More formatting, and naming issues

* Wrote test for ChainSelection side of change

* Added dispute coordinator side test

* FMT

* Addressing Marcin's comments

* fmt

* Addressing further Marcin comment

* Removing unnecessary test line

* Rough draft addressing Robert changes

* Clean up and test modification

* Majorly refactored scraper change

* Minor fixes for ChainSelection

* Polish and fmt

* Condensing inclusions per candidate logic

* Addressing Tsveto's comments

* Addressing Robert's Comments

* Altered inclusions struct to use nested BTreeMaps

* Naming fix

* Fixing inclusions struct comments

* Update node/core/dispute-coordinator/src/scraping/mod.rs

Add comment to split_off() use

Co-authored-by: Marcin S. <[email protected]>

* Optimizing removal at block height for inclusions

* fmt

* Using copy trait

Co-authored-by: Marcin S. <[email protected]>

* XCM: `ExpectTransactStatus` instruction (#6578)

* Introduce ExpectTransactStatus instruction

* Remove other changes

* Implement

* Implement rest

* Benchmark

* Update xcm/src/v3/mod.rs

Co-authored-by: Keith Yeung <[email protected]>

* Update xcm/src/v3/mod.rs

Co-authored-by: Keith Yeung <[email protected]>

Co-authored-by: Keith Yeung <[email protected]>

* construct mmr leaf prior to session pallet hook (#6577)

* move pallet_mmr ahead of pallet_session

addresses #11797

* document construction of mmr leaf prior to session

* update weights (0.9.37) (#6546) (#6555)

* kusama: update weights

* polkadot: update weights

* westend: update weights

* rococo: update weights

* Companion for Substrate #13157 (#6582)

* Update getrandom and version_check

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Companion for substrate: Make DispatchError impl MEL (#6583)

* bump parity-scale-codec version

* update lockfile for {"substrate"}

Co-authored-by: parity-processbot <>

* Update changelog templates to use new labels (#6585)

* Update changelog template to use new T labels

* update readme

* changelog gen: remove T6

* sync versions with release branch (0.9.37) (#6556)

* sync version to current release (0.9.37)

* sync spec_version with current release (9370)

* sync transaction_version with current release (9370)

* Disallow decoding of phantom asset (#6597)

* Disallow decoding of phantom asset

* Remove pub

* remove references to rpm (#6595)

* Warn validators with slow hardware (#6269)

* Warn validators with slow hardware

* update

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix printing logic

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* fix

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* add deps

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* update lockfile for {"substrate"}

* fmt

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: parity-processbot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Add Polkadotters bootnodes for Westend, Kusama and Polkadot (#6476)

* Add Polkadotters bootnodes for Westend, Kusama and Polkadot

* Remove newlines in bootnodes config section

* Fix some unjustified disputes (#6103)

* Fix indentation + add warning

on participation errors.

* Don't vote invalid on internal errors.

* Don't dispute on code compression error.

* Remove CodeDecompressionError

* Candidate not invalid if PVF preparation fails.

Instead: Report error.

* Fix malus

* Add clarifying comment.

* cargo fmt

* Fix indentation.

* Bump lru from 0.8.1 to 0.9.0 (#6538)

Bumps [lru](https://github.com/jeromefroe/lru-rs) from 0.8.1 to 0.9.0.
- [Release notes](https://github.com/jeromefroe/lru-rs/releases)
- [Changelog](https://github.com/jeromefroe/lru-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jeromefroe/lru-rs/compare/0.8.1...0.9.0)

---
updated-dependencies:
- dependency-name: lru
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* [ci] Change check-labels GHA (#6594)

* [ci] Change check-labels GHA

* change file spec name

* change script when no label

* add tags

* XCM: Add HRMP to SafeCallFilter (#6606)

* XCM: Add HRMP to SafeCallFilter

* Whitelist several paras_registrar calls

* libc crate update (#6616)

cargo update -p libc

* Bump parity-db (#6617)

* New runners for weights and new weights (#6508)

* [DO NOT MERGE] test new vm runner for weights 2

* add image

* fix merge conflict

* apply weights

* new weights

Co-authored-by: parity-processbot <>

* nix upgrade to 0.26.1 (#6622)

* Add pov_size and code_size histograms (#6633)

Signed-off-by: Andrei Sandu <[email protected]>

Signed-off-by: Andrei Sandu <[email protected]>

* Enable `try-runtime` flag in CI (#6587)

* Env gate tests

The CI should check that it compiles, but not run it.

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Enable try-runtime feature in CI

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix test

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Fix XCM transact bench (#6635)

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* clear migrations included in 0.9.37 (#6630)

* polkadot: clear migrations included in 0.9.37

* rococo: clear migrations included in 0.9.37

* westend: clear migrations included in 0.9.37

* kusama: clear migrations included in 0.9.37

* remove unused import: xcm_config::CheckAccount

* Co reducing fast-unstake bench time and more (#6552)

* update stuff

* remove

* update

* update

* update weights

* fix tests

* update weights

* fix a few small things

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev pallet-fast-unstake

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev pallet-fast-unstake

* ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev pallet-fast-unstake

* reduce batch size

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev pallet-fast-unstake

* update

* fix

* fix

* ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev pallet-fast-unstake

* update lockfile for {"substrate"}

* fmt

* Env gate migration try_fast_unstake_all

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: command-bot <>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* 13110 Companion: Add WeightToFee and LengthToFee Runtime API (#6536)

* Add WeightToFee runtime API impls

* Forgot a couple

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* Use master Cargo.lock

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

* cargo update -p sp-io

Signed-off-by: Oliver Tale-Yazdi <[email protected]>

---------

Signed-off-by: Oliver Tale-Yazdi <[email protected]>
Co-authored-by: Oliver Tale-Yazdi <[email protected]>

* Storage monitor added to polkadot node (#6619)

* Storage monitor added to polkadot node

Related to: https://github.com/paritytech/substrate/issues/12399

* Cargo.lock updated

* Cargo.lock update

* Revert "Cargo.lock update"

This reverts commit 04dfe5fe65ea7d1476d4632ad647f2c1c669ab1a.

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <[email protected]>

* missing_docs fixes

* Cargo.lock updated

---------

Co-authored-by: parity-processbot <>
Co-authored-by: Bastian Köcher <[email protected]>

* Make zombienet tests native friendly (#6588)

* Make zombienet tests native friendly

* Update zombienet_tests/README.md

Co-authored-by: Bastian Köcher <[email protected]>

* Get rid of `{{ZOMBIENET_NATIVE_BINARY_PREFIX}}`

* Remove pointless changes

* add set_env helper and doc in readme

* changes from feedback

---------

Co-authored-by: Bastian Köcher <[email protected]>
Co-authored-by: Javier Viola <[email protected]>

* Companion: Remove `uncles` related code (#6615)

* Companion: Remove `uncles` related code

https://github.com/paritytech/substrate/pull/13216

* Fix compilation

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* Update benchmark's successful origin api (#6598)

* Update benchmark's successful origin api

* rustfmt

* unwrap for indirect origin dep

* replace unwrap by expect with a message

---------

Co-authored-by: parity-processbot <>

* companion for substrate #13237 (#6626)

* companion for https://github.com/paritytech/substrate/pull/13237

* test-runtime: add MaxSetIdSessionEntries

* update lockfile for {"substrate"}

---------

Co-authored-by: parity-processbot <>

* ignore fast-unstake remote test (#6641)

* Bump docker/build-push-action from 3 to 4 (#6651)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Some late short-term fixes for dispute slashing (#6249)

* disputes/slashing: slash only backers for ForInvalid

* add an assertion in mock impl

* fix tests

* do not slash backers on onconcluded disputes

* slash an intersection of backers and losers

* zombienet/disputes: check for offence only for invalid disputes

* add backing votes to disputes bench builder

* Update runtime/parachains/src/builder.rs

* Brad implementers guide revisions 2 (#6239)

* Add disputes subsystems fix

* Updated dispute approval vote import reasoning

* Improved wording of my changes

* Resolving issues brought up in comments

* Update disputes prioritisation in `dispute-coordinator` (#6130)

* Scraper processes CandidateBacked events

* Change definition of best-effort

* Fix `dispute-coordinator` tests

* Unit test for dispute filtering

* Clarification comment

* Add tests

* Fix logic

If a dispute is not backed, not included and not confirmed we
don't participate but we do import votes.

* Add metrics for refrained participations

* Revert "Add tests"

This reverts commit 7b8391a087922ced942cde9cd2b50ff3f633efc0.

* Revert "Unit test for dispute filtering"

This reverts commit 92ba5fe678214ab360306313a33c781338e600a0.

* fix dispute-coordinator tests

* Fix scraping

* new tests

* Small fixes in guide

* Apply suggestions from code review

Co-authored-by: Andrei Sandu <[email protected]>

* Fix some comments and remove a pointless test

* Code review feedback

* Clarification comment in tests

* Some tests

* Reference counted `CandidateHash` in scraper

* Proper handling for Backed and Included candidates in scraper

Backed candidates which are not included should be kept for a
predetermined window of finalized blocks. E.g. if a candidate is backed
but not included in block 2, and the window size is 2, the same
candidate should be cleaned after block 4 is finalized.

Add reference counting for candidates in scraper. A candidate can be
added on multiple block heights so we have to make sure we don't clean
it prematurely from the scraper.

Add tests.

* Update comments in tests

* Guide update

* Fix cleanup logic for `backed_candidates_by_block_number`

* Simplify cleanup

* Make spellcheck happy

* Update tests

* Extract candidate backing logic in separate struct

* Code review feedba…
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sandreim sandreim sandreim approved these changes

@mrcnski mrcnski mrcnski approved these changes

@tdimitrov tdimitrov tdimitrov approved these changes

@vstakhov vstakhov vstakhov approved these changes

Assignees

@bredamatt bredamatt

Labels
A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. D3-trivial 🧸 PR contains trivial changes in a runtime directory that do not require an audit.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@bredamatt @sandreim @vstakhov @tdimitrov @mrcnski