short-term fix for para inherent weight overestimation

[ordian]

closes #849

Context

For the background on this and the long-term fix, see #849 (comment).

Changes

• The weigh files are renamed from runtime_(parachains|common).* to polkadot_runtime_(parachains|common).*. The reason for it is the renaming introduced in Unify dependency aliases #4633. The new weight command and files are generated now include polkadot_ prefix.
• The WeightInfo for paras_inherent now includes enter_empty which calculates the cost of processing an empty parachains inherent. This cost is subtracted dynamically when calculating other weights (so the other weights remain the same)

Benefits

See #849 (comment), but TL;DR is that we are not blocked on weights for scaling the number of validators and cores further.

Resolved questions:

• why new benchmarks for westend are doing fewer db IOPS?
  Is it due polkadot-sdk update (db IOPS diff)?
  or the bench setup is no longer valid?
  https://github.com/polkadot-fellows/runtimes/blob/7723274a2c5cbb10213379271094d5180716ca7d/relay/polkadot/src/weights/runtime_parachains_paras_inherent.rs#L131-L196
  Answer: see background section of inclusion: bench enact_candidate weight #5270

TODOs:

• Rerun benchmarks for Rococo and Westend
• PRDoc

[ordian]

bot bench polkadot-pallet --pallet=polkadot_runtime_parachains::paras_inherent --runtime=westend
bot bench polkadot-pallet --pallet=polkadot_runtime_parachains::paras_inherent --runtime=rococo
bot clean

[paritytech-cicd-pr]

The CI pipeline was cancelled due to failure one of the required jobs.
Job name: cargo-clippy
Logs: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/6778934

[kianenigma]

What is the long term fix?

[sandreim]

Are these really needed if we just called build() on a default BenchBuilder instance?

[ordian]

bot bench polkadot-pallet --pallet=polkadot_runtime_parachains::paras_inherent --runtime=westend
bot bench polkadot-pallet --pallet=polkadot_runtime_parachains::paras_inherent --runtime=rococo
bot clean

[alexggh]

Looks good to me, but there is something I don't clearly understand, as far as I can tell we are using fallback_max_validators() in a lot of places, don't we need to adjust those to get realistic results for 1000 validators ?

[tdimitrov]

I think for a short term fix this is good. @alexggh do have a point about the 1000 validators but it can be a follow up or part of the proper fix.

[ordian]

bot bench polkadot-pallet --pallet=polkadot_runtime_parachains::paras_inherent --runtime=westend
bot bench polkadot-pallet --pallet=polkadot_runtime_parachains::paras_inherent --runtime=rococo
bot clean

[command-bot]

@ordian Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=rococo --target_dir=polkadot --pallet=polkadot_runtime_parachains::paras_inherent has finished. Result:

HttpError: Not Found
HttpError: Not Found
    at /app/node_modules/@octokit/request/dist-node/index.js:86:21
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async sendRequestWithRetries (/app/node_modules/octokit-auth-probot/node_modules/@octokit/auth-app/dist-node/index.js:466:12)
    at async Job.doExecute (/app/node_modules/bottleneck/light.js:405:18)

[command-bot]

@ordian Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" --subcommand=pallet --runtime=westend --target_dir=polkadot --pallet=polkadot_runtime_parachains::paras_inherent has finished. Result: https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7109801 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/7109801/artifacts/download.

[kianenigma]

Empirically, how much is the weight consumed by paras inherent in Polkadot and Kusama RC now, vs. how much will it be when this is deployed? (and which one is closer to reality -- although this is hard to measure 🙈)

[ordian]

Empirically, how much is the weight consumed by paras inherent in Polkadot and Kusama RC now, vs. how much will it be when this is deployed? (and which one is closer to reality -- although this is hard to measure 🙈)

I've provided some back-of-the-envelope calculations in this comment: #849 (comment), but TL;DR is that the dominant cost are from enter_bitfields (which we multiply by the number of para validators, which is 500 on Kusama at the moment and will grow to 1k). This number was already reduced in polkadot-fellows/runtimes#433 by 55% and the reasoning is explained in background section of #5270. This PR will further reduce it cost by more than a half (need to rerun the benches on Kusama, but you can see the cost for westend for comparison). Please note, that the cost of enter_bitfields by itself will remain the same, but subtract dynamically the cost of enter_empty. These and other annoyances will be addressed with a long-term refactoring mentioned in the first comment.
However, #5270 will bring some (a lot) of the costs back by properly accounting for e.g. processed messages in enacted candidates.