Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

Run all benchmarks "as tests" in short setup on every PR #2846

Merged
merged 31 commits into from
Jul 22, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
f52e70b
Run all benchmarks "as tests" in short setup on every PR
bkontur Jul 10, 2023
6d58d27
Fix
bkontur Jul 10, 2023
fc9ffa4
Update scripts/ci/gitlab/pipeline/short-benchmarks.yml
bkontur Jul 11, 2023
0c2e008
Updated TODO
bkontur Jul 11, 2023
17827d5
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
Jul 11, 2023
78002bd
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
Jul 12, 2023
644c3f2
Removed `--execution wasm`
bkontur Jul 12, 2023
c639bb8
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 12, 2023
1690316
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 12, 2023
dee4513
[asset-conv] Unused Balances type
bkontur Jul 13, 2023
9baf9f6
[asset-conv] native asset change
bkontur Jul 13, 2023
c40cba3
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 14, 2023
0c1aedb
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 14, 2023
e93fecc
Dedicated `AssetBalance` type for `pallet_assets` instances
bkontur Jul 14, 2023
f40746f
Improved local vs foreign asset handling + test for not allowing pool…
bkontur Jul 14, 2023
2145285
Removed `into_multiasset_id`
bkontur Jul 14, 2023
70daf3b
Fix
bkontur Jul 14, 2023
1f6c814
Refactor
bkontur Jul 14, 2023
fcece4c
Fixed create_pool for benchmark with LocalAndForeignAssets (finally f…
bkontur Jul 14, 2023
82ffbb2
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 14, 2023
d9636dc
Revert
bkontur Jul 15, 2023
6a19ff0
fmt
bkontur Jul 15, 2023
1d82b31
Migrates pools with `MultiLocation { parents: 0, interior: Here }` to…
bkontur Jul 17, 2023
fb101a3
Allow `set_storage` for `AllowMultiAssetPools` / `LiquidityWithdrawal…
bkontur Jul 17, 2023
58db05f
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 18, 2023
84547be
Benchmarks work
bkontur Jul 18, 2023
8a4c351
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 19, 2023
f05cd2e
Removed comment + more defensive migration
bkontur Jul 19, 2023
c79882b
`T::Currency::transfer` -> `Balances::transfer_all` in migration
bkontur Jul 19, 2023
e3c3b83
Change pool_id in migration
bkontur Jul 19, 2023
a5af430
Merge remote-tracking branch 'origin/master' into bko-short-benchmarks
bkontur Jul 22, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,14 @@
stages:
- test
- build
# used for manual job run for regenerate weights for release-* branches (not needed anymore, just leave it here for a while as PlanB)
- benchmarks-build
# used for manual job run for regenerate weights for release-* branches (not needed anymore, just leave it here for a while as PlanB)
- benchmarks-run
- publish
- integration-tests
- zombienet
- short-benchmarks

default:
interruptible: true
Expand Down Expand Up @@ -142,7 +145,10 @@ include:
- scripts/ci/gitlab/pipeline/test.yml
# # build jobs
- scripts/ci/gitlab/pipeline/build.yml
# short-benchmarks jobs
- scripts/ci/gitlab/pipeline/short-benchmarks.yml
# # benchmarks jobs
# # used for manual job run for regenerate weights for release-* branches (not needed anymore, just leave it here for a while as PlanB)
- scripts/ci/gitlab/pipeline/benchmarks.yml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this one is automatically creating a MR to update the weights? I dont know if that is used.

Copy link
Contributor

@alvicsam alvicsam Jul 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, but it works only with release-parachains-v* branches. Last time it was used month ago

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ggwpez yes, it is used for the releases, when the benchmark job is triggered from git lab. We used to do it manually so far

Copy link
Contributor Author

@bkontur bkontur Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EgorPopelyaev @ggwpez @joepetrowski @chevdor @alvicsam @bkchr
so for the very next release to regenerate weights, do we want to use:

  1. old way with manual gitlab job?
  2. new bot bench $ all ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bkontur lets's proceed with the new bot solution if it does a thing, but I would still keep the old pipeline yaml, as a plan B. At least for the next release

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EgorPopelyaev ok, good point

# # publish jobs
- scripts/ci/gitlab/pipeline/publish.yml
Expand Down
15 changes: 15 additions & 0 deletions scripts/ci/gitlab/pipeline/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -121,3 +121,18 @@ build-runtime-testing:
needs:
- job: build-runtime-starters
artifacts: false

build-short-benchmark:
stage: build
extends:
- .docker-env
- .common-refs
- .collect-artifacts
# this is an artificial job dependency, for pipeline optimization using GitLab's DAGs
needs:
- job: check-rustdoc
artifacts: false
script:
- cargo build --profile release --locked --features=runtime-benchmarks
- mkdir -p ./artifacts
- cp ./target/release/polkadot-parachain ./artifacts/
56 changes: 56 additions & 0 deletions scripts/ci/gitlab/pipeline/short-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# This file is part of .gitlab-ci.yml
# Here are all jobs that are executed during "short-benchmarks" stage

# Run all pallet benchmarks only once to check if there are any errors
.short-benchmark-template: &short-bench
stage: short-benchmarks
extends:
- .common-refs
- .docker-env
needs:
- job: build-short-benchmark
artifacts: true
variables:
RUNTIME_CHAIN: benchmarked-runtime-chain
script:
- ./artifacts/polkadot-parachain benchmark pallet --execution wasm --wasm-execution compiled --chain $RUNTIME_CHAIN --pallet "*" --extrinsic "*" --steps 2 --repeat 1

short-benchmark-asset-hub-polkadot:
<<: *short-bench
variables:
RUNTIME_CHAIN: asset-hub-polkadot-dev

short-benchmark-asset-hub-kusama:
<<: *short-bench
variables:
RUNTIME_CHAIN: asset-hub-kusama-dev

short-benchmark-asset-hub-westend:
<<: *short-bench
variables:
RUNTIME_CHAIN: asset-hub-westend-dev

short-benchmark-bridge-hub-polkadot:
<<: *short-bench
variables:
RUNTIME_CHAIN: bridge-hub-polkadot-dev

short-benchmark-bridge-hub-kusama:
<<: *short-bench
variables:
RUNTIME_CHAIN: bridge-hub-kusama-dev

short-benchmark-bridge-hub-rococo:
<<: *short-bench
variables:
RUNTIME_CHAIN: bridge-hub-rococo-dev

short-benchmark-collectives-polkadot :
<<: *short-bench
variables:
RUNTIME_CHAIN: collectives-polkadot-dev

short-benchmark-glutton-kusama :
<<: *short-bench
variables:
RUNTIME_CHAIN: glutton-kusama-dev-1300