Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add migration to fix coretime state #458

Conversation

seadanda
Copy link
Contributor

@seadanda seadanda commented Sep 20, 2024

The LeaseOffset was not taken into consideration in the Polkadot migration that ran when ref 1143 enacted and upgraded the relay chain, so the end timeslices of the leases are short by about 64 days.

This PR adds a migration to fix the Coretime state, which currently includes end timeslices which are incorrect.

Full notes and plan here: https://hackmd.io/kurv6rTXTF6yvaUJ9uSNoQ for work to continue while I'm OOO

try-runtime tests are included in the PR and the HackMD linked above includes steps to manually test the upgrade with chopsticks.

Copy link
Contributor

@eskimor eskimor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments. In general this patch is relatively hard to reason about. Lot's of magic numbers and implicit state tracking - e.g. with core numbers.

assignment: CoreAssignment::Task(para_id),
}]));

Workplan::<Runtime>::insert((287565, core_id), workplan_entry);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed: This is indeed the region_begin.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: Check on what occasions this is sent to the relay chain.

}

// Incorrect potential renewals in state
const INCORRECT_RENEWAL_IDS: [(u16, u32); 6] =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed: all leases added below (compared to coretime chain state) are to be found here.

(2000, 313920),
(3338, 329040),
(2004, 329040),
(3344, 344160),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked (some samples): Values here are indeed offset amount of blocks larger than values in state.


// Incorrect potential renewals in state
const INCORRECT_RENEWAL_IDS: [(u16, u32); 6] =
[(6, 292605), (5, 292605), (13, 292605), (15, 292605), (47, 292605), (44, 292605)];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked with chain state: Matching. (This is the entire list of currently registered potential renewals, as expected.)

}

// Remove the existing 6 PotentialRenewals.
for &(core, when) in INCORRECT_RENEWAL_IDS.iter() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

// Sort the parachains who can renew. They are currently missing from the broker state
// entirely.
// TODO double check the core ids, these should be on top of the last available in the sale.
for (&(para_id, _), core_id) in POTENTIAL_RENEWALS.iter().zip(56u16..61u16) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked that the first core is indeed 56. There are 5 renewals in POTENTIAL_RENEWALS s.
image

@tdimitrov
Copy link
Contributor

I've pushed the changes I've suggested in order to ease the code review. Motivation for each change was added as a comment initially.

try-runtime tests are passing modulo the case for PotentialRenewals. I'm not sure how it should be handled.

const POTENTIAL_RENEWALS: [(u32, u32); 5] =
[(2048, 283680), (3375, 283680), (3358, 283680), (2053, 283680), (2056, 283680)];

const LEASES: [(u32, u32); 46] = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List fully verified. Complete and correct.

@eskimor
Copy link
Contributor

eskimor commented Sep 29, 2024

Reworked + fixed tests. The only thing that is really still missing now is weights. Can we just benchmark the function?

@eskimor
Copy link
Contributor

eskimor commented Sep 29, 2024

Complete Workplan after migration:

0, CoreAssignment::Task(1001), 287565
1, CoreAssignment::Task(1002), 287565
2, CoreAssignment::Task(1000), 287565
3, CoreAssignment::Task(1004), 287565
4, CoreAssignment::Task(1005), 287565
5, CoreAssignment::Task(2094), 287565
6, CoreAssignment::Task(2040), 287565
7, CoreAssignment::Task(2035), 287565
8, CoreAssignment::Task(3344), 287565
9, CoreAssignment::Task(3370), 287565
10, CoreAssignment::Task(3367), 287565
11, CoreAssignment::Task(2086), 287565
12, CoreAssignment::Task(2032), 287565
13, CoreAssignment::Task(3333), 287565
14, CoreAssignment::Task(2051), 287565
15, CoreAssignment::Task(2106), 287565
16, CoreAssignment::Task(3369), 287565
17, CoreAssignment::Task(2008), 287565
18, CoreAssignment::Task(2025), 287565
19, CoreAssignment::Task(2000), 287565
20, CoreAssignment::Task(2092), 287565
21, CoreAssignment::Task(2002), 287565
22, CoreAssignment::Task(3359), 287565
23, CoreAssignment::Task(2030), 287565
24, CoreAssignment::Task(3378), 287565
25, CoreAssignment::Task(2104), 287565
26, CoreAssignment::Task(2046), 287565
27, CoreAssignment::Task(3345), 287565
28, CoreAssignment::Task(3340), 287565
29, CoreAssignment::Task(3338), 287565
30, CoreAssignment::Task(2004), 287565
31, CoreAssignment::Task(3377), 287565
32, CoreAssignment::Task(3373), 287565
33, CoreAssignment::Task(2031), 287565
34, CoreAssignment::Task(3389), 287565
35, CoreAssignment::Task(3366), 287565
36, CoreAssignment::Task(2037), 287565
37, CoreAssignment::Task(2034), 287565
38, CoreAssignment::Task(2090), 287565
39, CoreAssignment::Task(3346), 287565
40, CoreAssignment::Task(2012), 287565
41, CoreAssignment::Task(3397), 287565
42, CoreAssignment::Task(2043), 287565
43, CoreAssignment::Task(2091), 287565
44, CoreAssignment::Task(2093), 287565
45, CoreAssignment::Task(2026), 287565
46, CoreAssignment::Task(3388), 287565
47, CoreAssignment::Task(2101), 287565
48, CoreAssignment::Task(3354), 287565
49, CoreAssignment::Task(2006), 287565
50, CoreAssignment::Task(2013), 287565
56, CoreAssignment::Task(2048), 287565
57, CoreAssignment::Task(3375), 287565
58, CoreAssignment::Task(3358), 287565
59, CoreAssignment::Task(2053), 287565
60, CoreAssignment::Task(2056), 287565
61, CoreAssignment::Pool, 287565

Output of:

./target/release/try-runtime create-snapshot  --uri wss://coretime-polkadot.dotters.network ./snapshot-coretime-current

RUST_BACKTRACE=1 RUST_LOG=runtime::bootstrapping::fix-migration=debug ./target/release/try-runtime  --runtime ../fellowship-runtimes/target/debug/wbuild/coretime-polkadot-runtime/coretime_polkadot_runtime.wasm on-runtime-upgrade snap -p ./snapshot-coretime-current

@joepetrowski joepetrowski marked this pull request as ready for review September 30, 2024 07:21
@eskimor
Copy link
Contributor

eskimor commented Sep 30, 2024

To determine the correct lease endings and the paras that have been dropped, I used the following adjusted migration from the relay chain:

https://github.com/polkadot-fellows/runtimes/compare/main...eskimor:fellowship-runtimes:rk-determine-missing-chains?expand=1

and ran it at the migrating block via try-runtime:

./target/release/try-runtime create-snapshot  --uri wss://rpc-polkadot.luckyfriday.io -a 0x07886d8260a5f0436eda29bf847af211eb53c975754002675ab0e78022441c62 ./snapshot-update
RUST_LOG=coretime-migration=trace ./target/release/try-runtime  --runtime ../fellowship-runtimes/target/debug/wbuild/polkadot-runtime/polkadot_runtime.wasm on-runtime-upgrade --disable-spec-version-check snap -p ./snapshot-update

The upgrading block hash I determined via subscan:
Filter list of events (System -> CodeUpdated):
https://polkadot.subscan.io/event?page=1&time_dimension=date&module=system&event_id=codeupdated
12 days ago is the one we are looking for:
https://polkadot.subscan.io/block/22572435

@seadanda seadanda changed the title Add migration to fix coretime state (WIP) Add migration to fix coretime state Oct 1, 2024
@seadanda seadanda mentioned this pull request Oct 1, 2024
CHANGELOG.md Outdated Show resolved Hide resolved
CHANGELOG.md Outdated Show resolved Hide resolved
@bkchr
Copy link
Contributor

bkchr commented Oct 1, 2024

/merge

@fellowship-merge-bot fellowship-merge-bot bot enabled auto-merge (squash) October 1, 2024 20:32
@fellowship-merge-bot
Copy link
Contributor

Enabled auto-merge in Pull Request

Available commands
  • /merge: Enables auto-merge for Pull Request
  • /merge cancel: Cancels auto-merge for Pull Request
  • /merge help: Shows this menu

For more information see the documentation

@fellowship-merge-bot fellowship-merge-bot bot merged commit afc36ca into polkadot-fellows:main Oct 1, 2024
48 checks passed
@seadanda seadanda deleted the donal-coretime-migration-fix branch October 1, 2024 22:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants