Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(drive): platform version patching and state migrations #1941

Merged
merged 38 commits into from
Jul 16, 2024

Conversation

shumkov
Copy link
Member

@shumkov shumkov commented Jul 10, 2024

Issue being fixed or feature implemented

To be able to recover from chain halt and fix urgent bugs without waiting for protocol version upgrade (which happens only on epoch change), we need to provide a mechanism to apply fixes and modify the corrupted state.

What was done?

  • Introduced PlatfromVersion patching logic based on protocol version and block height range. PlatfromVersion is a map from the protocol version to the function version. The patching mechanism allows to change of function versions at any specific height.
  • Introduced state migration logic that allows to call of migration functions at a specific height.

How Has This Been Tested?

Introduced example patches and migrations and running strategy tests with tracing enabled.

Breaking Changes

None

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

@shumkov shumkov added this to the v1.0.0 milestone Jul 10, 2024
@shumkov shumkov requested a review from QuantumExplorer as a code owner July 10, 2024 19:27
Copy link
Member

@QuantumExplorer QuantumExplorer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like I reviewed this too early, maybe it wasn't done, please keep PRs in draft form until ready.

Comment on lines 144 to 147
patch_platform_version(&block_info, platform_version, &mut block_platform_state)?;

// Perform state migration to fix bugs or support new features
self.migrate_state(&block_info, &mut block_platform_state)?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be one call only. We should call it, check_for_chain_halt_hot_fix.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it should do everything inside.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Migrations aren't only for chain halts. We might want to migrate data for features as well.

@@ -0,0 +1,27 @@
mod migration_42_example;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be inside the build...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok

Comment on lines 19 to 22
match block_info.height {
42 => self.migration_42_example(block_info, block_platform_state),
52 => self.migration_42_example(block_info, block_platform_state),
_ => {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did we leave in examples?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For people so they see how to follow. I will hide it for builds

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed all test data from the code. Now it's defined in tests

current_platform_version: &PlatformVersion,
block_platform_state: &mut PlatformState,
) -> Result<(), Error> {
// Check if a patch that matches protocol version and block height is already applied
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be able to apply more than one patch.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More than one patch for the same version? You can.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per block height

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -82,11 +85,31 @@ Your software version: {}, latest supported protocol version: {}."#,

next_platform_version
} else {
// Stay on the last committed platform version
// Stay on the last committed plat version
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

platform

Comment on lines 320 to 327
dbg!(
state
.last_committed_block_info()
.as_ref()
.unwrap()
.basic_info()
.height
);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dbg

@@ -42,6 +42,10 @@ pub struct PlatformStateV0 {
pub current_validator_set_quorum_hash: QuorumHash,
/// next quorum
pub next_validator_set_quorum_hash: Option<QuorumHash>,
/// This is a clone of current platform version based on
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not a clone

Comment on lines 38 to 58
static PATCHES: Lazy<HashMap<ProtocolVersion, HeightToPatchRanges>> = Lazy::new(|| {
HashMap::from_iter(vec![
#[cfg(feature = "test-patch-platform")]
{
(
1,
BTreeMap::from_iter(vec![
(5, patch_1_5_test as PatchFn),
(10, patch_1_10_test as PatchFn),
]),
)
},
#[cfg(feature = "test-patch-platform")]
{
(
TEST_PROTOCOL_VERSION_2,
BTreeMap::from_iter(vec![(30, patch_2_30_test as PatchFn)]),
)
},
])
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

patches should be in versioning crate.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved

Comment on lines -56 to -57
block_platform_state
.set_current_protocol_version_in_consensus(current_block_protocol_version);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to remain here.

Copy link
Member Author

@shumkov shumkov Jul 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed to move inside if-condition in run_block_proposal to make PlatformState consistent with patching

Comment on lines 95 to 97
// Set current protocol version to the block platform state
block_platform_state
.set_current_protocol_version_in_consensus(block_platform_version.protocol_version);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move above into condition

Comment on lines 99 to 110
// Patch platform version and run migrations
// It modifies the protocol version to function version mapping to apply hotfixes
// Also it performs migrations to fix corrupted state or prepare it for new features
let block_platform_version = if let Some(patched_platform_version) = self.patch_platform(
block_proposal.height,
&mut block_platform_state,
transaction,
)? {
patched_platform_version
} else {
block_platform_version
};
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would do an if needed { do things }

Copy link
Member Author

@shumkov shumkov Jul 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't do if, because we need to run some logic anyway (clean up patches). Also, we would need to acquire a read lock twice. I renamed the function and updated the comment.

@QuantumExplorer QuantumExplorer merged commit 257be50 into v1.0-dev Jul 16, 2024
75 checks passed
@QuantumExplorer QuantumExplorer deleted the feat/drive/version-patch-and-migrations branch July 16, 2024 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants