Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load latest metadata version from Wasm blobs. #1859

Merged
merged 3 commits into from
Nov 12, 2024
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
141 changes: 121 additions & 20 deletions macro/src/wasm_loader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

use std::{borrow::Cow, path::Path};

use codec::Decode;
use codec::{Decode, Encode};
use polkadot_sdk::{
sc_executor::{self, WasmExecutionMethod, WasmExecutor},
sc_executor_common::runtime_blob::RuntimeBlob,
Expand All @@ -14,6 +14,8 @@ use polkadot_sdk::{
};
use subxt_codegen::{CodegenError, Metadata};

static SUPPORTED_METADATA_VERSIONS: [u32; 2] = [14, 15];

/// Result type shorthand
pub type WasmMetadataResult<A> = Result<A, CodegenError>;

Expand All @@ -26,26 +28,13 @@ pub fn from_wasm_file(wasm_file_path: &Path) -> WasmMetadataResult<Metadata> {
}

fn call_and_decode(wasm_file: Vec<u8>) -> WasmMetadataResult<Metadata> {
let mut ext: sp_state_machine::BasicExternalities = Default::default();

let executor: WasmExecutor<sp_io::SubstrateHostFunctions> = WasmExecutor::builder()
.with_execution_method(WasmExecutionMethod::default())
.with_offchain_heap_alloc_strategy(sc_executor::HeapAllocStrategy::Dynamic {
maximum_pages: Some(64),
})
.with_max_runtime_instances(1)
.with_runtime_cache_size(1)
.build();

let runtime_blob =
RuntimeBlob::new(&wasm_file).map_err(|e| CodegenError::Wasm(e.to_string()))?;
let metadata_encoded = executor
.uncached_call(runtime_blob, &mut ext, true, "Metadata_metadata", &[])
.map_err(|_| CodegenError::Wasm("method \"Metadata_metadata\" doesnt exist".to_owned()))?;

let metadata = <Vec<u8>>::decode(&mut &metadata_encoded[..]).map_err(CodegenError::Decode)?;
let mut executor = Executor::new(&wasm_file)?;

decode(metadata)
if let Ok(versions) = executor.versions() {
executor.load_metadata_at_latest_version(versions)
} else {
executor.metadata()
}
}

fn decode(encoded_metadata: Vec<u8>) -> WasmMetadataResult<Metadata> {
Expand All @@ -57,3 +46,115 @@ fn maybe_decompress(file_contents: Vec<u8>) -> WasmMetadataResult<Vec<u8>> {
.map_err(|e| CodegenError::Wasm(e.to_string()))
.map(Cow::into_owned)
}

struct Executor {
runtime_blob: RuntimeBlob,
executor: WasmExecutor<sp_io::SubstrateHostFunctions>,
externalities: sp_state_machine::BasicExternalities,
}

impl Executor {
fn new(wasm_file: &[u8]) -> WasmMetadataResult<Self> {
let externalities: sp_state_machine::BasicExternalities = Default::default();

let executor: WasmExecutor<sp_io::SubstrateHostFunctions> = WasmExecutor::builder()
.with_execution_method(WasmExecutionMethod::default())
.with_offchain_heap_alloc_strategy(sc_executor::HeapAllocStrategy::Dynamic {
maximum_pages: Some(64),
})
.with_max_runtime_instances(1)
.with_runtime_cache_size(1)
.build();

let runtime_blob =
RuntimeBlob::new(wasm_file).map_err(|e| CodegenError::Wasm(e.to_string()))?;

Ok(Self {
runtime_blob,
executor,
externalities,
})
}

fn versions(&mut self) -> WasmMetadataResult<Vec<u32>> {
let version = self
.executor
.uncached_call(
self.runtime_blob.clone(),
&mut self.externalities,
true,
"Metadata_metadata_versions",
&[],
)
.map_err(|_| {
CodegenError::Wasm("method \"Metadata_metadata_versions\" doesnt exist".to_owned())
})?;
let versions = <Vec<u32>>::decode(&mut &version[..])
.map_err(CodegenError::Decode)
.map(|x| {
x.into_iter()
.filter(|version| SUPPORTED_METADATA_VERSIONS.contains(version))
.collect::<Vec<u32>>()
})?;

if versions.is_empty() {
return Err(CodegenError::Other(
"No supported metadata versions were returned".to_owned(),
));
}

Ok(versions)
}

fn metadata(&mut self) -> WasmMetadataResult<Metadata> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'd call this something like load_legacy_metadata or something to be consistent with below (and make clear it's the "old" call :))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to load_legacy_metadata

let encoded_metadata = self
.executor
.uncached_call(
self.runtime_blob.clone(),
&mut self.externalities,
false,
"Metadata_metadata",
&[],
)
.map_err(|_| {
CodegenError::Wasm("method \"Metadata_metadata\" doesnt exist".to_owned())
})?;
let encoded_metadata =
<Vec<u8>>::decode(&mut &encoded_metadata[..]).map_err(CodegenError::Decode)?;
decode(encoded_metadata)
}

fn load_metadata_at_latest_version(
Copy link
Collaborator

@jsdw jsdw Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nit and can be ignored, but personally I'd break the interface down a bit so that we maybe just expose basic wrappers around:

  • Metadata_metadata(),
  • Metadata_metadata_versions() and
  • Metadata_metadata_at_version(version)

These wrapperes just call the associated runtime API and give back the decoded result and nothing more.

In other words, load_metadata_at_latest_version here would just take in a single version to try and load, and the logic for handling versions is in the call_and_decode function rather than here. versions would equally just return the versions and not pre-filter them; we'd filter them also in call_and_decode or whatever.

This way, the "important" logic is all visible in one place and what these functions do is trivial.

&mut self,
versions: Vec<u32>,
) -> WasmMetadataResult<Metadata> {
let version = versions
.into_iter()
.max()
.expect("This is checked earlier and can't fail.");

let encoded_metadata = self
.executor
.uncached_call(
self.runtime_blob.clone(),
&mut self.externalities,
false,
"Metadata_metadata_at_version",
&version.encode(),
)
.map_err(|e| {
dbg!(e);
pkhry marked this conversation as resolved.
Show resolved Hide resolved
CodegenError::Wasm(
"method \"Metadata_metadata_at_version\" doesnt exist".to_owned(),
)
})?;
let Some(encoded_metadata) =
<Option<Vec<u8>>>::decode(&mut &encoded_metadata[..]).map_err(CodegenError::Decode)?
else {
return Err(CodegenError::Other(
format!("Received empty metadata at version: v{version}").to_owned(),
));
};
decode(encoded_metadata)
}
}
Loading