Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load latest metadata version from Wasm blobs. #1859

Merged
merged 3 commits into from
Nov 12, 2024

Conversation

pkhry
Copy link
Contributor

@pkhry pkhry commented Nov 8, 2024

Description

#1858 (comment)
The reason for this is that the WASM loader thing calls Metadata_metadata, which only returns V14 metadata. We should use Metadata_metadata_at_version and Metadata_versions to extract the appropriate metadata (V15 is possible, V14 if not)

closes #1858

@pkhry pkhry requested a review from a team as a code owner November 8, 2024 13:26
}

fn decode(encoded_metadata: Vec<u8>) -> WasmMetadataResult<Metadata> {
Metadata::decode(&mut encoded_metadata.as_ref()).map_err(Into::into)
// We slice the first byte from the metadata because it's wrapped inside an option and we know that its always `Some`
let metadata = <Vec<u8>>::decode(&mut &encoded_metadata[1..]).map_err(CodegenError::Decode)?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Would this panic on empty encoded_metadata?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, probably, will change

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the following code would work 😄

let metadata = <Option<Vec<u8>>>::decode(&mut &encoded_metadata[..]).map_err(CodegenError::Decode)?;

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refactored a bit to also fail on None

macro/src/wasm_loader.rs Outdated Show resolved Hide resolved
Copy link
Collaborator

@lexnv lexnv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job here!

Thinking outloud here, maybe a followup: it would be useful at some point to extend the functionality of this module:

  • have a WasmBlobExecutor wrapper that is handling just the executor building and runtime blob
  • then we could expose WasmBlobExecutor::versions which calls into Metadata_metadata_versions
  • similar for WasmBlobExecutor::get_metadata(version: ..)

This way we can choose in our macros which version to use (most prob the lastest, but sometimes we may want to fallback to lastest stable instead of unstable)

macro/src/wasm_loader.rs Outdated Show resolved Hide resolved
macro/src/wasm_loader.rs Outdated Show resolved Hide resolved
macro/src/wasm_loader.rs Outdated Show resolved Hide resolved
@pkhry pkhry requested a review from jsdw November 11, 2024 12:10
Ok(versions)
}

fn metadata(&mut self) -> WasmMetadataResult<Metadata> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'd call this something like load_legacy_metadata or something to be consistent with below (and make clear it's the "old" call :))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to load_legacy_metadata

decode(encoded_metadata)
}

fn load_metadata_at_latest_version(
Copy link
Collaborator

@jsdw jsdw Nov 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a nit and can be ignored, but personally I'd break the interface down a bit so that we maybe just expose basic wrappers around:

  • Metadata_metadata(),
  • Metadata_metadata_versions() and
  • Metadata_metadata_at_version(version)

These wrapperes just call the associated runtime API and give back the decoded result and nothing more.

In other words, load_metadata_at_latest_version here would just take in a single version to try and load, and the logic for handling versions is in the call_and_decode function rather than here. versions would equally just return the versions and not pre-filter them; we'd filter them also in call_and_decode or whatever.

This way, the "important" logic is all visible in one place and what these functions do is trivial.

macro/src/wasm_loader.rs Outdated Show resolved Hide resolved
@pkhry pkhry requested a review from jsdw November 12, 2024 10:34
Copy link
Collaborator

@jsdw jsdw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice one!

Not needed in this PR but I wonder how easily one could make a coupel of small-as-possible test runtimes exposing the new and old APIs to test that we get the expected V14 or V15 metadatas from them. May not be worth the effort though..

@pkhry pkhry merged commit 362aea3 into master Nov 12, 2024
13 checks passed
@pkhry pkhry deleted the pkhry/use_latest_metadata_in_wasm branch November 12, 2024 11:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

RUNTIME_APIS is empty when expanded with wasm path
5 participants