Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V14 Decoding #45

Merged
merged 55 commits into from
Oct 19, 2021
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
a3e7d11
Begin scaffolding out crate for v14 decoding
jsdw Oct 5, 2021
e300f9a
Ongoing work to port metadata over
jsdw Oct 5, 2021
9ed8410
Comment out metadata stuff to start fresh
jsdw Oct 6, 2021
d18eead
test binary to play with metadata stuff
jsdw Oct 6, 2021
f113916
Start working out how metadata will be laid out
jsdw Oct 6, 2021
dfec7d1
Basic decoding of metadata to handle calls
jsdw Oct 7, 2021
5cfbc7d
Probably don't need extrinsic info in metadata, actually
jsdw Oct 7, 2021
6e6b90d
allow type resolution via metadata; we'll need that for decoding
jsdw Oct 7, 2021
7133c32
re-write SubstrateType to be derived from scale_info::Type
jsdw Oct 8, 2021
1ace2ec
Tidy up Metadata bits now that we have new SubstrateType
jsdw Oct 8, 2021
154947c
Rewrite SubstrateValue and get things compiling again
jsdw Oct 8, 2021
6241c72
Remove now-unused code
jsdw Oct 8, 2021
0f98fd4
Cargo fmt and a little tidy up
jsdw Oct 8, 2021
46400f2
Wee tidy up
jsdw Oct 8, 2021
a4cc70f
Update Cargo.lock
jsdw Oct 11, 2021
e38451e
Scaffold out some of the decode logic
jsdw Oct 11, 2021
07b7959
cargo fmt
jsdw Oct 11, 2021
aa097b5
Decode signature (untested) and scaffold out arg decoding methods
jsdw Oct 11, 2021
a9f1f20
Add a comment about mut ptr to slice
jsdw Oct 11, 2021
2c8dd13
Remove substrate_type; we _can_ construct scale_info Types ourselves.…
jsdw Oct 11, 2021
bca25c2
First pass over decoding logic done
jsdw Oct 12, 2021
caea294
Tidy up, and make v14_test bin do something a little more useful
jsdw Oct 12, 2021
1f41c16
Some tidy up and error on decoding if not all bytes consumed
jsdw Oct 12, 2021
51a3aa4
Comment lowercasing
jsdw Oct 12, 2021
4b80064
Add copyright notice on v14 code
jsdw Oct 12, 2021
0f6a566
Add missing copyright header
jsdw Oct 12, 2021
d1c8968
Add a couple of comments in the decode logic
jsdw Oct 12, 2021
1d1daeb
remove some commented-out code
jsdw Oct 12, 2021
a2cb527
Use signed extension metadata for dynamic signed extension decoding, …
jsdw Oct 13, 2021
14d882c
Unit test decoding, and some simplifications and removing a couple of…
jsdw Oct 13, 2021
76ab0a0
Remove lifetime from DecodeError and make sure LateEof is correct w.r…
jsdw Oct 13, 2021
e96deab
Cargo fmt
jsdw Oct 13, 2021
2098c40
differentiate between decoding extrinsics prefixed with length and no…
jsdw Oct 14, 2021
5a0abd0
Tweak a comment
jsdw Oct 14, 2021
8daeb91
Rename SubstrateValue to Value and expose from crate. Add first integ…
jsdw Oct 14, 2021
f989aa1
remove slightly-redundant 'Value' suffix from inner enum types
jsdw Oct 14, 2021
02f7318
Pallet/call indexes don't necessarily align with their positions; nee…
jsdw Oct 14, 2021
2b4e55d
Support compact-encoded wrapper-structs and add a couple more integra…
jsdw Oct 14, 2021
d487d27
More integration tests
jsdw Oct 14, 2021
6a72b06
cargo fmt
jsdw Oct 14, 2021
72d49e3
Add a couple more integration tests
jsdw Oct 14, 2021
9bd6564
A little renaming, and remove unwanted DS_Store
jsdw Oct 15, 2021
b2b87cf
Add docs and fix/test ExtrinsicBytes
jsdw Oct 15, 2021
cd4deb0
More docs
jsdw Oct 15, 2021
5831844
cargo clippy
jsdw Oct 15, 2021
3f10df4
More documenting
jsdw Oct 15, 2021
5f2f917
Remove Sequence value type: It's identical to Composite, and potentia…
jsdw Oct 15, 2021
79b282d
cargo fmt
jsdw Oct 15, 2021
deafb08
more doc tweaks
jsdw Oct 15, 2021
fe608de
Cargo clippy (tabs in doc comments keep coming back..)
jsdw Oct 15, 2021
859145c
add sqlx-data for offline mode
jsdw Oct 15, 2021
4111d6c
Update core_v14/src/decoder/extrinsic_bytes.rs
jsdw Oct 15, 2021
827a56d
newline in gitignore
jsdw Oct 15, 2021
4da194c
Doc comment fixes
jsdw Oct 18, 2021
b0374da
More doc comment fixes
jsdw Oct 18, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions core_v14/src/decoder/decode_type.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
use crate::{
substrate_type::SubstrateType,
substrate_value::SubstrateValue,
};

#[derive(Debug, Clone, thiserror::Error)]
pub enum DecodeTypeError {

}

pub fn decode_type<'a>(data: &'a [u8], ty: &SubstrateType) -> Result<(SubstrateValue, &'a [u8]), (DecodeTypeError, &'a [u8])> {
todo!()
}
111 changes: 106 additions & 5 deletions core_v14/src/decoder/decoder.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
use crate::metadata::Metadata;
use crate::metadata::{Metadata, MetadataError};
use crate::substrate_type::{CompositeType, SubstrateType};
use crate::substrate_value::SubstrateValue;
use super::extrinsic_bytes::{ ExtrinsicBytes, ExtrinsicBytesError };
use super::decode_type::{ decode_type, DecodeTypeError };

pub struct Decoder {
metadata: Metadata
Expand All @@ -9,7 +11,17 @@ pub struct Decoder {
#[derive(Clone,Debug,thiserror::Error)]
pub enum DecodeError {
#[error("Failed to parse the provided vector of extrinsics: {0}")]
UnexpectedExtrinsicsShape(#[from] ExtrinsicBytesError)
UnexpectedExtrinsicsShape(#[from] ExtrinsicBytesError),
#[error("Failed to decode type: {0}")]
DecodeTypeError(#[from] DecodeTypeError),
#[error("Failed to decode: expected more data")]
EarlyEof(&'static str),
#[error("Failed to decode unsupported extrinsic version '{0}'")]
CannotDecodeExtrinsicVersion(u8),
#[error("Cannot find call corresponding to extrinsic with pallet index {0} and call index {1}")]
CannotFindCall(u8, u8),
#[error("Failed to decode extrinsic: {0}")]
CannotFindType(#[from] MetadataError),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe s/Cannot/Unknown/?

}

impl Decoder {
Expand All @@ -21,7 +33,7 @@ impl Decoder {
}

/// Decode a SCALE encoded vector of extrinsics against the metadata provided
pub fn decode_extrinsics(&self, data: &[u8]) -> Result<Vec<SubstrateValue>, DecodeError> {
pub fn decode_extrinsics(&self, data: &[u8]) -> Result<Vec<GenericExtrinsic>, DecodeError> {
let extrinsic_bytes = ExtrinsicBytes::new(data)?;
log::trace!("Decoding {} Total Extrinsics.", extrinsic_bytes.len());

Expand All @@ -35,7 +47,96 @@ impl Decoder {
}

/// Decode a SCALE encoded extrinsic against the metadata provided
pub fn decode_extrinsic(&self, data: &[u8]) -> Result<SubstrateValue, DecodeError> {
todo!()
pub fn decode_extrinsic(&self, mut data: &[u8]) -> Result<GenericExtrinsic, DecodeError> {
if data.len() == 0 {
return Err(DecodeError::EarlyEof("extrinsic length should be > 0"));
}

let info = interpret_extrinsic_version(data[0]);

// We only know how to decode V4 extrinsics at the moment
if info.version != 4 {
return Err(DecodeError::CannotDecodeExtrinsicVersion(info.version));
}

// If the extrinsic is signed, decode the signature first. Remember that V4
// extrinsics are laid out roughly as follows:
//
// first byte: abbbbbbb (a = 0 for unsigned, 1 for signed, b = version)
// signature, which is made up of (in order):
// - sp_runtime::MultiAddress enum (sender)
// - sp_runtime::MultiSignature enum
// - sp_runtime::generic::Era enum
// - compact encoded u32 (nonce; prior transaction count)
// - compact encoded u128 (tip paid to block producer/treasury)
// call, which is made up roughly of:
// - enum pallet index (for pallets variant)
// - call index (for inner variant)
// - call args (types can be pulled from metadata for each arg we expect)
let mut signature = None;
if info.is_signed {

}

if data.len() < 2 {
return Err(DecodeError::EarlyEof("expected at least 2 more bytes for the pallet/call index"));
}

// Work out which call the extrinsic data represents and get type info for it:
let pallet_index = data[0];
let call_index = data[1];
data = &data[2..];
let (pallet_name, call) = match self.metadata.call_by_variant_index(pallet_index, call_index) {
Some(call) => call,
None => return Err(DecodeError::CannotFindCall(pallet_index, call_index))
};

// Decode each of the argument values in the extrinsic:
let mut arguments = vec![];
for arg in call.args() {
let ty = self.metadata.resolve_type(arg)?;
let val = match decode_type(data, &ty) {
Ok((val, rest)) => {
data = rest;
val
},
Err((err, _rest)) => {
return Err(err.into())
}
};
arguments.push(val);
}

// Return a composite type representing the extrinsic arguments:
Ok(GenericExtrinsic {
pallet: pallet_name.to_owned(),
call: call.name().to_owned(),
signature,
arguments,
})
}
}

pub struct GenericExtrinsic {
Copy link
Contributor

@insipx insipx Oct 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be good to derive Serialize, Deserialize for all the Generic decoded types

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrm, MultiAddress doesn't impl (de)serialize :( Two options:

  • impl Encode/Decode for everything and not impl (de)serialize,
  • or define my own MultiAddress type which matches the shape of the substrate one and does impl those traits.

What do you reckon?

/// The name of the pallet that the extrinsic called into
pub pallet: String,
/// The name of the call made
pub call: String,
/// The signature (if any) associated with the extrinsic
pub signature: Option<SubstrateValue>,
/// The arguments to pass to the call
pub arguments: Vec<SubstrateValue>
}

struct ExtrinsicVersionInfo {
/// Which version is this extrinsic?
version: u8,
/// Does this extrinsic have a signature?
is_signed: bool
}

fn interpret_extrinsic_version(byte: u8) -> ExtrinsicVersionInfo {
let is_signed = byte & 0b1000_0000 != 0;
let version = byte & 0b0111_1111;
ExtrinsicVersionInfo { version, is_signed }
}
1 change: 1 addition & 0 deletions core_v14/src/decoder/mod.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
mod decoder;
mod decode_type;
mod extrinsic_bytes;

pub use decoder::Decoder;
8 changes: 5 additions & 3 deletions core_v14/src/metadata/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -91,11 +91,13 @@ impl Metadata {

/// Given the `u8` variant index of a pallet and call, this returns information about
/// the call if it's fgound, or `None` if it no such call exists at those indexes.
pub fn call_by_variant_index(&self, pallet: u8, call: u8) -> Option<&MetadataCall> {
pub fn call_by_variant_index(&self, pallet: u8, call: u8) -> Option<(&str, &MetadataCall)> {
self.pallets
.get(pallet as usize)
.map(|p| p.calls.get(call as usize))
.flatten()
.and_then(|p| {
let call = p.calls.get(call as usize)?;
Some((&*p.name, call))
})
}

/// Return information about the metadata extrinsic format.
Expand Down