-
Notifications
You must be signed in to change notification settings - Fork 700
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Warp Payload Types #2116
Add Warp Payload Types #2116
Changes from 8 commits
f13427e
35724b9
d74b0f4
760a4ea
a59e373
04e83c3
e4133af
3f9a7c3
fe767f0
9a08314
afb7261
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,44 @@ | ||
# Payload | ||
|
||
An Avalanche Unsigned Warp Message already includes a `networkID`, `sourceChainID`, and `payload` field. The `payload` field is parsed into one of the types included in this package to be further handled by the VM. | ||
|
||
## AddressedCall | ||
|
||
AddressedCall: | ||
``` | ||
+---------------------+--------+----------------------------------+ | ||
| codecID : uint16 | 2 bytes | | ||
+---------------------+--------+----------------------------------+ | ||
| typeID : uint32 | 4 bytes | | ||
+---------------------+--------+----------------------------------+ | ||
| sourceAddress : []byte | 4 + len(address) | | ||
+---------------------+--------+----------------------------------+ | ||
| payload : []byte | 4 + len(payload) | | ||
+---------------------+--------+----------------------------------+ | ||
| 14 + len(payload) + len(address) | | ||
+----------------------------------+ | ||
``` | ||
|
||
- `codecID` is the codec version used to serialize the payload and is hardcoded to `0x0000` | ||
- `typeID` is the payload type identifier and is `0x00000000` for `AddressedCall` | ||
- `sourceAddress` is the address that sent this message from the source chain | ||
- `payload` is an arbitrary byte array payload | ||
|
||
## BlockHash | ||
|
||
BlockHash: | ||
``` | ||
+-----------------+----------+-----------+ | ||
| codecID : uint16 | 2 bytes | | ||
+-----------------+----------+-----------+ | ||
| typeID : uint32 | 4 bytes | | ||
+-----------------+----------+-----------+ | ||
| blockHash : [32]byte | 32 bytes | | ||
+-----------------+----------+-----------+ | ||
| 38 bytes | | ||
+-----------+ | ||
``` | ||
|
||
- `codecID` is the codec version used to serialize the payload and is hardcoded to `0x0000` | ||
- `typeID` is the payload type identifier and is `0x00000001` for `BlockHash` | ||
- `blockHash` is a blockHash from the `sourceChainID`. A signed block hash payload indicates that the signer has accepted the block on the source chain. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
// Copyright (C) 2019-2023, Ava Labs, Inc. All rights reserved. | ||
// See the file LICENSE for licensing terms. | ||
|
||
package payload | ||
|
||
import "fmt" | ||
|
||
var _ byteSetter = (*AddressedCall)(nil) | ||
|
||
// AddressedCall defines the format for delivering a call across VMs including a | ||
// source address and a payload. | ||
// Implementation of destinationAddress can be implemented on top of this payload. | ||
type AddressedCall struct { | ||
SourceAddress []byte `serialize:"true"` | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I guess we want to make this generic for all possible chains? (EVM chains use 20 bytes addresses) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yup, the intent here is to generalize this so that other VMs can use the same format. This will require changing the unpacking in Subnet-EVM / Teleporter as well. |
||
Payload []byte `serialize:"true"` | ||
|
||
bytes []byte | ||
} | ||
|
||
// NewAddressedCall creates a new *AddressedCall and initializes it. | ||
func NewAddressedCall(sourceAddress []byte, payload []byte) (*AddressedCall, error) { | ||
ap := &AddressedCall{ | ||
SourceAddress: sourceAddress, | ||
Payload: payload, | ||
} | ||
return ap, ap.initialize() | ||
} | ||
|
||
// ParseAddressedCall converts a slice of bytes into an initialized | ||
// AddressedCall. | ||
func ParseAddressedCall(b []byte) (*AddressedCall, error) { | ||
var unmarshalledPayloadIntf any | ||
if _, err := c.Unmarshal(b, &unmarshalledPayloadIntf); err != nil { | ||
return nil, err | ||
} | ||
payload, ok := unmarshalledPayloadIntf.(*AddressedCall) | ||
if !ok { | ||
return nil, fmt.Errorf("%w: %T", errWrongType, unmarshalledPayloadIntf) | ||
} | ||
payload.bytes = b | ||
return payload, nil | ||
} | ||
|
||
// initialize recalculates the result of Bytes(). | ||
func (a *AddressedCall) initialize() error { | ||
payloadIntf := any(a) | ||
bytes, err := c.Marshal(codecVersion, &payloadIntf) | ||
if err != nil { | ||
return fmt.Errorf("couldn't marshal warp addressed payload: %w", err) | ||
} | ||
a.bytes = bytes | ||
return nil | ||
} | ||
|
||
func (a *AddressedCall) setBytes(bytes []byte) { | ||
a.bytes = bytes | ||
} | ||
|
||
// Bytes returns the binary representation of this payload. It assumes that the | ||
// payload is initialized from either NewAddressedCall or ParseAddressedCall. | ||
func (a *AddressedCall) Bytes() []byte { | ||
return a.bytes | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
// Copyright (C) 2019-2023, Ava Labs, Inc. All rights reserved. | ||
// See the file LICENSE for licensing terms. | ||
|
||
package payload | ||
|
||
import ( | ||
"fmt" | ||
|
||
"github.com/ava-labs/avalanchego/ids" | ||
) | ||
|
||
var _ byteSetter = (*BlockHash)(nil) | ||
|
||
// BlockHash includes the block hash | ||
type BlockHash struct { | ||
BlockHash ids.ID `serialize:"true"` | ||
|
||
bytes []byte | ||
} | ||
|
||
// NewBlockHash creates a new *BlockHash and initializes it. | ||
func NewBlockHash(blockHash ids.ID) (*BlockHash, error) { | ||
bhp := &BlockHash{ | ||
BlockHash: blockHash, | ||
} | ||
return bhp, bhp.initialize() | ||
} | ||
|
||
// ParseBlockHash converts a slice of bytes into an initialized | ||
// BlockHash | ||
func ParseBlockHash(b []byte) (*BlockHash, error) { | ||
var unmarshalledPayloadIntf any | ||
if _, err := c.Unmarshal(b, &unmarshalledPayloadIntf); err != nil { | ||
return nil, err | ||
} | ||
payload, ok := unmarshalledPayloadIntf.(*BlockHash) | ||
if !ok { | ||
return nil, fmt.Errorf("%w: %T", errWrongType, unmarshalledPayloadIntf) | ||
} | ||
payload.bytes = b | ||
return payload, nil | ||
} | ||
|
||
// initialize recalculates the result of Bytes(). | ||
func (b *BlockHash) initialize() error { | ||
payloadIntf := any(b) | ||
bytes, err := c.Marshal(codecVersion, &payloadIntf) | ||
if err != nil { | ||
return fmt.Errorf("couldn't marshal block hash payload: %w", err) | ||
} | ||
b.bytes = bytes | ||
return nil | ||
} | ||
|
||
func (b *BlockHash) setBytes(bytes []byte) { | ||
b.bytes = bytes | ||
} | ||
|
||
// Bytes returns the binary representation of this payload. It assumes that the | ||
// payload is initialized from either NewBlockHash or ParseBlockHash. | ||
func (b *BlockHash) Bytes() []byte { | ||
return b.bytes | ||
} |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we have some way to parse either payload? It seems a bit odd to me that we expect the user to know which payload format to expect There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is currently used in Subnet-EVM to serve two execution functions in the Warp precompile. In that case, the function returns an error if the warp message includes anything other than the expected payload and does not differentiate between an invalid signature and requesting the wrong type, so it was not needed. Will add as a convenience function and so we can add a check when verifying warp messages that it matches a valid payload type rather than just checking the signature's validity. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
// Copyright (C) 2019-2023, Ava Labs, Inc. All rights reserved. | ||
// See the file LICENSE for licensing terms. | ||
|
||
package payload | ||
|
||
import ( | ||
"errors" | ||
"fmt" | ||
|
||
"github.com/ava-labs/avalanchego/codec" | ||
"github.com/ava-labs/avalanchego/codec/linearcodec" | ||
"github.com/ava-labs/avalanchego/utils/units" | ||
"github.com/ava-labs/avalanchego/utils/wrappers" | ||
) | ||
|
||
var errWrongType = errors.New("wrong payload type") | ||
|
||
const ( | ||
codecVersion = 0 | ||
|
||
MaxMessageSize = 24 * units.KiB | ||
|
||
// Note: Modifying this variable can have subtle implications on memory | ||
// usage when parsing malformed payloads. | ||
MaxSliceLen = 24 * units.KiB | ||
) | ||
|
||
// Codec does serialization and deserialization for Warp messages. | ||
var c codec.Manager | ||
|
||
func init() { | ||
c = codec.NewManager(MaxMessageSize) | ||
lc := linearcodec.NewCustomMaxLength(MaxSliceLen) | ||
|
||
errs := wrappers.Errs{} | ||
errs.Add( | ||
lc.RegisterType(&AddressedCall{}), | ||
lc.RegisterType(&BlockHash{}), | ||
c.RegisterCodec(codecVersion, lc), | ||
) | ||
if errs.Errored() { | ||
panic(errs.Err) | ||
} | ||
} | ||
|
||
// byteSetter provides an interface to set the bytes of an underlying type to [b] | ||
// after unmarshalling into that type. | ||
type byteSetter interface { | ||
setBytes(b []byte) | ||
} | ||
|
||
func Parse(bytes []byte) (byteSetter, error) { | ||
var intf interface{} | ||
if _, err := c.Unmarshal(bytes, &intf); err != nil { | ||
return nil, err | ||
} | ||
|
||
payload, ok := intf.(byteSetter) | ||
if !ok { | ||
return nil, fmt.Errorf("%w: %T", errWrongType, intf) | ||
} | ||
payload.setBytes(bytes) | ||
return payload, nil | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,143 @@ | ||
// Copyright (C) 2019-2023, Ava Labs, Inc. All rights reserved. | ||
// See the file LICENSE for licensing terms. | ||
|
||
package payload | ||
|
||
import ( | ||
"encoding/base64" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/require" | ||
|
||
"github.com/ava-labs/avalanchego/codec" | ||
"github.com/ava-labs/avalanchego/ids" | ||
"github.com/ava-labs/avalanchego/utils" | ||
) | ||
|
||
func TestAddressedCall(t *testing.T) { | ||
require := require.New(t) | ||
shortID := ids.GenerateTestShortID() | ||
|
||
addressedPayload, err := NewAddressedCall( | ||
shortID[:], | ||
[]byte{1, 2, 3}, | ||
) | ||
require.NoError(err) | ||
|
||
addressedPayloadBytes := addressedPayload.Bytes() | ||
addressedPayload2, err := ParseAddressedCall(addressedPayloadBytes) | ||
require.NoError(err) | ||
require.Equal(addressedPayload, addressedPayload2) | ||
} | ||
|
||
func TestParseAddressedCallJunk(t *testing.T) { | ||
require := require.New(t) | ||
_, err := ParseAddressedCall(utils.RandomBytes(1024)) | ||
require.ErrorIs(err, codec.ErrUnknownVersion) | ||
} | ||
|
||
func TestParseAddressedCall(t *testing.T) { | ||
require := require.New(t) | ||
base64Payload := "AAAAAAAAAAAAEAECAwAAAAAAAAAAAAAAAAAAAAADCgsM" | ||
payload := &AddressedCall{ | ||
SourceAddress: []byte{1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, | ||
Payload: []byte{10, 11, 12}, | ||
} | ||
|
||
require.NoError(payload.initialize()) | ||
|
||
require.Equal(base64Payload, base64.StdEncoding.EncodeToString(payload.Bytes())) | ||
|
||
parsedPayload, err := ParseAddressedCall(payload.Bytes()) | ||
require.NoError(err) | ||
require.Equal(payload, parsedPayload) | ||
} | ||
|
||
func TestBlockHash(t *testing.T) { | ||
require := require.New(t) | ||
|
||
blockHashPayload, err := NewBlockHash(ids.GenerateTestID()) | ||
require.NoError(err) | ||
|
||
blockHashPayloadBytes := blockHashPayload.Bytes() | ||
blockHashPayload2, err := ParseBlockHash(blockHashPayloadBytes) | ||
require.NoError(err) | ||
require.Equal(blockHashPayload, blockHashPayload2) | ||
} | ||
|
||
func TestParseBlockHashJunk(t *testing.T) { | ||
require := require.New(t) | ||
_, err := ParseBlockHash(utils.RandomBytes(1024)) | ||
require.ErrorIs(err, codec.ErrUnknownVersion) | ||
} | ||
|
||
func TestParseBlockHash(t *testing.T) { | ||
require := require.New(t) | ||
base64Payload := "AAAAAAABBAUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=" | ||
payload := &BlockHash{ | ||
BlockHash: ids.ID{4, 5, 6}, | ||
} | ||
|
||
require.NoError(payload.initialize()) | ||
|
||
require.Equal(base64Payload, base64.StdEncoding.EncodeToString(payload.Bytes())) | ||
|
||
parsedPayload, err := ParseBlockHash(payload.Bytes()) | ||
require.NoError(err) | ||
require.Equal(payload, parsedPayload) | ||
} | ||
|
||
func TestParseWrongPayloadType(t *testing.T) { | ||
require := require.New(t) | ||
blockHashPayload, err := NewBlockHash(ids.GenerateTestID()) | ||
require.NoError(err) | ||
|
||
shortID := ids.GenerateTestShortID() | ||
addressedPayload, err := NewAddressedCall( | ||
shortID[:], | ||
[]byte{1, 2, 3}, | ||
) | ||
require.NoError(err) | ||
|
||
_, err = ParseAddressedCall(blockHashPayload.Bytes()) | ||
require.ErrorIs(err, errWrongType) | ||
|
||
_, err = ParseBlockHash(addressedPayload.Bytes()) | ||
require.ErrorIs(err, errWrongType) | ||
} | ||
|
||
func TestParseJunk(t *testing.T) { | ||
require := require.New(t) | ||
_, err := Parse(utils.RandomBytes(1024)) | ||
require.ErrorIs(err, codec.ErrUnknownVersion) | ||
} | ||
|
||
func TestParsePayload(t *testing.T) { | ||
require := require.New(t) | ||
base64BlockHashPayload := "AAAAAAABBAUGAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=" | ||
blockHashPayload := &BlockHash{ | ||
BlockHash: ids.ID{4, 5, 6}, | ||
} | ||
|
||
require.NoError(blockHashPayload.initialize()) | ||
|
||
require.Equal(base64BlockHashPayload, base64.StdEncoding.EncodeToString(blockHashPayload.Bytes())) | ||
|
||
parsedBlockHashPayload, err := Parse(blockHashPayload.Bytes()) | ||
require.NoError(err) | ||
require.Equal(blockHashPayload, parsedBlockHashPayload) | ||
|
||
base64AddressedPayload := "AAAAAAAAAAAAEAECAwAAAAAAAAAAAAAAAAAAAAADCgsM" | ||
addressedPayload := &AddressedCall{ | ||
SourceAddress: []byte{1, 2, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, | ||
Payload: []byte{10, 11, 12}, | ||
} | ||
|
||
require.NoError(addressedPayload.initialize()) | ||
|
||
require.Equal(base64AddressedPayload, base64.StdEncoding.EncodeToString(addressedPayload.Bytes())) | ||
|
||
parsedAddressedPayload, err := Parse(addressedPayload.Bytes()) | ||
require.NoError(err) | ||
require.Equal(addressedPayload, parsedAddressedPayload) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we call this
ProofHash
orProofRoot
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we discussed, this may not be strictly a block hash
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is currently used as a
BlockHash
and making it more generic means that the caller would need to know what kind of hash it's referring to (some context is already needed ofc), so I opted to leave this asBlockHash
for now.If we want to change the name in the future, it is backwards compatible to do so, so I'd prefer to leave this until we actually use it as something other than a
BlockHash
.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As you alluded to, blocks aren't "universal" so the same problem applies to them.
I think locking this terminology in when we communicate it broadly will make us appear a little short-sighted, whereas using the
Proof
/ProofRoot
moniker is forward-looking.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tbh I'd expect using a
BlockHash
and using that to prove out the rest of the header contents to be simpler.Do you see this as a more targeted? If a VM would support multiple different types of proofs, how do you foresee them providing context as to what it's proving? Do you foresee them being registered under different typeIDs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess I was really excited by the generality/simplicity of using an arbitrary proof structure.
For example, it may be way easier to write a solidity contract to verify a trie of warp message IDs rather than a HyperSDK block. That approach also provides a generic interface that any VM can implement and use previously tested/implemented contracts (which I think will be a huge unlock).
We can enshrine "type 0" of the type to be "block hash" if you want to call that out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In terms of using "hash" in the name, I'm not sure what it should be called. I'm not sure all "proof roots" are "hashes" (maybe they are)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I guess the context will be up to the Solidity contracts to know what they are talking to based on the chainID, so that they know how it should be interpreted.
As for naming, I'm fine with
Hash
orProof