-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(da): RetrieveBatchesV2 method #937
Draft
keruch
wants to merge
45
commits into
kirill/interchain-da
Choose a base branch
from
kirill/interchain-da-retrieve-batch
base: kirill/interchain-da
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 2 commits
Commits
Show all changes
45 commits
Select commit
Hold shift + click to select a range
99b89e1
feat(da): RetrieveBatchesV2 method
keruch 9f6c2fb
feat(da): adjustments after manual testing
keruch 7cb2843
feat(da): added interchain-da proto contracts (#932)
keruch b09c75a
fix(rpc): panic and publish health event only on create batch error (…
zale144 2a5992c
feat(blockmanager): removed namespace from blockmanager (#943)
mtsitrin 6a33534
fix(blockManager): multiple accumulateddata trigger (#960)
mtsitrin 41d6053
fix(manager): Add start height to Submit batch to SL log message (#964)
zale144 9fa4109
fix(prune): fix guard for sequencer (#966)
danwt cce7cd3
fix(doc): remove misleading comment on last submitted height field (#…
danwt 9da82f1
fix(sync): adds missing error log (#965)
danwt a9cbe2d
chore(code standards): fix wrong godoc for struct field (#976)
yuhangcangqian 0d3be11
fix(p2p): set gossipsub buffersize to avoid missed blocks (#975)
srene 242acb7
fix(submission): fix counting and time (#969)
danwt e922dea
feat(da): add metric for consecutive failed da submissions (#986)
srene c5f8f07
feat(rpc): Add sync info metrics (#979)
zale144 d57d5b7
feat(da): add default retry value for celestia (#985)
srene 9e3201d
chore: add Initial changelog (#990)
hoangdv2429 402dd00
fix(rpc): Fix status `CatchingUp` field updating (#971)
zale144 83078f5
tests(pruning): Add pruning unit-test (#996)
mtsitrin 93905ee
hotfix(submit): early catch and log for empty batch (#997)
danwt 0bbe5be
fix(submit loop): add more logging around skew calculation (#1000)
danwt 4c9c2c3
feat(p2p): block sync protocol (#915)
srene 01fc2d5
fix(local pub sub): fix must subscribe to handle context cancelled (#…
danwt d3b4311
fix(test): submit loop test uses no op logger and prints more diagnos…
danwt 603c160
refactor(p2p): rename p2p block event (#1006)
srene f3e69a3
fix(submit loop): correctly load commit on startup (#1011)
danwt fab177a
fix(manager): full-node syncing fix (#1013)
srene d31bcf1
code-standards(submit loop): small refactor to submit loop to move ti…
danwt 7ee8303
fix(manager): unsubmitted bytes for batch calculation fix (#1019)
srene fd3cf3c
feat(da): submitted batch size metric (#1020)
srene 25c069c
fix(p2p): improve blocksync logs (#1030)
srene 88ba1fe
feat: sequencer rotation (#992)
mtsitrin 54e2fa5
feat: rollapp consensus params (#991)
srene 9aff932
feat: submit timestamps in block descriptors (#1032)
spoo-bar fe0f3cb
fix(manager): applylocalblock change mutex (#1036)
srene 6bf24d0
fix(rpc): fix websocket subscription panic when no closing error (#1046)
srene e9709f6
fix(manager): gossip any pending blocks not gossiped before (in case …
srene f745143
feat(da): upgrade for new celestia-openrpc version for celestia-node …
srene 562f673
fix(p2p): update height in blocksync loop (#1035)
srene d348ff0
fix(da): celestia openrpc version update (#1056)
srene 5de7192
fix(manager): use block params from consensus param (#1042)
srene 2da988f
refactor(manager): rename `rollappConsensusParams` to `rollappParams…
srene 6f910b0
temp
keruch d3d58ff
Merge branch 'main' into kirill/interchain-da-retrieve-batch
keruch de50e94
merge commit
keruch File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
package interchain | ||
|
||
import ( | ||
"errors" | ||
"fmt" | ||
|
||
"github.com/dymensionxyz/dymint/da/interchain/ioutils" | ||
"github.com/dymensionxyz/dymint/types" | ||
) | ||
|
||
// EncodeBatch encodes the batch to the interchain DA layer format. | ||
// The batch is represented in binary and gzipped. | ||
func EncodeBatch(b types.Batch) ([]byte, error) { | ||
// Prepare the blob data | ||
blob, err := b.MarshalBinary() | ||
if err != nil { | ||
return nil, fmt.Errorf("can't marshal batch: %w", err) | ||
} | ||
|
||
// Gzip the blob | ||
gzipped, err := ioutils.Gzip(blob) | ||
if err != nil { | ||
return nil, fmt.Errorf("can't gzip batch: %w", err) | ||
} | ||
|
||
return gzipped, nil | ||
} | ||
|
||
// DecodeBatch decodes the batch from the interchain DA layer format. | ||
// The incoming batch must be a gzipped binary. | ||
func DecodeBatch(b []byte) (types.Batch, error) { | ||
if !ioutils.IsGzip(b) { | ||
return types.Batch{}, errors.New("batch is not gzip-compressed") | ||
} | ||
|
||
// Gunzip the blob | ||
binary, err := ioutils.Gunzip(b) | ||
if err != nil { | ||
return types.Batch{}, fmt.Errorf("can't gunzip batch: %w", err) | ||
} | ||
|
||
// Prepare the blob data | ||
var batch types.Batch | ||
err = batch.UnmarshalBinary(binary) | ||
if err != nil { | ||
return types.Batch{}, fmt.Errorf("can't unmarshal batch: %w", err) | ||
} | ||
|
||
return batch, nil | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
package interchain_test | ||
|
||
import ( | ||
"reflect" | ||
"testing" | ||
|
||
"github.com/stretchr/testify/require" | ||
|
||
"github.com/dymensionxyz/dymint/da/interchain" | ||
"github.com/dymensionxyz/dymint/types" | ||
) | ||
|
||
func FuzzEncodeDecodeBatch1(f *testing.F) { | ||
f.Add(uint64(0), uint64(0)) | ||
|
||
f.Fuzz(func(t *testing.T, h1, h2 uint64) { | ||
// Generate batches with random headers | ||
expected := types.Batch{ | ||
StartHeight: h1, | ||
EndHeight: h2, | ||
Blocks: []*types.Block{}, | ||
Commits: []*types.Commit{}, | ||
} | ||
|
||
data, err := interchain.EncodeBatch(expected) | ||
require.NoError(t, err) | ||
|
||
actual, err := interchain.DecodeBatch(data) | ||
require.NoError(t, err) | ||
|
||
require.Equal(t, expected, actual) | ||
}) | ||
} | ||
Comment on lines
+17
to
+33
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what exactly do you expect to reveal by only fuzzing the heights? |
||
|
||
func TestEncodeDecodeBatch(t *testing.T) { | ||
expected := types.Batch{ | ||
StartHeight: 1, | ||
EndHeight: 123, | ||
Blocks: []*types.Block{ | ||
{ | ||
Header: types.Header{ | ||
Height: 1, | ||
}, | ||
}, | ||
{ | ||
Header: types.Header{ | ||
Height: 2, | ||
}, | ||
}, | ||
}, | ||
Commits: []*types.Commit{ | ||
{ | ||
Height: 1, | ||
}, | ||
{ | ||
Height: 2, | ||
}, | ||
}, | ||
} | ||
|
||
data, err := interchain.EncodeBatch(expected) | ||
require.NoError(t, err) | ||
|
||
actual, err := interchain.DecodeBatch(data) | ||
require.NoError(t, err) | ||
|
||
require.True(t, reflect.DeepEqual(expected, actual)) | ||
require.Equal(t, expected, actual) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,59 @@ | ||
package interchain | ||
|
||
import "github.com/dymensionxyz/dymint/da" | ||
import ( | ||
"fmt" | ||
|
||
"github.com/dymensionxyz/dymint/da" | ||
"github.com/dymensionxyz/dymint/types" | ||
interchainda "github.com/dymensionxyz/dymint/types/pb/interchain_da" | ||
) | ||
|
||
func (c *DALayerClient) RetrieveBatches(daMetaData *da.DASubmitMetaData) da.ResultRetrieveBatch { | ||
panic("RetrieveBatches method is not supported by the interchain DA clint") | ||
} | ||
|
||
func (c *DALayerClient) RetrieveBatchesV2(da.ResultSubmitBatchV2) da.ResultRetrieveBatchV2 { | ||
panic("implement me") | ||
func (c *DALayerClient) RetrieveBatchesV2(b da.ResultSubmitBatchV2) da.ResultRetrieveBatchV2 { | ||
batch, err := c.retrieveBatches(b) | ||
if err != nil { | ||
return da.ResultRetrieveBatchV2{ | ||
BaseResult: da.BaseResult{ | ||
Code: da.StatusError, | ||
Message: fmt.Sprintf("can't retrieve batch from the interchain DA layer: %s", err.Error()), | ||
Error: err, | ||
}, | ||
Batch: types.Batch{}, | ||
} | ||
} | ||
|
||
return da.ResultRetrieveBatchV2{ | ||
BaseResult: da.BaseResult{ | ||
Code: da.StatusSuccess, | ||
Message: "Retrieve successful", | ||
}, | ||
Batch: batch, | ||
} | ||
} | ||
|
||
func (c *DALayerClient) retrieveBatches(b da.ResultSubmitBatchV2) (types.Batch, error) { | ||
var commitment *interchainda.Commitment | ||
err := c.cdc.UnpackAny(b.DAPath.Commitment, &commitment) | ||
if err != nil { | ||
return types.Batch{}, fmt.Errorf("can't unpack commitment: %w", err) | ||
} | ||
|
||
resp, err := c.daClient.Blob(c.ctx, interchainda.BlobID(commitment.BlobId)) | ||
if err != nil { | ||
return types.Batch{}, fmt.Errorf("can't get blob from the interchain DA layer: %w", err) | ||
} | ||
|
||
if resp.BlobMetadata.BlobHash != commitment.BlobHash { | ||
return types.Batch{}, fmt.Errorf("commitment blob hash doesn't match interchain DA layer blob hash") | ||
} | ||
|
||
batch, err := DecodeBatch(resp.Blob) | ||
if err != nil { | ||
return types.Batch{}, fmt.Errorf("can't decode batch from interchain DA layer: %w", err) | ||
} | ||
|
||
return batch, nil | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dont need 'can't in the messages