Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DBSync Module #118

Merged
merged 24 commits into from
Apr 24, 2023
Merged

Add DBSync Module #118

merged 24 commits into from
Apr 24, 2023

Conversation

codchen
Copy link
Collaborator

@codchen codchen commented Apr 12, 2023

Describe your changes and provide context

Added a new internal module called dbsync that enables syncing application DB files directly. Its discovery mechanism resembles that of statesync with some simplification. Still a WIP

Testing performed to validate your change

unit tests
tested in loadtest cluster

@codchen codchen force-pushed the tony-chen-formalize-new-statesync branch from 5e2b9dc to a53e57c Compare April 21, 2023 02:16
Copy link
Contributor

@yzang2019 yzang2019 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great! No blocking comments, left some minor things though

Height: r.config.TrustHeight,
Hash: r.config.TrustHashBytes(),
}
if err := r.waitForEnoughPeers(ctx, 2); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be helpful to add some info logs before and after this

@@ -1257,6 +1260,43 @@ func (cfg *InstrumentationConfig) ValidateBasic() error {
return nil
}

type DBSyncConfig struct {
Enable bool `mapstructure:"db-sync-enable"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the changing the config name

@@ -31,6 +32,7 @@ type Application interface {
OfferSnapshot(context.Context, *RequestOfferSnapshot) (*ResponseOfferSnapshot, error) // Offer a snapshot to the application
LoadSnapshotChunk(context.Context, *RequestLoadSnapshotChunk) (*ResponseLoadSnapshotChunk, error) // Load a snapshot chunk
ApplySnapshotChunk(context.Context, *RequestApplySnapshotChunk) (*ResponseApplySnapshotChunk, error) // Apply a shapshot chunk
LoadLatest(context.Context, *RequestLoadLatest) (*ResponseLoadLatest, error)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a small comment for what this interface do? Like the above ones?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

config/config.go Outdated
SnapshotDirectory: "",
TimeoutInSeconds: 0,
NoFileSleepInSeconds: 5,
FileWorkerCount: 0,
Copy link
Contributor

@yzang2019 yzang2019 Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have better and more common defaults here? I'd suggest let's provide some ready to be used default value for:

  • SnapshotInterval
  • TimeoutInSeconds
  • NoFileSleepInSeconds
  • FileWorkerCount
  • FileWorkerTimeout
  • TrustPeriod

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

syncer := NewSyncer(logger, config, baseConfig, reactor.requestMetadata, reactor.requestFile, reactor.commitState, reactor.postSync)
reactor.syncer = syncer

reactor.BaseService = *service.NewBaseService(logger, "BlockSync", reactor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be DbSync?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch

@codchen codchen force-pushed the tony-chen-formalize-new-statesync branch from 6b5c89a to 37f0c27 Compare April 24, 2023 02:25
@codchen codchen force-pushed the tony-chen-formalize-new-statesync branch from b1dba5b to b03c748 Compare April 24, 2023 08:16
@codchen codchen force-pushed the tony-chen-formalize-new-statesync branch from b03c748 to 78ec60b Compare April 24, 2023 08:20
@codchen codchen merged commit 2a47361 into main Apr 24, 2023
Timwood0x10 pushed a commit to Timwood0x10/sei-tendermint that referenced this pull request Jun 7, 2023
## Describe your changes and provide context
This adds the midblock interfaces and module manager functionality to
sei-cosmos


## Testing performed to validate your change
Added unit test for module manager behavior and also tested with local
sei with sei-chain changes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants