Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronize state to an external database #7099

Closed
aaronc opened this issue Aug 18, 2020 · 1 comment
Closed

Synchronize state to an external database #7099

aaronc opened this issue Aug 18, 2020 · 1 comment

Comments

@aaronc
Copy link
Member

aaronc commented Aug 18, 2020

This is linked to meta-issue #7096.

Summary

This issue discusses approaches to facilitating indexing of state (not transactions) to an external database. Transaction indexing is already covered by Tendermint, but we don't have a generic state syncing mechanism in the SDK.

Problem Definition

Various projects would like to index chain state to an external database. Mature database software like PostgreSQL provide robust secondary index support and a full-featured query language which can make working with data much easier.

Proposal

The SDK should support some basic mechanism for streaming state changes to an external database.

The state synchronization mechanism should provide some level of fault tolerance so that if the connection is lost, the sync process can pick up from where it left off.

The SDK provides a basic tracekv.Store which serializes kv-store changes to stdout. An acceptable mechanism for synchronizing to an external database would likely be similar but we should think through how to make it robust.

Indexing state to an external database is also supported by the proposed schema module in #7097 which would allow the indexer to know how to decode key value pairs and possibly automatically generate database tables. But that functionality is orthogonal to the question of how to sync.

/cc @AFDudley

@tac0turtle
Copy link
Member

this is done via state streaming

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants