Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dragonmouth based ingester #147

Closed
wants to merge 23 commits into from

Conversation

fanatid
Copy link
Collaborator

@fanatid fanatid commented Nov 28, 2023

Goal

Reduce the operational cost and complexity of running DAS by switching to Dragonmouth grpc stream for account and transaction updates.

Upgrade the ingestion engine by refactoring configuration, workers, and metrics.

Approach

Create a new crate named das-ingest which has 3 primary subcommands.

  • stream - connects to Dragonmouth and pushes relevant events into a redis queue
  • ingest - subscribes to redis streams, spawns workers, and processes events using program_transformers crate.
  • download - fetches metadata json for asset_data replacement to bgtasks

Based on #144

closes #83

@kespinola kespinola changed the title add nft_ingester2 Dragonmouth based ingester Mar 15, 2024
@kespinola
Copy link
Collaborator

moved here

#183

@kespinola kespinola closed this Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Proposal: gRPC based indexer
2 participants