Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Refactor Data Storage #125

Open
wants to merge 19 commits into
base: main
Choose a base branch
from
Open

Refactor Data Storage #125

wants to merge 19 commits into from

Commits on Sep 27, 2024

  1. Add subsquid parquet indexing

    @marcus-snx
    marcus-snx committed Sep 27, 2024
    Configuration menu
    Copy the full SHA
    49da37b View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2024

  1. Add ClickHouse

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0750419 View commit details
    Browse the repository at this point in the history

  2. Add ClickHouse ingestion script

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    ea3b75a View commit details
    Browse the repository at this point in the history

  3. Convert big nums to string in squid

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    7035fb0 View commit details
    Browse the repository at this point in the history

  4. Modify chunk size in squid for parquet partitions

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    0f4cb21 View commit details
    Browse the repository at this point in the history

  5. Add forced flush for squid on reaching chain head

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    6bbdb25 View commit details
    Browse the repository at this point in the history

  6. Add ClickHouse ingestion dag

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    d16ad0c View commit details
    Browse the repository at this point in the history

  7. Remove script from package.json

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    b76c4c9 View commit details
    Browse the repository at this point in the history

  8. Add ClickHouse dbt adapter and update models

    @marcus-snx
    marcus-snx committed Oct 3, 2024
    Configuration menu
    Copy the full SHA
    8971c82 View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2024

  1. Fix data type

    @marcus-snx
    marcus-snx committed Oct 4, 2024
    Configuration menu
    Copy the full SHA
    fe64b36 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2024

  1. Update dbt models

    @marcus-snx
    marcus-snx committed Oct 6, 2024
    Configuration menu
    Copy the full SHA
    34ef206 View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. Update docker compose

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    d29f9ab View commit details
    Browse the repository at this point in the history

  2. Update clickhouse import & export dags

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    72d8a86 View commit details
    Browse the repository at this point in the history

  3. Add clickhouse export to parquet script

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    1e43437 View commit details
    Browse the repository at this point in the history

  4. Update clickhouse import script

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    01da7cb View commit details
    Browse the repository at this point in the history

  5. Refactor data paths import script

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    3a5038f View commit details
    Browse the repository at this point in the history

  6. Fix import script

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    2c470b1 View commit details
    Browse the repository at this point in the history

  7. Fix dag transform task for clickhouse

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    fcb4943 View commit details
    Browse the repository at this point in the history

  8. Update data directory path

    @marcus-snx
    marcus-snx committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    02ddd5f View commit details
    Browse the repository at this point in the history