Grpc ingest triton build (#142)

* fmt * clippy * ci: add lock, fmt, clippy checks * include migration to workspace * include das_api to workspace * add global clippy lints * use workspace * add crate program_transformers * nft_ingester: use program_transformers crate * remove not used deps * remove AccountInfo * remove plerkle from program_transformers * nft_ingester2: grpc2redis * add redis streaming for ingester * create pg pool * parse incoming message from redis * add force shutdown with signals * insert download metadata tasks * download-metadata subtask * refactor: rename nft_ingest2 * fix: applying account and transction filters to grpc subscription request --------- Co-authored-by: Kirill Fomichev <[email protected]>
rpcpool · Apr 16, 2024 · 36cba58 · 36cba58
1 parent 8f7baa7
commit 36cba58
Show file tree

Hide file tree

Showing 25 changed files with 2,873 additions and 114 deletions.
diff --git a/.github/workflows/build-api.yml b/.github/workflows/build-api.yml
@@ -12,10 +12,10 @@ on:
   workflow_dispatch:
   push:
     branches:
-      - 'main'
+      - "main"
 
 env:
-   CARGO_TERM_COLOR: always
+  CARGO_TERM_COLOR: always
 
 jobs:
   build-api:
@@ -33,15 +33,15 @@ jobs:
       - name: set build cache
         uses: actions/cache@v3
         with:
-            path: |
-                ~/.cargo/bin/
-                ~/.cargo/registry/index/
-                ~/.cargo/registry/cache/
-                ~/.cargo/git/db/
-                digital-asset-rpc-infrastructure/target/
-            key: ${{ matrix.os }}_digital-asset-rpc-infrastructure_${{ hashFiles('digital-asset-rpc-infrastructure/Cargo.lock') }}
-            restore-keys: |
-              ${{ matrix.os }}_digital-asset-rpc-infrastructure
+          path: |
+            ~/.cargo/bin/
+            ~/.cargo/registry/index/
+            ~/.cargo/registry/cache/
+            ~/.cargo/git/db/
+            digital-asset-rpc-infrastructure/target/
+          key: ${{ matrix.os }}_digital-asset-rpc-infrastructure_${{ hashFiles('digital-asset-rpc-infrastructure/Cargo.lock') }}
+          restore-keys: |
+            ${{ matrix.os }}_digital-asset-rpc-infrastructure
 
       - name: build digital asset rpc infra
         run: cargo build --verbose --release
@@ -54,7 +54,7 @@ jobs:
           mv target/release/migration target/release/migration22
           mv target/release/das_api target/release/das_api22
 
-      # This steps can be omited to save space, are mostly in place to validate binaries (manually) and path to them 
+      # This steps can be omited to save space, are mostly in place to validate binaries (manually) and path to them
       # Omiting this will save on storage consumption on the account
       - name: Publish artifact
         if: matrix.os == 'ubuntu-22.04'

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -8,11 +8,11 @@ on:
   pull_request:
   push:
     branches:
-      - 'main'
+      - "main"
   workflow_dispatch:
 
 env:
-   CARGO_TERM_COLOR: always
+  CARGO_TERM_COLOR: always
 
 jobs:
   test:
@@ -25,20 +25,20 @@ jobs:
       - name: set build cache
         uses: actions/cache@v3
         with:
-            path: |
-                ~/.cargo/bin/
-                ~/.cargo/registry/index/
-                ~/.cargo/registry/cache/
-                ~/.cargo/git/db/
-                digital-asset-rpc-infrastructure/target/
-            key: cargo-${{ hashFiles('**/Cargo.lock') }}-0001
+          path: |
+            ~/.cargo/bin/
+            ~/.cargo/registry/index/
+            ~/.cargo/registry/cache/
+            ~/.cargo/git/db/
+            digital-asset-rpc-infrastructure/target/
+          key: cargo-${{ hashFiles('**/Cargo.lock') }}-0001
 
       # Cargo.lock
       - name: Check lock file
         run: |
           cargo tree
           git checkout Cargo.lock
-          cargo tree --frozen
+          cargo tree
 
       # fmt
       - name: Check fmt