chainhook predicates scan should use rocksdb if present
#485
Assignees
Labels
Comments
MicaiahReid
added a commit
that referenced
this issue
Mar 19, 2024
### Description Previously, when using the `chainhook predicates scan` command, Chainhook would download a very large archive file to use as a data source for the scan **every time the command was run**. The Chainhook service (`chainhook service start`), however, implements rocksdb to allow storing this dataset locally. This PR uses the rocksdb that a user may have created locally to improve performance of the `chainhook predicates scan` command. If a rocksdb doesn't exist or can't be created, Chainhook will fallback to the original method of downloading the archive. Combined with #513, this PR reduces a simple scan of a few blocks from ~20 minutes to .02 seconds. Fixes #485
MicaiahReid
added a commit
that referenced
this issue
Mar 27, 2024
### Description Previously, when using the `chainhook predicates scan` command, Chainhook would download a very large archive file to use as a data source for the scan **every time the command was run**. The Chainhook service (`chainhook service start`), however, implements rocksdb to allow storing this dataset locally. This PR uses the rocksdb that a user may have created locally to improve performance of the `chainhook predicates scan` command. If a rocksdb doesn't exist or can't be created, Chainhook will fallback to the original method of downloading the archive. Combined with #513, this PR reduces a simple scan of a few blocks from ~20 minutes to .02 seconds. Fixes #485
MicaiahReid
added a commit
that referenced
this issue
Mar 27, 2024
### Description Previously, when using the `chainhook predicates scan` command, Chainhook would download a very large archive file to use as a data source for the scan **every time the command was run**. The Chainhook service (`chainhook service start`), however, implements rocksdb to allow storing this dataset locally. This PR uses the rocksdb that a user may have created locally to improve performance of the `chainhook predicates scan` command. If a rocksdb doesn't exist or can't be created, Chainhook will fallback to the original method of downloading the archive. Combined with #513, this PR reduces a simple scan of a few blocks from ~20 minutes to .02 seconds. Fixes #485
github-actions bot
pushed a commit
that referenced
this issue
Mar 27, 2024
## [1.4.0](v1.3.1...v1.4.0) (2024-03-27) ### Features * detect http / rpc errors as early as possible ([ad78669](ad78669)) * use stacks.rocksdb for predicate scan ([#514](#514)) ([a4f1663](a4f1663)), closes [#513](#513) [#485](#485) ### Bug Fixes * enable debug logs in release mode ([#537](#537)) ([fb49e28](fb49e28)) * improve error handling, and more! ([#524](#524)) ([86b5c78](86b5c78)), closes [#498](#498) [#521](#521) [#404](#404) [/github.com//issues/517#issuecomment-1992135101](https://github.com/hirosystems//github.com/hirosystems/chainhook/issues/517/issues/issuecomment-1992135101) [#517](#517) [#506](#506) [#510](#510) [#519](#519) * log errors on block download failure; implement max retries ([#503](#503)) ([0fc38cb](0fc38cb)) * **metrics:** update latest ingested block on reorg ([#515](#515)) ([8f728f7](8f728f7)) * order and filter blocks used to seed forking block pool ([#534](#534)) ([a11bc1c](a11bc1c)) * seed forking handler with unconfirmed blocks to improve startup stability ([#505](#505)) ([485394e](485394e)), closes [#487](#487) * skip db consolidation if no new dataset was downloaded ([#513](#513)) ([983a165](983a165)) * update scan status for non-triggering predicates ([#511](#511)) ([9073f42](9073f42)), closes [#498](#498)
|
🎉 This issue has been resolved in version 1.4.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
github-actions bot
pushed a commit
that referenced
this issue
Apr 6, 2024
## [1.5.0](v1.4.1...v1.5.0) (2024-04-06) ### Features * add brc-20 schemas for ordhook ([#551](#551)) ([1e25a8f](1e25a8f)) * detect http / rpc errors as early as possible ([e515116](e515116)) * rm blocks ([f35498d](f35498d)) * use stacks.rocksdb for predicate scan ([#514](#514)) ([0baae10](0baae10)), closes [#513](#513) [#485](#485) ### Bug Fixes * build error ([03b3938](03b3938)) * log errors on block download failure; implement max retries ([#503](#503)) ([3bfb0e1](3bfb0e1)) * **metrics:** update latest ingested block on reorg ([#515](#515)) ([823713a](823713a)) * order and filter blocks used to seed forking block pool ([#534](#534)) ([a2865b7](a2865b7)) * revisit 7+ blocks reorg handling ([#553](#553)) ([184fd07](184fd07)) * seed forking handler with unconfirmed blocks to improve startup stability ([#505](#505)) ([b77aca2](b77aca2)), closes [#487](#487) * skip db consolidation if no new dataset was downloaded ([#513](#513)) ([b1469a6](b1469a6)) * update scan status for non-triggering predicates ([#511](#511)) ([32cdfee](32cdfee)), closes [#498](#498)
|
🎉 This issue has been resolved in version 1.5.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
vabanaerytk
added a commit
to vabanaerytk/chainhook
that referenced
this issue
Aug 7, 2024
## [1.4.0](hirosystems/chainhook@v1.3.1...v1.4.0) (2024-03-27) ### Features * detect http / rpc errors as early as possible ([4a3e7e8](hirosystems/chainhook@4a3e7e8)) * use stacks.rocksdb for predicate scan ([#514](hirosystems/chainhook#514)) ([bff1372](hirosystems/chainhook@bff1372)), closes [#513](hirosystems/chainhook#513) [#485](hirosystems/chainhook#485) ### Bug Fixes * enable debug logs in release mode ([#537](hirosystems/chainhook#537)) ([5ff21e0](hirosystems/chainhook@5ff21e0)) * improve error handling, and more! ([#524](hirosystems/chainhook#524)) ([2cfedd7](hirosystems/chainhook@2cfedd7)), closes [#498](hirosystems/chainhook#498) [#521](hirosystems/chainhook#521) [#404](hirosystems/chainhook#404) [/github.com/hirosystems/chainhook/issues/517#issuecomment-1992135101](https://github.com/hirosystems//github.com/hirosystems/chainhook/issues/517/issues/issuecomment-1992135101) [#517](hirosystems/chainhook#517) [#506](hirosystems/chainhook#506) [#510](hirosystems/chainhook#510) [#519](hirosystems/chainhook#519) * log errors on block download failure; implement max retries ([#503](hirosystems/chainhook#503)) ([0454a4f](hirosystems/chainhook@0454a4f)) * **metrics:** update latest ingested block on reorg ([#515](hirosystems/chainhook#515)) ([7653766](hirosystems/chainhook@7653766)) * order and filter blocks used to seed forking block pool ([#534](hirosystems/chainhook#534)) ([ff775de](hirosystems/chainhook@ff775de)) * seed forking handler with unconfirmed blocks to improve startup stability ([#505](hirosystems/chainhook#505)) ([063e0e2](hirosystems/chainhook@063e0e2)), closes [#487](hirosystems/chainhook#487) * skip db consolidation if no new dataset was downloaded ([#513](hirosystems/chainhook#513)) ([f34dee3](hirosystems/chainhook@f34dee3)) * update scan status for non-triggering predicates ([#511](hirosystems/chainhook#511)) ([9647e72](hirosystems/chainhook@9647e72)), closes [#498](hirosystems/chainhook#498)
vabanaerytk
added a commit
to vabanaerytk/chainhook
that referenced
this issue
Aug 7, 2024
## [1.5.0](hirosystems/chainhook@v1.4.1...v1.5.0) (2024-04-06) ### Features * add brc-20 schemas for ordhook ([#551](hirosystems/chainhook#551)) ([4cfa300](hirosystems/chainhook@4cfa300)) * detect http / rpc errors as early as possible ([4ef6650](hirosystems/chainhook@4ef6650)) * rm blocks ([3d57290](hirosystems/chainhook@3d57290)) * use stacks.rocksdb for predicate scan ([#514](hirosystems/chainhook#514)) ([e8bc60b](hirosystems/chainhook@e8bc60b)), closes [#513](hirosystems/chainhook#513) [#485](hirosystems/chainhook#485) ### Bug Fixes * build error ([f0fb623](hirosystems/chainhook@f0fb623)) * log errors on block download failure; implement max retries ([#503](hirosystems/chainhook#503)) ([7bd6efd](hirosystems/chainhook@7bd6efd)) * **metrics:** update latest ingested block on reorg ([#515](hirosystems/chainhook#515)) ([5a7d64e](hirosystems/chainhook@5a7d64e)) * order and filter blocks used to seed forking block pool ([#534](hirosystems/chainhook#534)) ([bb2a545](hirosystems/chainhook@bb2a545)) * revisit 7+ blocks reorg handling ([#553](hirosystems/chainhook#553)) ([3632176](hirosystems/chainhook@3632176)) * seed forking handler with unconfirmed blocks to improve startup stability ([#505](hirosystems/chainhook#505)) ([a84fc4c](hirosystems/chainhook@a84fc4c)), closes [#487](hirosystems/chainhook#487) * skip db consolidation if no new dataset was downloaded ([#513](hirosystems/chainhook#513)) ([440a8e3](hirosystems/chainhook@440a8e3)) * update scan status for non-triggering predicates ([#511](hirosystems/chainhook#511)) ([115ac28](hirosystems/chainhook@115ac28)), closes [#498](hirosystems/chainhook#498)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
chainhook/components/chainhook-cli/src/cli/mod.rs
Lines 481 to 482 in 5503c3d
The text was updated successfully, but these errors were encountered: