Restarting predicate server mid-scan does not continue scan #402
Assignees
Labels
Comments
MicaiahReid
added a commit
that referenced
this issue
Sep 11, 2023
### Description Improves how we handle a restart of `chainhook service` while predicates are scanning/streaming. Here are the cases we now handle: 1. Predicates that were in `scanning` status when Chainhook was terminated will resume scanning starting from their `last_evaluated_block_height`. *Note: because we only save predicate status every 10 scans, we could end up re-emiting matches on a resetart* 2. Predicates that were in `new` status when Chainhook was terminated will start scanning at the predicate's `start_block` 3. Predicates that were in `streaming` status will _return_ to a `scanning` status, starting at `last_evaluated_block_height` to catch up on the missed blocks. Note, the `number_of_blocks_to_scan` is set to 0 for this temporary catch-up, as it's difficult to compute the number of remaining blocks in the context of this change 4. If predicates were passed in at startup, we also register those to begin scanning, which previously didn't happen 5. We now allow passing in a predicate at startup _and_ registering additional predicates with the predicate registration server. This means that if you use the same startup predicate repeatedly, it will already be saved in redis and _not_ be reloaded. Fixes: #298, fixes #390, fixes #402, fixes #403 --- ### Checklist - [x] All tests pass - [ ] Tests added in this PR (if applicable)
2 tasks
MicaiahReid
added a commit
that referenced
this issue
Sep 14, 2023
### Description This PR accomplishes a few things: - renames/adds/removes predicate statuses according to the flowchart in docs/images/predicate-status-flowchart/PredicateStatusFlowchart.png - adds more context to errors returned in Interrupted status - adds status data to `GET /v1/chainhooks` endpoint - fixes bug in chainhook service where block streaming continued past `end_block` Fixes #396, fixes #324 Also: Improves how we handle a restart of `chainhook service` while predicates are scanning/streaming. Here are the cases we now handle: 1. Predicates that were in `scanning` status when Chainhook was terminated will resume scanning starting from their `last_evaluated_block_height`. *Note: because we only save predicate status every 10 scans, we could end up re-emiting matches on a resetart* 2. Predicates that were in `new` status when Chainhook was terminated will start scanning at the predicate's `start_block` 3. Predicates that were in `streaming` status will _return_ to a `scanning` status, starting at `last_evaluated_block_height` to catch up on the missed blocks. Note, the `number_of_blocks_to_scan` is set to 0 for this temporary catch-up, as it's difficult to compute the number of remaining blocks in the context of this change 4. If predicates were passed in at startup, we also register those to begin scanning, which previously didn't happen 5. We now allow passing in a predicate at startup _and_ registering additional predicates with the predicate registration server. This means that if you use the same startup predicate repeatedly, it will already be saved in redis and _not_ be reloaded. Fixes: #298, fixes #390, fixes #402, fixes #403 #### Breaking change? The rename of `ScanningData`'s `number_of_blocks_sent` field could technically be considered breaking, let's discuss. ### Checklist - [x] All tests pass - [x] Tests added in this PR (if applicable) Test coverage before: 23.2% Test coverage after: 37.72%
github-actions bot
pushed a commit
that referenced
this issue
Oct 10, 2023
## [1.1.0](v1.0.0...v1.1.0) (2023-10-10) ### Features * allow matching with regex for stacks print_event ([#380](#380)) ([131809e](131809e)), closes [#348](#348) * augment predicate status returned by GET/LIST endpoints ([#397](#397)) ([a100263](a100263)), closes [#396](#396) [#324](#324) [#390](#390) [#402](#402) [#403](#403) * introduce "data_handler_tx" ([ee486f3](ee486f3)) ### Bug Fixes * build error ([85d4d91](85d4d91)) * build errors ([b9ff1aa](b9ff1aa)) * build errro ([be0c229](be0c229)) * bump retries and delays ([aff3690](aff3690)) * chainhook not being registered ([5a809c6](5a809c6)) * ensure that the parent block was previously received. else, fetch it ([2755266](2755266)) * migrate to finer zmq lib ([4eb5a07](4eb5a07)) * prevent panic when scanning from genesis block ([#408](#408)) ([1868a06](1868a06)) * remove event_handlers ([6fecfd2](6fecfd2)) * retrieve blocks until tip ([5213f5f](5213f5f)) * revisit approach ([67a34dc](67a34dc)) * use crossbeam channels ([ea33553](ea33553)) * workflow ([d434c93](d434c93))
|
🎉 This issue has been resolved in version 1.1.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
MicaiahReid
pushed a commit
that referenced
this issue
Oct 10, 2023
## [1.1.0](v1.0.0...v1.1.0) (2023-10-10) ### Features * allow matching with regex for stacks print_event ([#380](#380)) ([131809e](131809e)), closes [#348](#348) * augment predicate status returned by GET/LIST endpoints ([#397](#397)) ([a100263](a100263)), closes [#396](#396) [#324](#324) [#390](#390) [#402](#402) [#403](#403) * introduce "data_handler_tx" ([ee486f3](ee486f3)) ### Bug Fixes * build error ([85d4d91](85d4d91)) * build errors ([b9ff1aa](b9ff1aa)) * build errro ([be0c229](be0c229)) * bump retries and delays ([aff3690](aff3690)) * chainhook not being registered ([5a809c6](5a809c6)) * ensure that the parent block was previously received. else, fetch it ([2755266](2755266)) * migrate to finer zmq lib ([4eb5a07](4eb5a07)) * prevent panic when scanning from genesis block ([#408](#408)) ([1868a06](1868a06)) * remove event_handlers ([6fecfd2](6fecfd2)) * retrieve blocks until tip ([5213f5f](5213f5f)) * revisit approach ([67a34dc](67a34dc)) * use crossbeam channels ([ea33553](ea33553)) * workflow ([d434c93](d434c93))
vabanaerytk
added a commit
to vabanaerytk/chainhook
that referenced
this issue
Aug 7, 2024
## [1.1.0](hirosystems/chainhook@v1.0.0...v1.1.0) (2023-10-10) ### Features * allow matching with regex for stacks print_event ([#380](hirosystems/chainhook#380)) ([af0fcad](hirosystems/chainhook@af0fcad)), closes [#348](hirosystems/chainhook#348) * augment predicate status returned by GET/LIST endpoints ([#397](hirosystems/chainhook#397)) ([0f32704](hirosystems/chainhook@0f32704)), closes [#396](hirosystems/chainhook#396) [#324](hirosystems/chainhook#324) [#390](hirosystems/chainhook#390) [#402](hirosystems/chainhook#402) [#403](hirosystems/chainhook#403) * introduce "data_handler_tx" ([a7704d8](hirosystems/chainhook@a7704d8)) ### Bug Fixes * build error ([9d47f87](hirosystems/chainhook@9d47f87)) * build errors ([27b0ae4](hirosystems/chainhook@27b0ae4)) * build errro ([6363fad](hirosystems/chainhook@6363fad)) * bump retries and delays ([483663c](hirosystems/chainhook@483663c)) * chainhook not being registered ([7d2b156](hirosystems/chainhook@7d2b156)) * ensure that the parent block was previously received. else, fetch it ([79561c8](hirosystems/chainhook@79561c8)) * migrate to finer zmq lib ([2e55a20](hirosystems/chainhook@2e55a20)) * prevent panic when scanning from genesis block ([#408](hirosystems/chainhook#408)) ([4ab8dd8](hirosystems/chainhook@4ab8dd8)) * remove event_handlers ([e3664d2](hirosystems/chainhook@e3664d2)) * retrieve blocks until tip ([d21cacc](hirosystems/chainhook@d21cacc)) * revisit approach ([47d2ef6](hirosystems/chainhook@47d2ef6)) * use crossbeam channels ([50595ab](hirosystems/chainhook@50595ab)) * workflow ([84c4ec9](hirosystems/chainhook@84c4ec9))
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: