Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add last triggered timestamp to GET and LIST endpoints #324

Closed
BLuEScioN opened this issue Jul 7, 2023 · 10 comments · Fixed by #397
Closed

Add last triggered timestamp to GET and LIST endpoints #324

BLuEScioN opened this issue Jul 7, 2023 · 10 comments · Fixed by #397
Assignees
@MicaiahReid
Labels
released
Milestone
Q3-2023

Comments

@BLuEScioN
Copy link

BLuEScioN commented Jul 7, 2023 •
edited by sabbyanandan
Loading

Aug 21st, 2023:
After discussing this requirement with @MicaiahReid and @BLuEScioN, we will add last_triggered (i.e., timestamp) as part of the GET and LIST endpoints.
Sabby will open a separate issue to account for the error messages.

Was 2nd:

Add these fields to the GET and LIST endpoint:
1: Last Updated: Timestamp
2: Last Triggered: Timestamp
Add this to GET endpoint:
1: 1-line of the last log: String (maybe 100 chars?)

Was 1st:

Currently its only possible to tell whether a chainhook is still active by determining whether the endblock specified in the chainhook config has been surpassed or not, or whether the startblock has been reached yet. But there is a third bit of information that may determine whether or not the chainhook is still active - the number of occurences before expiration. Currently, there is no way to retrieve how many times the chainhook has been activated. Thus, there is no way to accurately determine if the chainhook is still active. I recommend that an isActive prop be added to the GET and/or LIST endpoints to make it easier to consume this information. This kind of information would be very useful to display for chainhooks on the Platform.
@BLuEScioN BLuEScioN converted this from a draft issue Jul 7, 2023
@sabbyanandan
Copy link

@lgalabru: Additionally, it'd be great to include the following in the response payload for each chainhook (if it is not expensive).

  • Last run's timestamp
  • Last run's 1-line log or any details about it

@andresgalante
Copy link
Member

@smcclellan We'll start working on devex improvements for chainhooks and this issue became a dependency on it.

When do you think you can prioritize it?

@sabbyanandan
Copy link

sabbyanandan commented Aug 14, 2023 •
edited
Loading

@lgalabru: It'd be great to get the "liveness" visibility of the each hook. How about:

1: Is Active: Boolean (already supported)
2: Last Updated: Timestamp
3: Last Triggered: Timestamp
3: 1-line of the last log: String (maybe 100 chars?)

@smcclellan smcclellan moved this from 📋 Backlog to 🆕 New in DevTools Aug 14, 2023
@smcclellan
Copy link
Contributor

The ability to query this info is already live on GET /v1/chainhooks/:uuid.

This issue will track verifying the freshness of the data being returned AND have the data returned at the LIST endpoint.

@smcclellan smcclellan added this to the Q3-2023 milestone Aug 14, 2023
@smcclellan smcclellan moved this from 🆕 New to 📋 Backlog in DevTools Aug 14, 2023
@smcclellan
Copy link
Contributor

@andresgalante Targeting by August 28th (next Iteration). Does that work for your timing?

@sabbyanandan
Copy link

@smcclellan: I would prioritize this as urgent.

@sabbyanandan
Copy link

The ability to query this info is already live on GET /v1/chainhooks/:uuid.

Also, @smcclellan, we may have to add additional bits what I listed in my comment, correct? I don't think they are in the response.

@smcclellan smcclellan changed the title add information to GET and LIST endpoints for chainhooks that states whether the chainhook is still active Add timestamps, last log line to GET and LIST endpoints for chainhooks Aug 17, 2023
@MicaiahReid
Copy link
Contributor

MicaiahReid commented Aug 20, 2023 •
edited
Loading

Hey @BLuEScioN, I'm starting work on this, and I've got some questions.

Last Updated: Timestamp

What do you mean by "updated" here? My understanding is that you can't update a predicate, only delete and create.

1-line of the last log

What logs are you referring to here? We don't keep previous logs on-hand, and storing them would be a pretty large change. Maybe we can get on a call so we can come up with ideas on how to get the info you are needing.

@sabbyanandan sabbyanandan changed the title Add timestamps, last log line to GET and LIST endpoints for chainhooks Add last triggered timestamp to GET and LIST endpoints Aug 21, 2023
@sabbyanandan
Copy link

I updated the description based on the discussion and what we decided. I also created #396 to track this as a separate enhancement.

@smcclellan: Let's scope 396, and we can discuss the priorities then.

@MicaiahReid MicaiahReid moved this from 📋 Backlog to 🏗 In Progress in DevTools Aug 22, 2023
@MicaiahReid MicaiahReid mentioned this issue Aug 24, 2023
Merged
2 tasks
@MicaiahReid MicaiahReid moved this from 🏗 In Progress to 👀 In Review in DevTools Aug 24, 2023
@smcclellan smcclellan modified the milestones: Q3-2023, Q4-2023 Sep 1, 2023
MicaiahReid added a commit that referenced this issue Sep 14, 2023
@MicaiahReid
feat: augment predicate status returned by GET/LIST endpoints (#397)
a100263 
### Description

This PR accomplishes a few things:
- renames/adds/removes predicate statuses according to the flowchart in
docs/images/predicate-status-flowchart/PredicateStatusFlowchart.png
 - adds more context to errors returned in Interrupted status
 - adds status data to `GET /v1/chainhooks` endpoint
- fixes bug in chainhook service where block streaming continued past
`end_block`

Fixes #396, fixes #324 

Also: 
Improves how we handle a restart of `chainhook service` while predicates
are scanning/streaming. Here are the cases we now handle:
1. Predicates that were in `scanning` status when Chainhook was
terminated will resume scanning starting from their
`last_evaluated_block_height`. *Note: because we only save predicate
status every 10 scans, we could end up re-emiting matches on a resetart*
2. Predicates that were in `new` status when Chainhook was terminated
will start scanning at the predicate's `start_block`
3. Predicates that were in `streaming` status will _return_ to a
`scanning` status, starting at `last_evaluated_block_height` to catch up
on the missed blocks. Note, the `number_of_blocks_to_scan` is set to 0
for this temporary catch-up, as it's difficult to compute the number of
remaining blocks in the context of this change
4. If predicates were passed in at startup, we also register those to
begin scanning, which previously didn't happen
5. We now allow passing in a predicate at startup _and_ registering
additional predicates with the predicate registration server. This means
that if you use the same startup predicate repeatedly, it will already
be saved in redis and _not_ be reloaded.

Fixes: #298, fixes #390, fixes #402, fixes #403
#### Breaking change?

The rename of `ScanningData`'s `number_of_blocks_sent` field could
technically be considered breaking, let's discuss.


### Checklist

- [x] All tests pass
- [x] Tests added in this PR (if applicable)
Test coverage before: 23.2%
Test coverage after: 37.72%
@github-project-automation github-project-automation bot moved this from 👀 In Review to ✅ Done in DevTools Sep 14, 2023
github-actions bot pushed a commit that referenced this issue Oct 10, 2023
@semantic-release-bot
chore(release): 1.1.0 [skip ci]
8ca9afa 
## [1.1.0](v1.0.0...v1.1.0) (2023-10-10)

### Features

* allow matching with regex for stacks print_event ([#380](#380)) ([131809e](131809e)), closes [#348](#348)
* augment predicate status returned by GET/LIST endpoints ([#397](#397)) ([a100263](a100263)), closes [#396](#396) [#324](#324) [#390](#390) [#402](#402) [#403](#403)
* introduce "data_handler_tx" ([ee486f3](ee486f3))

### Bug Fixes

* build error ([85d4d91](85d4d91))
* build errors ([b9ff1aa](b9ff1aa))
* build errro ([be0c229](be0c229))
* bump retries and delays ([aff3690](aff3690))
* chainhook not being registered ([5a809c6](5a809c6))
* ensure that the parent block was previously received. else, fetch it ([2755266](2755266))
* migrate to finer zmq lib ([4eb5a07](4eb5a07))
* prevent panic when scanning from genesis block ([#408](#408)) ([1868a06](1868a06))
* remove event_handlers ([6fecfd2](6fecfd2))
* retrieve blocks until tip ([5213f5f](5213f5f))
* revisit approach ([67a34dc](67a34dc))
* use crossbeam channels ([ea33553](ea33553))
* workflow ([d434c93](d434c93))
@github-actions
Copy link

🎉 This issue has been resolved in version 1.1.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

MicaiahReid pushed a commit that referenced this issue Oct 10, 2023
@semantic-release-bot @MicaiahReid
chore(release): 1.1.0 [skip ci]
09b68ca 
## [1.1.0](v1.0.0...v1.1.0) (2023-10-10)

### Features

* allow matching with regex for stacks print_event ([#380](#380)) ([131809e](131809e)), closes [#348](#348)
* augment predicate status returned by GET/LIST endpoints ([#397](#397)) ([a100263](a100263)), closes [#396](#396) [#324](#324) [#390](#390) [#402](#402) [#403](#403)
* introduce "data_handler_tx" ([ee486f3](ee486f3))

### Bug Fixes

* build error ([85d4d91](85d4d91))
* build errors ([b9ff1aa](b9ff1aa))
* build errro ([be0c229](be0c229))
* bump retries and delays ([aff3690](aff3690))
* chainhook not being registered ([5a809c6](5a809c6))
* ensure that the parent block was previously received. else, fetch it ([2755266](2755266))
* migrate to finer zmq lib ([4eb5a07](4eb5a07))
* prevent panic when scanning from genesis block ([#408](#408)) ([1868a06](1868a06))
* remove event_handlers ([6fecfd2](6fecfd2))
* retrieve blocks until tip ([5213f5f](5213f5f))
* revisit approach ([67a34dc](67a34dc))
* use crossbeam channels ([ea33553](ea33553))
* workflow ([d434c93](d434c93))
@sabbyanandan sabbyanandan mentioned this issue Mar 18, 2024
Open
vabanaerytk added a commit to vabanaerytk/chainhook that referenced this issue Aug 7, 2024
@vabanaerytk @MicaiahReid
chore(release): 1.1.0 [skip ci]
e4fc545 
## [1.1.0](hirosystems/chainhook@v1.0.0...v1.1.0) (2023-10-10)

### Features

* allow matching with regex for stacks print_event ([#380](hirosystems/chainhook#380)) ([af0fcad](hirosystems/chainhook@af0fcad)), closes [#348](hirosystems/chainhook#348)
* augment predicate status returned by GET/LIST endpoints ([#397](hirosystems/chainhook#397)) ([0f32704](hirosystems/chainhook@0f32704)), closes [#396](hirosystems/chainhook#396) [#324](hirosystems/chainhook#324) [#390](hirosystems/chainhook#390) [#402](hirosystems/chainhook#402) [#403](hirosystems/chainhook#403)
* introduce "data_handler_tx" ([a7704d8](hirosystems/chainhook@a7704d8))

### Bug Fixes

* build error ([9d47f87](hirosystems/chainhook@9d47f87))
* build errors ([27b0ae4](hirosystems/chainhook@27b0ae4))
* build errro ([6363fad](hirosystems/chainhook@6363fad))
* bump retries and delays ([483663c](hirosystems/chainhook@483663c))
* chainhook not being registered ([7d2b156](hirosystems/chainhook@7d2b156))
* ensure that the parent block was previously received. else, fetch it ([79561c8](hirosystems/chainhook@79561c8))
* migrate to finer zmq lib ([2e55a20](hirosystems/chainhook@2e55a20))
* prevent panic when scanning from genesis block ([#408](hirosystems/chainhook#408)) ([4ab8dd8](hirosystems/chainhook@4ab8dd8))
* remove event_handlers ([e3664d2](hirosystems/chainhook@e3664d2))
* retrieve blocks until tip ([d21cacc](hirosystems/chainhook@d21cacc))
* revisit approach ([47d2ef6](hirosystems/chainhook@47d2ef6))
* use crossbeam channels ([50595ab](hirosystems/chainhook@50595ab))
* workflow ([84c4ec9](hirosystems/chainhook@84c4ec9))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@MicaiahReid MicaiahReid

Labels
released
Projects
DevTools
Archived in project
Milestone
Q3-2023
Development

Successfully merging a pull request may close this issue.

feat: augment predicate status returned by GET/LIST endpoints hirosystems/chainhook
5 participants
@sabbyanandan @andresgalante @smcclellan @MicaiahReid @BLuEScioN