Add recent uptimes to validator metadata in the client api #800
Conversation
| window_length: | ||
| type: integer | ||
| format: uint64 | ||
| description: The length of the window this object describes, in blocks. |
There was a problem hiding this comment.
Maybe mention that this is currently always 14400 (~last 24 hours).
| partial_length: | ||
| type: integer | ||
| format: uint64 | ||
| description: The length of the partial windows within window_length, in blocks. |
There was a problem hiding this comment.
Maybe mention that this will currently always be 1200 (~2 hours).
| @@ -316,6 +318,10 @@ func (m *processor) queueDbUpdates(batch *storage.QueryBatch, data allData) erro | |||
| } | |||
| } | |||
|
|
|||
| if m.mode == analyzer.SlowSyncMode { | |||
| batch.Queue(validatorUptimesViewRefreshQuery) | |||
There was a problem hiding this comment.
I don't think we need to run this after every processed block. Since the "partial windows" are 1200 blocks in size, refreshing every ~12 blocks (1% of the window size) or ~60 blocks (5% of the window size) should be reasonable. Any change is unlikely to be noticeable before that.
It might be clearer to run this periodically, such as every minute or so, with the interval being configurable.
| @@ -44,6 +44,12 @@ const ( | |||
| maxTotalCount = 1000 | |||
| ) | |||
|
|
|||
| var ( | |||
| // These two should be kept the same as in the views.validator_uptimes materialized view. | |||
There was a problem hiding this comment.
These two can be made constants instead of variables.
| UNNEST(signer_entity_ids) AS signer_entity_id, | ||
| (ROW_NUMBER() OVER (ORDER BY height DESC) - 1) / 1200 AS slot_id |
There was a problem hiding this comment.
Can you double check that the unnest does not cause unexpected results with row_number here? I'm wondering if each (signer x height) will have its own row which would then cause row_number to vastly surpass 14400 and thus overflow the slot_id.
| SELECT * FROM views.validator_uptimes | ||
| WHERE ($1::text IS NULL OR signer_entity_id = $1::text) | ||
| ORDER BY signer_entity_id` | ||
| /*ValidatorUptimes = ` |
| @@ -433,6 +433,58 @@ const ( | |||
| LIMIT $4::bigint | |||
| OFFSET $5::bigint` | |||
|
|
|||
| ValidatorUptimes = ` | |||
| SELECT * FROM views.validator_uptimes | |||
| WHERE ($1::text IS NULL OR signer_entity_id = $1::text) | |||
There was a problem hiding this comment.
Looks like the input param $1 is the staking address, can add a join with chain.entities.address to filter by address here. Alternatively, it might be cleaner to just include the entity address in the materialized view to avoid the join.
| signers.signer_entity_id, | ||
| ARRAY_AGG(COALESCE(slot_counts.signed_blocks_count, 0) ORDER BY slot_counts.slot_id) AS slot_signed, | ||
| COALESCE(SUM(signed_blocks_count), 0) AS overall_signed | ||
| FROM | ||
| -- Ensure we have all windows for each signer, even if they didn't sign in a particular window. | ||
| (SELECT DISTINCT signer_entity_id FROM slot_counts) AS signers | ||
| CROSS JOIN all_slots | ||
| LEFT JOIN slot_counts ON signers.signer_entity_id = slot_counts.signer_entity_id AND all_slots.slot_id = slot_counts.slot_id |
There was a problem hiding this comment.
Neat sql-fu! Seems like the e2e tests might not be suited to testing this properly; mind validating it locally by running a local nexus instance and printing the table / querying the api? Thanks!
| uptimeWindowBlocks = uint64(14400) | ||
| uptimeSlotBlocks = uint64(1200) |
There was a problem hiding this comment.
Hm wdyt of hardcoding these into the postgres query instead of here? It'd be nice to colocate them together so that if it changes, we only need to make changes in one place.
| partial_length: | ||
| type: integer | ||
| format: uint64 | ||
| description: The length of the partial windows within window_length, in blocks. |
There was a problem hiding this comment.
| partial_length: | |
| type: integer | |
| format: uint64 | |
| description: The length of the partial windows within window_length, in blocks. | |
| segment_length: | |
| type: integer | |
| format: uint64 | |
| description: The length of the window segment, in blocks. We subdivide the window into segments of equal length and aggregate the uptime of each segment into `segment_uptimes`. Currently defaults to 1200 blocks, which is approximately 2 hours. |
There was a problem hiding this comment.
I'm not very enthusiastic about segment, I considered using chunk or subwindow too. Feel free to change
| window_length: | ||
| type: integer | ||
| format: uint64 | ||
| description: The length of the window this object describes, in blocks. |
There was a problem hiding this comment.
| description: The length of the window this object describes, in blocks. | |
| description: The length of the historical window for which this object provides uptime information, in blocks. Currently defaults to 14400 blocks, or approximately 24 hours. |
| partial_uptimes: | ||
| type: array | ||
| description: An array showing the signed block counts for each partial slot within window_length. |
There was a problem hiding this comment.
| partial_uptimes: | |
| type: array | |
| description: An array showing the signed block counts for each partial slot within window_length. | |
| segment_uptimes: | |
| type: array | |
| description: An array showing the signed block counts for each sub-segment within window_length. The segments are in reverse-chronological order; ie the first element represents the most recent segment of blocks. |
| @@ -316,6 +318,10 @@ func (m *processor) queueDbUpdates(batch *storage.QueryBatch, data allData) erro | |||
| } | |||
| } | |||
|
|
|||
| if m.mode == analyzer.SlowSyncMode { | |||
| batch.Queue(validatorUptimesViewRefreshQuery) | |||
There was a problem hiding this comment.
Let's refresh this in a new ItemBasedAnalyzer, see accounts_list.go for an example. Alternatively, feel free to refactor that analyzer to refresh both views; I think we'll soon have more views that need refreshing, and it'd be good to have them all in one spot so we can tweak how often they get recalculated.
No description provided.