Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add lightclient routes #181

Closed
wants to merge 10 commits into from
Closed
16 changes: 14 additions & 2 deletions apis/eventstream/index.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ get:
- finalized_checkpoint
- chain_reorg
- contribution_and_proof
- lightclient_head_update
responses:
"200":
description: Opened SSE stream.
Expand Down Expand Up @@ -73,6 +74,18 @@ get:
value: |
event: contribution_and_proof
data: {"message": {"aggregator_index": "997", "contribution": {"slot": "168097", "beacon_block_root": "0x56f1fd4262c08fa81e27621c370e187e621a67fc80fe42340b07519f84b42ea1", "subcommittee_index": "0", "aggregation_bits": "0xffffffffffffffffffffffffffffffff", "signature": "0x85ab9018e14963026476fdf784cc674da144b3dbdb47516185438768774f077d882087b90ad642469902e782a8b43eed0cfc1b862aa9a473b54c98d860424a702297b4b648f3f30bdaae8a8b7627d10d04cb96a2cc8376af3e54a9aa0c8145e3"}, "selection_proof": "0x87c305f04bfe5db27c2b19fc23e00d7ac496ec7d3e759cbfdd1035cb8cf6caaa17a36a95a08ba78c282725e7b66a76820ca4eb333822bd399ceeb9807a0f2926c67ce67cfe06a0b0006838203b493505a8457eb79913ce1a3bcd1cc8e4ef30ed"}, "signature": "0xac118511474a94f857300b315c50585c32a713e4452e26a6bb98cdb619936370f126ed3b6bb64469259ee92e69791d9e12d324ce6fd90081680ce72f39d85d50b0ff977260a8667465e613362c6d6e6e745e1f9323ec1d6f16041c4e358839ac"}
lightclient_head_update:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a need for _update here; the event is, by definition, an update.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, tho I'm unsure if lightclient_head can also be confusing since you can get multiple events for the same head which is not the same behavior as the head event.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was called lightlcient_head_update because the data structure emitted is called a LightClientHeadUpdate somewhere.

Similar to the LightClientUpdate in the specs.

Not saying its a great name, just saying..

description: |
The node has received or constructed a new or better SyncAggregate for a header. `header` is the block header signed by `sync_aggregate`.
Nodes should track the participation of emitted updates per header and only emit updates if `new_participation > prev_participation + 16`.
Updates may be collected via:
- Next block's syncAggregate field
- Aggregating messages from gossip topic `sync_committee_contribution_and_proof`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this considered safe, if this information is not yet in a (voted upon) block? Is there any way of knowing the level of finality of the provided head?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this considered safe, if this information is not yet in a (voted upon) block?

Yes, adding the SyncAggregate is only useful to give on-chain incentives to sync committee members. Gathering the data from gossip topics is identical.

Is there any way of knowing the level of finality of the provided head?

Not really, the main data points lightclients can rely on are:

  • participation of this SyncAggregate
  • if the block is finalized or not

- Aggregating messages from gossip topics `sync_committee_{subnet_id}`
Nodes may throttle emitting updates for a same header, but must emit the best update available eventually.
value: |
event: lightclient_head_update
data: {"message": {"sync_aggregate": {"sync_committee_bits": "0xfffffffffbfffffffffffffffffffffffffffffffffffffffffffefffffffffffffffffffffffeffffffffffeffffffffffffffffffffdffffffffffffffffff", "sync_committee_signature": "0x8a5e838431917331c461f38ec0f6c0b2f29c6cd2c9c9b7ed970b4700a8bc66bdb4b11ca129dd9c72fc1bbb1959c4a7cc0a0d8d107267f3cc364951747ad5ddd757555358e5910cec387179996a6e698d813df723ae02a825b6034af062913b91"}, "header": {"slot": "2642381", "proposer_index": "189806", "parent_root": "0xa00016e7462de60629c9b81b10d1f79c29ae43d81972a3c0e1e92801e3599d9d", "state_root": "0xaec6a955ff0ff9c0081828bad99342b3e60c4744a65e8c7ef1fd01bb1162b4ca", "body_root": "0xeed126eddef983e1270cf72bea05f62195590576a7472a6ae1b43a7bd1aa6f23"}}}
"400":
description: "The topics supplied could not be parsed"
content:
Expand All @@ -84,5 +97,4 @@ get:
code: 400
message: "Invalid topic: weather_forecast"
"500":
$ref: '../../beacon-node-oapi.yaml#/components/responses/InternalError'

$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
41 changes: 41 additions & 0 deletions apis/lightclient/committee_updates.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
get:
operationId: getCommitteeUpdates
summary: Get committee updates in an inclusive range of sync periods
description: |
Returns an array of best updates in the requested periods within the inclusive range `from` - `to`.
Best is defined by (in order of priority):
- Is finalized update + > 2/3 participations
- Highest participation (bit count)
- Oldest update
Tracks oldest instead of newest to reduce disk writes and the likelihood of re-orgs. Clients may choose
a different set of "best" criteria if necessary.
tags:
- Lightclient
parameters:
- name: from
in: query
required: true
description: "Sync period start of requested range"
schema:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/Uint64"
- name: to
in: query
required: true
description: "Sync period end of requested range (inclusive)"
schema:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/Uint64"
responses:
"200":
description: Success
content:
application/json:
schema:
title: GetCommitteeUpdatesResponse
type: object
properties:
data:
type: array
items:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/LightClientUpdate"
"500":
$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
22 changes: 22 additions & 0 deletions apis/lightclient/head_update.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
get:
operationId: getHeadUpdate
summary: Get latest best head update
description: |
Returns the latest best head update available. Clients should use the events endpoint and subscribe
to `lightclient_head_update` unless to get the very first head update after syncing, or if the events
endpoint is not supported by the server.
tags:
- Lightclient
responses:
"200":
description: Success
content:
application/json:
schema:
title: GetHeadUpdateResponse
type: object
properties:
data:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/LightclientHeadUpdate"
"500":
$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
55 changes: 55 additions & 0 deletions apis/lightclient/proof.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
get:
operationId: getProof
summary: Get multiproof of `paths` for `state_id`
description: |
Returns a multiproof of `paths` at the requested `stateId`.
The requested `stateId` may not be available. Regular nodes only keep recent states in memory.
tags:
- Lightclient
parameters:
- name: state_id
in: path
required: true
$ref: "../../beacon-node-oapi.yaml#/components/parameters/StateId"
- name: paths
in: query
required: true
description: Array of JSON serialized array of paths
schema:
type: array
items:
type: string
description: |
example: '''["balances", 0]'''

responses:
"200":
description: Success
content:
application/octet-stream:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should be a JSON response here as well. This will also give a location to explain the structure of the multiproof.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSON representation of this format is widely inefficient. Since JSON is the default I'll argue into only offering application/octet-stream to force consumers to use the efficient path

schema:
description: |
Serialized TreeOffset multiproof.
Format reference https://github.com/ethereum/consensus-specs/blob/dev/ssz/merkle-proofs.md#merkle-multiproofs
"400":
description: "Invalid state ID"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 400
message: "Invalid state ID: current"
"404":
description: "State not found"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 404
message: "State not found"
"500":
$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
49 changes: 49 additions & 0 deletions apis/lightclient/snapshot.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
get:
operationId: getSnapshot
summary: Fetch a snapshot with a proof to a trusted block root.
description: |
Fetch a snapshot with a proof to a trusted block root.
The trusted block root should be fetched with similar means to a weak subjectivity checkpoint.
Only block roots for checkpoints within the weak subjectivity period must be guaranteed to be available.
If the node started with a recent weak subjectivity checkpoint, the node may not backfill snapshots
for older checkpoints.
tags:
- Lightclient
parameters:
- name: block_root
in: path
required: true
$ref: "../../beacon-node-oapi.yaml#/components/schemas/Root"
responses:
"200":
description: Success
content:
application/json:
schema:
title: GetSnapshotResponse
type: object
properties:
data:
$ref: "../../beacon-node-oapi.yaml#/components/schemas/LightclientSnapshot"
"400":
description: "Invalid block root"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 400
message: "Invalid block root: head"
"404":
description: "Snapshot not found"
content:
application/json:
schema:
allOf:
- $ref: "../../beacon-node-oapi.yaml#/components/schemas/ErrorMessage"
- example:
code: 404
message: "Snapshot not found for block root"
"500":
$ref: "../../beacon-node-oapi.yaml#/components/responses/InternalError"
17 changes: 17 additions & 0 deletions beacon-node-oapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ tags:
description: Set of endpoints to debug chain and shouldn't be exposed publicly.
- name: Events
description: Set of endpoints to for event subscription.
- name: Lightclient
description: Endpoints to support server-based lightclients
- name: Node
description: Endpoints to query node related informations
- name: Validator
Expand Down Expand Up @@ -96,6 +98,15 @@ paths:
/eth/v1/debug/beacon/heads:
$ref: './apis/debug/heads.yaml'

/eth/v1/lightclient/committee_updates:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The API has historically defined committee as a beacon committee, and sync_committee as a sync committee, so I believe you want the latter here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if it's scoped in the lightclient namespace is there a risk of confusion?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I support using sync_committee for clarity here.

$ref: "./apis/lightclient/committee_updates.yaml"
/eth/v1/lightclient/head_update:
$ref: "./apis/lightclient/head_update.yaml"
/eth/v1/lightclient/proof/{state_id}:
$ref: "./apis/lightclient/proof.yaml"
/eth/v1/lightclient/snapshot/{block_root}:
$ref: "./apis/lightclient/snapshot.yaml"

/eth/v1/node/identity:
$ref: "./apis/node/identity.yaml"
/eth/v1/node/peers:
Expand Down Expand Up @@ -237,6 +248,12 @@ components:
$ref: './types/altair/sync_committee.yaml#/Altair/SyncCommitteeContribution'
Altair.SyncCommittee:
$ref: './types/altair/sync_committee.yaml#/Altair/SyncCommitteeByValidatorIndices'
LightClientUpdate:
$ref: './types/altair/lightclient.yaml#/Altair/LightClientUpdate'
LightclientHeadUpdate:
$ref: './types/altair/lightclient.yaml#/Altair/LightclientHeadUpdate'
LightclientSnapshot:
$ref: './types/altair/lightclient.yaml#/Altair/LightclientSnapshot'

parameters:
StateId:
Expand Down
58 changes: 58 additions & 0 deletions types/altair/lightclient.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
Altair:
LightClientUpdate:
type: object
properties:
header:
$ref: '../block.yaml#/BeaconBlockHeader'
nextSyncCommittee:
$ref: './sync_committee.yaml#/SyncCommittee'
nextSyncCommitteeBranch:
$ref: '#/Altair/SyncCommitteeBranch'
finalityHeader:
$ref: '../block.yaml#/BeaconBlockHeader'
finalityBranch:
$ref: '#/Altair/FinalityBranch'
syncCommitteeBits:
$ref: './sync_aggregate.yaml#/Altair/SyncCommitteeBits'
syncCommitteeSignature:
allOf:
- $ref: '../primitive.yaml#/Signature'
- description: 'Signature by the validator(s) over the block root of `slot`'
forkVersion:
$ref: '../primitive.yaml#/ForkVersion'

FinalityBranch:
type: array
description: "Branch to state.finalized_checkpoint.root"
items:
allOf:
- $ref: '../primitive.yaml#/Root'
minItems: 6
maxItems: 6

SyncCommitteeBranch:
type: array
description: "Branch to state.next_sync_committee or state.current_sync_committee"
items:
allOf:
- $ref: '../primitive.yaml#/Root'
minItems: 5
maxItems: 5

LightclientHeadUpdate:
type: object
properties:
header:
$ref: '../block.yaml#/BeaconBlockHeader'
syncAggregate:
$ref: './sync_aggregate.yaml#/SyncAggregate'

LightclientSnapshot:
type: object
properties:
header:
$ref: '../block.yaml#/BeaconBlockHeader'
currentSyncCommittee:
$ref: './sync_committee.yaml#/SyncCommittee'
currentSyncCommitteeBranch:
$ref: '#/Altair/SyncCommitteeBranch'
10 changes: 6 additions & 4 deletions types/altair/sync_aggregate.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,11 @@ Altair:
description: "The [`SyncAggregate`](https://github.com/ethereum/eth2.0-specs/blob/v1.1.0-alpha.1/specs/altair/beacon-chain.md#syncaggregate) object from the Eth2.0 Altair spec."
properties:
sync_committee_bits:
type: string
pattern: "^0x[a-fA-F0-9]+$"
example: "0x01"
description: "Aggregation bits of sync"
$ref: '#/Altair/SyncCommitteeBits'
sync_committee_signature:
$ref: '../primitive.yaml#/Signature'
SyncCommitteeBits:
allOf:
- description: 'A bit is set if a signature from the validator at the corresponding index in the subcommittee is present in the aggregate `signature`.'
- $ref: "../primitive.yaml#/Hex"
- example: "0x01"
6 changes: 1 addition & 5 deletions types/altair/sync_committee.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,7 @@ Altair:
- $ref: '../primitive.yaml#/Uint64'
- description: 'The index of the subcommittee that the contribution pertains to.'
aggregation_bits:
allOf:
- description: 'A bit is set if a signature from the validator at the corresponding index in the subcommittee is present in the aggregate `signature`.'
- $ref: "../primitive.yaml#/Hex"
- example: "0x01"
$ref: './sync_aggregate.yaml#/Altair/SyncCommitteeBits'
signature:
allOf:
- $ref: '../primitive.yaml#/Signature'
Expand All @@ -99,4 +96,3 @@ Altair:
allOf:
- $ref: '#/Altair/ValidatorsByIndex'
- description: 'Subcommittee slices of the current sync committee'