Update getBlobSidecars endpoint path and add indices query parameter

[jimmygchen]

Description

There are a few updates with the getBlobSidecars endpoint in this beacon-APIs PR which was merged last week.

I think the main changes that haven't been implemented are:

• path changed from beacon/blobs to beacon/blob_sidecars
• new indices query parameter

[Gua00va]

I would like to work on this, if this is still open.

[Gua00va]

@jimmygchen where is this endpoint implemented?

[jimmygchen]

Hi @Gua00va that would be great, thank you!

Currently Deneb & EIP-4844 changes are being worked on the deneb-free-blobs branch - please use this as base branch for this issue.

The endpoint is implemented here:
https://github.com/sigp/lighthouse/blob/deneb-free-blobs/beacon_node/http_api/src/lib.rs#L1398

The spec change can be found here: ethereum/beacon-APIs#286 or latest master branch.

[jimmygchen]

@Gua00va If you're interested in testing this out on devnet-6, here's a docker-compose set up to run a node:

https://github.com/jimmygchen/eip4844-devnet

you can modify this line to use your fork (but you'll need the changes on realbigsean/devnet6 as it contains the devnet config.

https://github.com/jimmygchen/eip4844-devnet/blob/9d0dc3a980861642b7db3a393206abc1643575c0/lighthouse/Dockerfile.lighthouse#L5

and some testing instructions:

https://hackmd.io/@jimmygchen/H1XUtBIfn

I've just submitted a blob and able to retrieve it with (note the outdated path), the blob should be around for 18 days I think.

curl http://localhost:5052/eth/v1/beacon/blobs/13446

Up to you how you'd like to do it though, just thought it may help!

[Gua00va]

@jimmygchen What do u mean here by adding new indices query parameter" ? Do you mean adding a block_id parameter?

[jimmygchen]

@jimmygchen What do u mean here by adding new indices query parameter" ? Do you mean adding a block_id parameter?

So for each block there could be up to 6 blob sidecars, a user could request a subset of the sidecars using this indices query parameter, e.g. to request for first 2 sidecars, caller could add this query param ?indices=1,2 to the URL.

Description from Beacon API spec

Array of indices for blob sidecars to request for in the specified block. Returns all blob sidecars in the block if not specified.

Original comment from Kasey in the spec PR:

.. adding an indices parameter to specifically request a subset of the blob sidecars for a block? This would allow smaller requests sizes, and could be helpful for L2s using a CL node to resolve a single transaction hash pre-image. This would be a query string parameter, similar to the validator indices param for the state validators request, ie when absent the API would interpret the request as "give me everything".

[Gua00va]

@jimmygchen should I make a new custom object for the type or can it be handled any other way? Precisely, how should I approach passing an array as a query?

[jimmygchen]

@jimmygchen should I make a new custom object for the type or can it be handled any other way? Precisely, how should I approach passing an array as a query?

Given that you probably want something like a Option<Vec<u64>, it might be easiest to go with making a new struct for this and utilize the existing option_query_vec function, similar to ValidatorBalancesQuery?

lighthouse/common/eth2/src/types.rs

Lines 643 to 648 in dfcb336

	#[derive(Clone, Deserialize)]
	#[serde(deny_unknown_fields)]
	pub struct ValidatorBalancesQuery {
	#[serde(default, deserialize_with = "option_query_vec")]
	pub id: Option<Vec<ValidatorId>>,
	}

[realbigsean]

resolved in #4455