EIP-4881 deposit snapshot support

[michaelsproul]

A new endpoint for quickly synchronising the execution layer deposit cache during checkpoint sync is being standardised in ethereum/beacon-APIs#245.

This API will be available in Lighthouse in our next release v3.3.0, and we would like to support it for our checkpointz instance (mainnet.checkpoint.sigp.io) and other checkpointz servers running Lighthouse.

Unfortunately my own Go skills are too poor to help with implementation in checkpointz, but if someone had time to implement this I would be very grateful 😊

[samcm]

We can certainly have a look at that @michaelsproul! Thanks for the issue 👍

[pietjepuk2]

Taking a look at the finalization conditions in the EIP, am I correct in understanding that the checkpoint that should be served is not necessarily the latest finalized anymore?

If there are more deposits than can be handled in an epoch (or worse, in EPOCHS_PER_ETH1_VOTING_PERIOD), the above-linked "deposit finalization conditions" are not satisfied, and then the checkpoint that should be served should be an older one. Hypothetically (if there are a lot of deposits), a much older one.

@michaelsproul Does Lighthouse already take this into account when querying the eth/v1/beacon/deposit_snapshot endpoint (i.e. return an older one in the hypothetical edge-case)? Or is it just the deposit tree of the latest finalized checkpoint?

[ethDreamer]

Hello @pietjepuk2,

am I correct in understanding that the checkpoint that should be served is not necessarily the latest finalized anymore?

Not quite. The checkpoint that the DepositTreeSnapshot corresponds to is completely independent of the finalized checkpoint that is served to the node.

In most cases, the DepositTreeSnapshot checkpoint will be older than the latest finalized checkpoint. But that's totally fine. The deposit snapshot is just the starting point for syncing deposits, which without the snapshot is all the way back at the launch of the deposit contract. Nodes are perfectly free to start following the chain from the latest finalized checkpoint, but start syncing deposits from an older checkpoint :)

Lighthouse will automatically serve the latest snapshot that satisfies the deposit finalization conditions.

[pietjepuk2]

OK, that makes total sense! Got a bit confused with all the checkpoint/finalization nomenclature, thanks for elaborating :)

[ethDreamer]

@pietjepuk2
Glad I could help! :)

By the way I just timed it on my machine (16 core, 32 thread, 64 GB RAM). I already synced the execution engine (nethermind) and just started from an empty datadir for lighthouse.

Time it takes for lighthouse to go from nothing to having the deposit cache synced:

Without Snapshots	With Snapshots
13 minutes 12 seconds	8 seconds

In practice you still need to wait ~2 minutes for lighthouse to sync to the head of the chain before you can attest / propose, but still 2 minutes vs 13 minutes is a nice improvement :)

[samcm]

Haven't had a chance to implement this yet but just noting that it's still on my TODO list 🙏. I'll be making some updates to support Shapella in the next few weeks so will likely be bundled in around then 👍

[pietjepuk2]

I'm not entirely sure how relevant snapshot deposit support is on the long term with https://eips.ethereum.org/EIPS/eip-6110 . I don't know how many clients will support 4881 (or 6110 for that matter), but thought I'd say least mention it here.

[ethDreamer]

I'm not entirely sure how relevant snapshot deposit support is on the long term with https://eips.ethereum.org/EIPS/eip-6110 . I don't know how many clients will support 4881 (or 6110 for that matter), but thought I'd say least mention it here.

Teku, Lighthouse, & Nimbus support 4881 already and Prysm has someone in the process of implementing it. Not sure about lodestar. It is possible that 4881 is obsoleted in the future but this will take years as this is still in the planning process and there are a lot of other issues more pressing at the moment.

[ethDreamer]

Bumping this. Any progress @samcm ?

[samcm]

Having a look now. Am I correct in saying that there isn't any additional logic required in serving this endpoint? Checkpointz currently operates like this:

• Wait around until a majority of the configured upstreams agree on a new finalized checkpoint
• Pick one of those nodes and fetch the "bundle" for the checkpoint (beacon block and beacon state)
• If the block is on an epoch boundary start serving the "bundle" via the HTTP API.

Is the change as simple as including the deposit snapshot in the "bundle" and serving with the equivalent beacon block and beacon state?

[ethDreamer]

you could implement it this way, though for many (beacon block, beacon state) tuples, the snapshot will be the same. It changes less often than the finalized checkpoint.

[ethDreamer]

Mind if I bump this again @samcm ?

[samcm]

I've got the bulk of this working @ethDreamer, with the last remaining task being ssz support. Might just roll with JSON support for now - can revisit if needed down the track