Add snapshot booting proposal #6936

TristanDebrunner · 2019-11-13T23:01:51Z

Problem

A validator booting from a snapshot doesn't ensure that it is caught up and connected to the correct cluster before it starts voting.

Summary of Changes

Add a design proposal for confirming that a validator is caught up.

Towards #6727

book/src/proposals/booting-from-snapshot.md

garious · 2019-11-14T03:24:02Z

The PR problem description doesn't describe a problem.

TristanDebrunner · 2019-11-14T16:32:19Z

That better @garious?

TristanDebrunner · 2019-11-14T17:33:30Z

After re-writing the problem description, I think it may be better to split this into two parts:

Generate votes only if the proposed bank is not censoring the validator #6744 with the addition of rooting banks when some threshold roots them.
The parts of this proposal pertaining to ensuring that the validator is joining the correct cluster.

TristanDebrunner · 2019-11-14T17:43:16Z

After re-writing the problem description, I think it may be better to split this into two parts:

Though I'm not sure where restoring lockouts fits in, might be a third part.

book/src/proposals/booting-from-snapshot.md

TristanDebrunner · 2019-11-15T22:09:30Z

I just spoke offline with greg, and he suggested that all the vote txs applied to the working banks should be included in the snapshots, then a validator can verify the signatures and be fully confident in the snapshot just from the information it contains.

garious · 2019-11-15T22:43:55Z

My suggestion was that any vote on the slot height of the snapshot be included in the snapshot. After signature verification, the validator can be certain that a trusted validator calculated a bank hash that matches the hash of the accounts in the snapshot. Any use of validator stake to add weight to the notion of a "trusted" validator is completely optional. And the votes don't need to be on the ledger, since it's a slashable offense to vote on any other blockhash at that same slot height.

carllin · 2019-11-15T23:43:24Z

@garious, @TristanDebrunner, I think it would have to be a vote on the bank hash of the snapshot, otherwise, a somebody malicious could make up bank state in the snapshot.

A few questions:

The votes are sufficient to show the bank state in the snapshot is correct, but not that the network actually set this snapshot slot as the root. How should we detect that the network has picked a different root than the one in the snapshot?
Even with these votes included in the snapshot, we still have to wait for the canary tx procedure to complete to make sure the validator is caught up, correct? If we have to wait for the canary tx procedure, then it seems repetitive to also include the votes in the snapshot (in order for the canary to be detected, you'll need to see the votes from your trusted validiators) ?

garious · 2019-11-18T17:31:45Z

@carllin, regarding the canary procedure, I think you might be two steps ahead of me and solving problems I haven't worked up to yet. If we have a valid snapshot, I think the next problem is: validator is not caught up to tip and therefore cannot vote and collect rewards. So, of course, the validator would make repair requests. What problems will it encounter there?

carllin · 2019-11-18T23:12:19Z

@garious, we are hoping repairman + repair will be sufficient to catch the validator up, but this has not been validated on a network going full throttle

stale · 2020-03-30T21:13:20Z