Regular Sync Inserts PostChainStart Deposits Again Into the State

[rauljordan]

This is part of #1586. We have a major bug in which after restarting a node, regular sync processes previous post-chainstart deposits again into the state, leading validators to double their money. We should not allow this to be the case.

[prestonvanloon]

This happens consistently under the following conditions:

• Some post chainstart validators have their deposits included in blocks, but they are not yet activated
• In a multi-node scenario (possibly single node as well)

[prestonvanloon]

i think there are three (or more) problems here:

• pending deposits are not properly pruned after sync startup
• pending deposits are not verified not to have been already included in the blockchain when requesting via RPC
• pending deposits are not verified not to have been already included in the blockchain when processing a block with new deposits

[prestonvanloon]

I recommend solving from the bottom up. So in this order:

• reject blocks that have duplicate deposits (deposits included in a block that have already been included)
• prevent serving duplicate deposits via RPC
• properly prune pending deposits on sync

[prestonvanloon]

We should include the spec change from PR ethereum/consensus-specs#594 & PR ethereum/consensus-specs#709 to solve the missing precondition for including a new validator deposit

This change has a deposit index that must be equal to the state.deposit_index.

https://github.com/ethereum/eth2.0-specs/blob/dev/specs/core/0_beacon-chain.md#process_deposit

def process_deposit(state: BeaconState, deposit: Deposit) -> None:
    """
    Process a deposit from Ethereum 1.0.
    Note that this function mutates ``state``.
    """
    # Deposits must be processed in order
    assert deposit.index == state.deposit_index

    ...

[prestonvanloon]

Pending deposits can be dropped on the following condition:

• deposit.index < finalized_state.deposit_index

Pending deposits can only be served via RPC on the following condition:

• deposit.index >= current_state.deposit_index

New blocks can be considered valid with deposits only if deposits are processed in order where:

• deposit.index == state.deposit_index