diff --git a/specs/core/0_beacon-chain.md b/specs/core/0_beacon-chain.md index af6f789760..ba96b04026 100644 --- a/specs/core/0_beacon-chain.md +++ b/specs/core/0_beacon-chain.md @@ -43,7 +43,7 @@ - [Beacon chain blocks](#beacon-chain-blocks) - [`BeaconBlock`](#beaconblock) - [`BeaconBlockBody`](#beaconblockbody) - - [`ProposalSignedData`](#proposalsigneddata) + - [`Proposal`](#proposal) - [Beacon chain state](#beacon-chain-state) - [`BeaconState`](#beaconstate) - [`Validator`](#validator) @@ -94,7 +94,6 @@ - [`bls_verify`](#bls_verify) - [`bls_verify_multiple`](#bls_verify_multiple) - [`bls_aggregate_pubkeys`](#bls_aggregate_pubkeys) - - [`validate_proof_of_possession`](#validate_proof_of_possession) - [`process_deposit`](#process_deposit) - [Routines for updating validator status](#routines-for-updating-validator-status) - [`activate_validator`](#activate_validator) @@ -117,7 +116,7 @@ - [Block roots](#block-roots) - [Per-block processing](#per-block-processing) - [Slot](#slot-1) - - [Proposer signature](#proposer-signature) + - [Block signature](#block-signature) - [RANDAO](#randao) - [Eth1 data](#eth1-data) - [Transactions](#transactions) @@ -299,14 +298,10 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git { # Proposer index 'proposer_index': 'uint64', - # First proposal data - 'proposal_data_1': ProposalSignedData, - # First proposal signature - 'proposal_signature_1': 'bytes96', - # Second proposal data - 'proposal_data_2': ProposalSignedData, - # Second proposal signature - 'proposal_signature_2': 'bytes96', + # First proposal + 'proposal_1': Proposal, + # Second proposal + 'proposal_2': Proposal, } ``` @@ -363,9 +358,9 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git 'slot': 'uint64', # Shard number 'shard': 'uint64', - # Hash of root of the signed beacon block + # Root of the signed beacon block 'beacon_block_root': 'bytes32', - # Hash of root of the ancestor at the epoch boundary + # Root of the ancestor at the epoch boundary 'epoch_boundary_root': 'bytes32', # Shard block's hash of root 'shard_block_root': 'bytes32', @@ -474,16 +469,17 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git ```python { - ## Header ## + # Header 'slot': 'uint64', 'parent_root': 'bytes32', 'state_root': 'bytes32', 'randao_reveal': 'bytes96', 'eth1_data': Eth1Data, - 'signature': 'bytes96', - ## Body ## + # Body 'body': BeaconBlockBody, + # Signature + 'signature': 'bytes96', } ``` @@ -500,7 +496,7 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git } ``` -#### `ProposalSignedData` +#### `Proposal` ```python { @@ -508,8 +504,10 @@ The following data structures are defined as [SimpleSerialize (SSZ)](https://git 'slot': 'uint64', # Shard number (`BEACON_CHAIN_SHARD_NUMBER` for beacon chain) 'shard': 'uint64', - # Block's hash of root + # Block root 'block_root': 'bytes32', + # Signature + 'signature': 'bytes96', } ``` @@ -670,6 +668,10 @@ Note: We aim to migrate to a S[T/N]ARK-friendly hash function in a future Ethere `def hash_tree_root(object: SSZSerializable) -> Bytes32` is a function for hashing objects into a single root utilizing a hash tree structure. `hash_tree_root` is defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#tree-hash). +### `signed_root` + +`def signed_root(object: SSZContainer) -> Bytes32` is a function defined in the [SimpleSerialize spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/simple-serialize.md#signed-roots) to compute signed messages. + ### `slot_to_epoch` ```python @@ -745,6 +747,7 @@ def get_permuted_index(index: int, list_size: int, seed: Bytes32) -> int: See the 'generalized domain' algorithm on page 3. """ assert index < list_size + assert list_size <= 2**40 for round in range(SHUFFLE_ROUND_COUNT): pivot = bytes_to_int(hash(seed + int_to_bytes1(round))[0:8]) % list_size @@ -795,23 +798,16 @@ def get_shuffling(seed: Bytes32, validators: List[Validator], epoch: Epoch) -> List[List[ValidatorIndex]] """ - Shuffle ``validators`` into crosslink committees seeded by ``seed`` and ``epoch``. - Return a list of ``committees_per_epoch`` committees where each - committee is itself a list of validator indices. + Shuffle active validators and split into crosslink committees. + Return a list of committees (each a list of validator indices). """ - + # Shuffle active validator indices active_validator_indices = get_active_validator_indices(validators, epoch) + length = len(active_validator_indices) + shuffled_indices = [active_validator_indices[get_permuted_index(i, length, seed)] for i in range(length)] - committees_per_epoch = get_epoch_committee_count(len(active_validator_indices)) - - # Shuffle - shuffled_active_validator_indices = [ - active_validator_indices[get_permuted_index(i, len(active_validator_indices), seed)] - for i in active_validator_indices - ] - - # Split the shuffled list into committees_per_epoch pieces - return split(shuffled_active_validator_indices, committees_per_epoch) + # Split the shuffled active validator indices + return split(shuffled_indices, get_epoch_committee_count(length)) ``` **Invariant**: if `get_shuffling(seed, validators, epoch)` returns some value `x` for some `epoch <= get_current_epoch(state) + ACTIVATION_EXIT_DELAY`, it should return the same value `x` for the same `seed` and `epoch` and possible future modifications of `validators` forever in phase 0, and until the ~1 year deletion delay in phase 2 and in the future. @@ -995,6 +991,7 @@ def get_beacon_proposer_index(state: BeaconState, def merkle_root(values: List[Bytes32]) -> Bytes32: """ Merkleize ``values`` (where ``len(values)`` is a power of two) and return the Merkle root. + Note that the leaves are not hashed. """ o = [0] * len(values) + values for i in range(len(values) - 1, 0, -1): @@ -1240,60 +1237,33 @@ def get_entry_exit_effect_epoch(epoch: Epoch) -> Epoch: `bls_aggregate_pubkeys` is a function for aggregating multiple BLS public keys into a single aggregate key, defined in the [BLS Signature spec](https://github.com/ethereum/eth2.0-specs/blob/master/specs/bls_signature.md#bls_aggregate_pubkeys). -### `validate_proof_of_possession` +### `process_deposit` + +Used to add a [validator](#dfn-validator) or top up an existing [validator](#dfn-validator)'s balance by some `deposit` amount: ```python -def validate_proof_of_possession(state: BeaconState, - pubkey: BLSPubkey, - proof_of_possession: BLSSignature, - withdrawal_credentials: Bytes32) -> bool: +def process_deposit(state: BeaconState, deposit: Deposit) -> None: """ - Verify the given ``proof_of_possession``. + Process a deposit from Ethereum 1.0. + Note that this function mutates ``state``. """ - proof_of_possession_data = DepositInput( - pubkey=pubkey, - withdrawal_credentials=withdrawal_credentials, - proof_of_possession=EMPTY_SIGNATURE, - ) + deposit_input = deposit.deposit_data.deposit_input - return bls_verify( - pubkey=pubkey, - message_hash=hash_tree_root(proof_of_possession_data), - signature=proof_of_possession, + assert bls_verify( + pubkey=deposit_input.pubkey, + message_hash=signed_root(deposit_input, "proof_of_possession"), + signature=deposit_input.proof_of_possession, domain=get_domain( state.fork, get_current_epoch(state), DOMAIN_DEPOSIT, ) ) -``` - -### `process_deposit` - -Used to add a [validator](#dfn-validator) or top up an existing [validator](#dfn-validator)'s balance by some `deposit` amount: - -```python -def process_deposit(state: BeaconState, - pubkey: BLSPubkey, - amount: Gwei, - proof_of_possession: BLSSignature, - withdrawal_credentials: Bytes32) -> None: - """ - Process a deposit from Ethereum 1.0. - Note that this function mutates ``state``. - """ - # Validate the given `proof_of_possession` - proof_is_valid = validate_proof_of_possession( - state, - pubkey, - proof_of_possession, - withdrawal_credentials, - ) - - if not proof_is_valid: - return validator_pubkeys = [v.pubkey for v in state.validator_registry] + pubkey = deposit_input.pubkey + amount = deposit.deposit_data.amount + withdrawal_credentials = deposit_input.withdrawal_credentials if pubkey not in validator_pubkeys: # Add new validator @@ -1520,13 +1490,7 @@ def get_genesis_beacon_state(genesis_validator_deposits: List[Deposit], # Process genesis deposits for deposit in genesis_validator_deposits: - process_deposit( - state=state, - pubkey=deposit.deposit_data.deposit_input.pubkey, - amount=deposit.deposit_data.amount, - proof_of_possession=deposit.deposit_data.deposit_input.proof_of_possession, - withdrawal_credentials=deposit.deposit_data.deposit_input.withdrawal_credentials, - ) + process_deposit(state, deposit) # Process genesis activations for validator_index, _ in enumerate(state.validator_registry): @@ -1652,21 +1616,20 @@ Below are the processing steps that happen at every `block`. * Verify that `block.slot == state.slot`. -#### Proposer signature +#### Block signature -* Let `block_without_signature_root` be the `hash_tree_root` of `block` where `block.signature` is set to `EMPTY_SIGNATURE`. -* Let `proposal_root = hash_tree_root(ProposalSignedData(state.slot, BEACON_CHAIN_SHARD_NUMBER, block_without_signature_root))`. -* Verify that `bls_verify(pubkey=state.validator_registry[get_beacon_proposer_index(state, state.slot)].pubkey, message_hash=proposal_root, signature=block.signature, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_PROPOSAL))`. +* Let `proposer = state.validator_registry[get_beacon_proposer_index(state, state.slot)]`. +* Let `proposal = Proposal(block.slot, BEACON_CHAIN_SHARD_NUMBER, signed_root(block, "signature"), block.signature)`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=signed_root(proposal, "signature"), signature=proposal.signature, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_PROPOSAL))`. #### RANDAO -* Let `proposer = state.validator_registry[get_beacon_proposer_index(state, state.slot)]`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=int_to_bytes32(get_current_epoch(state)), signature=block.randao_reveal, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_RANDAO))`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=hash_tree_root(get_current_epoch(state)), signature=block.randao_reveal, domain=get_domain(state.fork, get_current_epoch(state), DOMAIN_RANDAO))`. * Set `state.latest_randao_mixes[get_current_epoch(state) % LATEST_RANDAO_MIXES_LENGTH] = xor(get_randao_mix(state, get_current_epoch(state)), hash(block.randao_reveal))`. #### Eth1 data -* If there exists an `eth1_data_vote` in `states.eth1_data_votes` for which `eth1_data_vote.eth1_data == block.eth1_data` (there will be at most one), set `eth1_data_vote.vote_count += 1`. +* If there exists an `eth1_data_vote` in `state.eth1_data_votes` for which `eth1_data_vote.eth1_data == block.eth1_data` (there will be at most one), set `eth1_data_vote.vote_count += 1`. * Otherwise, append to `state.eth1_data_votes` a new `Eth1DataVote(eth1_data=block.eth1_data, vote_count=1)`. #### Transactions @@ -1678,12 +1641,12 @@ Verify that `len(block.body.proposer_slashings) <= MAX_PROPOSER_SLASHINGS`. For each `proposer_slashing` in `block.body.proposer_slashings`: * Let `proposer = state.validator_registry[proposer_slashing.proposer_index]`. -* Verify that `proposer_slashing.proposal_data_1.slot == proposer_slashing.proposal_data_2.slot`. -* Verify that `proposer_slashing.proposal_data_1.shard == proposer_slashing.proposal_data_2.shard`. -* Verify that `proposer_slashing.proposal_data_1.block_root != proposer_slashing.proposal_data_2.block_root`. +* Verify that `proposer_slashing.proposal_1.slot == proposer_slashing.proposal_2.slot`. +* Verify that `proposer_slashing.proposal_1.shard == proposer_slashing.proposal_2.shard`. +* Verify that `proposer_slashing.proposal_1.block_root != proposer_slashing.proposal_2.block_root`. * Verify that `proposer.slashed_epoch > get_current_epoch(state)`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=hash_tree_root(proposer_slashing.proposal_data_1), signature=proposer_slashing.proposal_signature_1, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_data_1.slot), DOMAIN_PROPOSAL))`. -* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=hash_tree_root(proposer_slashing.proposal_data_2), signature=proposer_slashing.proposal_signature_2, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_data_2.slot), DOMAIN_PROPOSAL))`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=signed_root(proposer_slashing.proposal_1, "signature"), signature=proposer_slashing.proposal_1.signature, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_1.slot), DOMAIN_PROPOSAL))`. +* Verify that `bls_verify(pubkey=proposer.pubkey, message_hash=signed_root(proposer_slashing.proposal_2, "signature"), signature=proposer_slashing.proposal_2.signature, domain=get_domain(state.fork, slot_to_epoch(proposer_slashing.proposal_2.slot), DOMAIN_PROPOSAL))`. * Run `slash_validator(state, proposer_slashing.proposer_index)`. ##### Attester slashings @@ -1777,13 +1740,7 @@ def verify_merkle_branch(leaf: Bytes32, branch: List[Bytes32], depth: int, index * Run the following: ```python -process_deposit( - state=state, - pubkey=deposit.deposit_data.deposit_input.pubkey, - amount=deposit.deposit_data.amount, - proof_of_possession=deposit.deposit_data.deposit_input.proof_of_possession, - withdrawal_credentials=deposit.deposit_data.deposit_input.withdrawal_credentials, -) +process_deposit(state, deposit) ``` * Set `state.deposit_index += 1`. @@ -1797,8 +1754,7 @@ For each `exit` in `block.body.voluntary_exits`: * Let `validator = state.validator_registry[exit.validator_index]`. * Verify that `validator.exit_epoch > get_entry_exit_effect_epoch(get_current_epoch(state))`. * Verify that `get_current_epoch(state) >= exit.epoch`. -* Let `exit_message = hash_tree_root(Exit(epoch=exit.epoch, validator_index=exit.validator_index, signature=EMPTY_SIGNATURE))`. -* Verify that `bls_verify(pubkey=validator.pubkey, message_hash=exit_message, signature=exit.signature, domain=get_domain(state.fork, exit.epoch, DOMAIN_EXIT))`. +* Verify that `bls_verify(pubkey=validator.pubkey, message_hash=signed_root(exit, "signature"), signature=exit.signature, domain=get_domain(state.fork, exit.epoch, DOMAIN_EXIT))`. * Run `initiate_validator_exit(state, exit.validator_index)`. ##### Transfers @@ -1815,8 +1771,7 @@ For each `transfer` in `block.body.transfers`: * Verify that `state.slot == transfer.slot`. * Verify that `get_current_epoch(state) >= state.validator_registry[transfer.from].withdrawable_epoch`. * Verify that `state.validator_registry[transfer.from].withdrawal_credentials == BLS_WITHDRAWAL_PREFIX_BYTE + hash(transfer.pubkey)[1:]`. -* Let `transfer_message = hash_tree_root(Transfer(from=transfer.from, to=transfer.to, amount=transfer.amount, fee=transfer.fee, slot=transfer.slot, signature=EMPTY_SIGNATURE))`. -* Verify that `bls_verify(pubkey=transfer.pubkey, message_hash=transfer_message, signature=transfer.signature, domain=get_domain(state.fork, slot_to_epoch(transfer.slot), DOMAIN_TRANSFER))`. +* Verify that `bls_verify(pubkey=transfer.pubkey, message_hash=signed_root(transfer, "signature"), signature=transfer.signature, domain=get_domain(state.fork, slot_to_epoch(transfer.slot), DOMAIN_TRANSFER))`. * Set `state.validator_balances[transfer.from] -= transfer.amount + transfer.fee`. * Set `state.validator_balances[transfer.to] += transfer.amount`. * Set `state.validator_balances[get_beacon_proposer_index(state, state.slot)] += transfer.fee`. @@ -2098,7 +2053,7 @@ def process_exit_queue(state: BeaconState) -> None: #### Final updates -* Set `state.latest_active_index_roots[(next_epoch + ACTIVATION_EXIT_DELAY) % LATEST_ACTIVE_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state, next_epoch + ACTIVATION_EXIT_DELAY))`. +* Set `state.latest_active_index_roots[(next_epoch + ACTIVATION_EXIT_DELAY) % LATEST_ACTIVE_INDEX_ROOTS_LENGTH] = hash_tree_root(get_active_validator_indices(state.validator_registry, next_epoch + ACTIVATION_EXIT_DELAY))`. * Set `state.latest_slashed_balances[(next_epoch) % LATEST_SLASHED_EXIT_LENGTH] = state.latest_slashed_balances[current_epoch % LATEST_SLASHED_EXIT_LENGTH]`. * Set `state.latest_randao_mixes[next_epoch % LATEST_RANDAO_MIXES_LENGTH] = get_randao_mix(state, current_epoch)`. * Remove any `attestation` in `state.latest_attestations` such that `slot_to_epoch(attestation.data.slot) < current_epoch`. diff --git a/specs/core/1_shard-data-chains.md b/specs/core/1_shard-data-chains.md index ba0dcb3f86..c3b781323f 100644 --- a/specs/core/1_shard-data-chains.md +++ b/specs/core/1_shard-data-chains.md @@ -1,9 +1,57 @@ # Ethereum 2.0 Phase 1 -- Shard Data Chains -###### tags: `spec`, `eth2.0`, `casper`, `sharding` - **NOTICE**: This document is a work-in-progress for researchers and implementers. It reflects recent spec changes and takes precedence over the [Python proof-of-concept implementation](https://github.com/ethereum/beacon_chain). +## Table of contents + + + +- [Ethereum 2.0 Phase 1 -- Shard Data Chains](#ethereum-20-phase-1----shard-data-chains) + - [Table of contents](#table-of-contents) + - [Introduction](#introduction) + - [Terminology](#terminology) + - [Constants](#constants) + - [Misc](#misc) + - [Time parameters](#time-parameters) + - [Max operations per block](#max-operations-per-block) + - [Signature domains](#signature-domains) + - [Helper functions](#helper-functions) + - [get_split_offset](#get_split_offset) + - [get_shuffled_committee](#get_shuffled_committee) + - [get_persistent_committee](#get_persistent_committee) + - [get_shard_proposer_index](#get_shard_proposer_index) + - [Data Structures](#data-structures) + - [Shard chain blocks](#shard-chain-blocks) + - [Shard block processing](#shard-block-processing) + - [Verifying shard block data](#verifying-shard-block-data) + - [Verifying a crosslink](#verifying-a-crosslink) + - [Shard block fork choice rule](#shard-block-fork-choice-rule) +- [Updates to the beacon chain](#updates-to-the-beacon-chain) + - [Data structures](#data-structures) + - [`Validator`](#validator) + - [`BeaconBlockBody`](#beaconblockbody) + - [`BranchChallenge`](#branchchallenge) + - [`BranchResponse`](#branchresponse) + - [`BranchChallengeRecord`](#branchchallengerecord) + - [`SubkeyReveal`](#subkeyreveal) + - [Helpers](#helpers) + - [`get_attestation_merkle_depth`](#get_attestation_merkle_depth) + - [`epoch_to_custody_period`](#epoch_to_custody_period) + - [`slot_to_custody_period`](#slot_to_custody_period) + - [`get_current_custody_period`](#get_current_custody_period) + - [`verify_custody_subkey_reveal`](#verify_custody_subkey_reveal) + - [`prepare_validator_for_withdrawal`](#prepare_validator_for_withdrawal) + - [`penalize_validator`](#penalize_validator) + - [Per-slot processing](#per-slot-processing) + - [Operations](#operations) + - [Branch challenges](#branch-challenges) + - [Branch responses](#branch-responses) + - [Subkey reveals](#subkey-reveals) + - [Per-epoch processing](#per-epoch-processing) + - [One-time phase 1 initiation transition](#one-time-phase-1-initiation-transition) + + + ### Introduction This document represents the specification for Phase 1 of Ethereum 2.0 -- Shard Data Chains. Phase 1 depends on the implementation of [Phase 0 -- The Beacon Chain](0_beacon-chain.md). @@ -16,19 +64,39 @@ Ethereum 2.0 consists of a central beacon chain along with `SHARD_COUNT` shard c Phase 1 depends upon all of the constants defined in [Phase 0](0_beacon-chain.md#constants) in addition to the following: -| Constant | Value | Unit | Approximation | -|-------------------------------|------------------|--------|---------------| -| `SHARD_CHUNK_SIZE` | 2**5 (= 32) | bytes | | -| `SHARD_BLOCK_SIZE` | 2**14 (= 16,384) | bytes | | -| `CROSSLINK_LOOKBACK` | 2**5 (= 32) | slots | | +#### Misc + +| Name | Value | Unit | +|-------------------------------|------------------|--------| +| `SHARD_CHUNK_SIZE` | 2**5 (= 32) | bytes | +| `SHARD_BLOCK_SIZE` | 2**14 (= 16,384) | bytes | +| `MINOR_REWARD_QUOTIENT` | 2**8 (= 256) | | + +#### Time parameters + +| Name | Value | Unit | Duration | +| - | - | :-: | :-: | +| `CROSSLINK_LOOKBACK` | 2**5 (= 32) | slots | 3.2 minutes | +| `MAX_BRANCH_CHALLENGE_DELAY` | 2**11 (= 2,048) | epochs | 9 days | +| `CUSTODY_PERIOD_LENGTH` | 2**11 (= 2,048) | epochs | 9 days | | `PERSISTENT_COMMITTEE_PERIOD` | 2**11 (= 2,048) | epochs | 9 days | +| `CHALLENGE_RESPONSE_DEADLINE` | 2**14 (= 16,384) | epochs | 73 days | -### Flags, domains, etc. +#### Max operations per block -| Constant | Value | +| Name | Value | +|-------------------------------|---------------| +| `MAX_BRANCH_CHALLENGES` | 2**2 (= 4) | +| `MAX_BRANCH_RESPONSES` | 2**4 (= 16) | +| `MAX_EARLY_SUBKEY_REVEALS` | 2**4 (= 16) | + +#### Signature domains + +| Name | Value | |------------------------|-----------------| -| `SHARD_PROPOSER_DOMAIN`| 129 | -| `SHARD_ATTESTER_DOMAIN`| 130 | +| `DOMAIN_SHARD_PROPOSER`| 129 | +| `DOMAIN_SHARD_ATTESTER`| 130 | +| `DOMAIN_CUSTODY_SUBKEY`| 131 | ## Helper functions @@ -159,9 +227,9 @@ To validate a block header on shard `shard_block.shard_id`, compute as follows: * Let `proposer_index = get_shard_proposer_index(state, shard_block.shard_id, shard_block.slot)`. * Verify that `proposer_index` is not `None`. * Let `msg` be the `shard_block` but with `shard_block.signature` set to `[0, 0]`. -* Verify that `bls_verify(pubkey=validators[proposer_index].pubkey, message_hash=hash(msg), signature=shard_block.signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), SHARD_PROPOSER_DOMAIN))` passes. +* Verify that `bls_verify(pubkey=validators[proposer_index].pubkey, message_hash=hash(msg), signature=shard_block.signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), DOMAIN_SHARD_PROPOSER))` passes. * Let `group_public_key = bls_aggregate_pubkeys([state.validators[index].pubkey for i, index in enumerate(persistent_committee) if get_bitfield_bit(shard_block.participation_bitfield, i) is True])`. -* Verify that `bls_verify(pubkey=group_public_key, message_hash=shard_block.parent_root, sig=shard_block.aggregate_signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), SHARD_ATTESTER_DOMAIN))` passes. +* Verify that `bls_verify(pubkey=group_public_key, message_hash=shard_block.parent_root, sig=shard_block.aggregate_signature, domain=get_domain(state, slot_to_epoch(shard_block.slot), DOMAIN_SHARD_ATTESTER))` passes. ### Verifying shard block data @@ -223,6 +291,306 @@ The `shard_chain_commitment` is only valid if it equals `compute_commitment(head The fork choice rule for any shard is LMD GHOST using the shard chain attestations of the persistent committee and the beacon chain attestations of the crosslink committee currently assigned to that shard, but instead of being rooted in the genesis it is rooted in the block referenced in the most recent accepted crosslink (ie. `state.crosslinks[shard].shard_block_root`). Only blocks whose `beacon_chain_ref` is the block in the main beacon chain at the specified `slot` should be considered (if the beacon chain skips a slot, then the block at that slot is considered to be the block in the beacon chain at the highest slot lower than a slot). +# Updates to the beacon chain + +## Data structures + +### `Validator` + +Add member values to the end of the `Validator` object: + +```python + 'open_branch_challenges': [BranchChallengeRecord], + 'next_subkey_to_reveal': 'uint64', + 'reveal_max_periods_late': 'uint64', +``` + +And the initializers: + +```python + 'open_branch_challenges': [], + 'next_subkey_to_reveal': get_current_custody_period(state), + 'reveal_max_periods_late': 0, +``` + +### `BeaconBlockBody` + +Add member values to the `BeaconBlockBody` structure: + +```python + 'branch_challenges': [BranchChallenge], + 'branch_responses': [BranchResponse], + 'subkey_reveals': [SubkeyReveal], +``` + +And initialize to the following: + +```python + 'branch_challenges': [], + 'branch_responses': [], + 'subkey_reveals': [], +``` + +### `BranchChallenge` + +Define a `BranchChallenge` as follows: + +```python +{ + 'responder_index': 'uint64', + 'data_index': 'uint64', + 'attestation': SlashableAttestation, +} +``` + +### `BranchResponse` + +Define a `BranchResponse` as follows: + +```python +{ + 'responder_index': 'uint64', + 'data': 'bytes32', + 'branch': ['bytes32'], + 'data_index': 'uint64', + 'root': 'bytes32', +} +``` + +### `BranchChallengeRecord` + +Define a `BranchChallengeRecord` as follows: + +```python +{ + 'challenger_index': 'uint64', + 'root': 'bytes32', + 'depth': 'uint64', + 'inclusion_epoch': 'uint64', + 'data_index': 'uint64', +} +``` + +### `SubkeyReveal` + +Define a `SubkeyReveal` as follows: + +```python +{ + 'validator_index': 'uint64', + 'period': 'uint64', + 'subkey': 'bytes96', + 'mask': 'bytes32', + 'revealer_index': 'uint64' +} +``` + +## Helpers + +### `get_attestation_merkle_depth` + +```python +def get_attestation_merkle_depth(attestation: Attestation) -> int: + start_epoch = attestation.data.latest_crosslink.epoch + end_epoch = slot_to_epoch(attestation.data.slot) + chunks_per_slot = SHARD_BLOCK_SIZE // 32 + chunks = (end_epoch - start_epoch) * EPOCH_LENGTH * chunks_per_slot + return log2(next_power_of_two(chunks)) +``` + +### `epoch_to_custody_period` + +```python +def epoch_to_custody_period(epoch: Epoch) -> int: + return epoch // CUSTODY_PERIOD_LENGTH +``` + +### `slot_to_custody_period` + +```python +def slot_to_custody_period(slot: Slot) -> int: + return epoch_to_custody_period(slot_to_epoch(slot)) +``` + +### `get_current_custody_period` + +```python +def get_current_custody_period(state: BeaconState) -> int: + return epoch_to_custody_period(get_current_epoch(state)) +``` + +### `verify_custody_subkey_reveal` + +```python +def verify_custody_subkey_reveal(pubkey: bytes48, + subkey: bytes96, + mask: bytes32, + mask_pubkey: bytes48, + period: int) -> bool: + # Legitimate reveal: checking that the provided value actually is the subkey + if mask == ZERO_HASH: + pubkeys=[pubkey] + message_hashes=[hash(int_to_bytes8(period))] + + # Punitive early reveal: checking that the provided value is a valid masked subkey + # (masking done to prevent "stealing the reward" from a whistleblower by block proposers) + # Secure under the aggregate extraction infeasibility assumption described on page 11-12 + # of https://crypto.stanford.edu/~dabo/pubs/papers/aggreg.pdf + else: + pubkeys=[pubkey, mask_pubkey] + message_hashes=[hash(int_to_bytes8(period)), mask] + + return bls_multi_verify( + pubkeys=pubkeys, + message_hashes=message_hashes, + signature=subkey, + domain=get_domain( + fork=state.fork, + epoch=period * CUSTODY_PERIOD_LENGTH, + domain_type=DOMAIN_CUSTODY_SUBKEY, + ) + ) +``` + +### `penalize_validator` + +Change the definition of `penalize_validator` as follows: + +```python +def penalize_validator(state: BeaconState, index: ValidatorIndex, whistleblower_index=None:ValidatorIndex) -> None: + """ + Penalize the validator of the given ``index``. + Note that this function mutates ``state``. + """ + exit_validator(state, index) + validator = state.validator_registry[index] + state.latest_penalized_balances[get_current_epoch(state) % LATEST_PENALIZED_EXIT_LENGTH] += get_effective_balance(state, index) + + block_proposer_index = get_beacon_proposer_index(state, state.slot) + whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT + if whistleblower_index is None: + state.validator_balances[block_proposer_index] += whistleblower_reward + else: + state.validator_balances[whistleblower_index] += ( + whistleblower_reward * INCLUDER_REWARD_QUOTIENT / (INCLUDER_REWARD_QUOTIENT + 1) + ) + state.validator_balances[block_proposer_index] += whistleblower_reward / (INCLUDER_REWARD_QUOTIENT + 1) + state.validator_balances[index] -= whistleblower_reward + validator.penalized_epoch = get_current_epoch(state) + validator.withdrawable_epoch = get_current_epoch(state) + LATEST_PENALIZED_EXIT_LENGTH +``` + +The only change is that this introduces the possibility of a penalization where the "whistleblower" that takes credit is NOT the block proposer. + +## Per-slot processing + +### Operations + +Add the following operations to the per-slot processing, in order the given below and _after_ all other operations (specifically, right after exits). + +#### Branch challenges + +Verify that `len(block.body.branch_challenges) <= MAX_BRANCH_CHALLENGES`. + +For each `challenge` in `block.body.branch_challenges`: + +* Verify that `slot_to_epoch(challenge.attestation.data.slot) >= get_current_epoch(state) - MAX_BRANCH_CHALLENGE_DELAY`. +* Verify that `state.validator_registry[responder_index].exit_epoch >= get_current_epoch(state) - MAX_BRANCH_CHALLENGE_DELAY`. +* Verify that `verify_slashable_attestation(state, challenge.attestation)` returns `True`. +* Verify that `challenge.responder_index` is in `challenge.attestation.validator_indices`. +* Let `depth = get_attestation_merkle_depth(challenge.attestation)`. Verify that `challenge.data_index < 2**depth`. +* Verify that there does not exist a `BranchChallengeRecord` in `state.validator_registry[challenge.responder_index].open_branch_challenges` with `root == challenge.attestation.data.shard_chain_commitment` and `data_index == data_index`. +* Append to `state.validator_registry[challenge.responder_index].open_branch_challenges` the object `BranchChallengeRecord(challenger_index=get_beacon_proposer_index(state, state.slot), root=challenge.attestation.data.shard_chain_commitment, depth=depth, inclusion_epoch=get_current_epoch(state), data_index=data_index)`. + +**Invariant**: the `open_branch_challenges` array will always stay sorted in order of `inclusion_epoch`. + +#### Branch responses + +Verify that `len(block.body.branch_responses) <= MAX_BRANCH_RESPONSES`. + +For each `response` in `block.body.branch_responses`: + +* Find the `BranchChallengeRecord` in `state.validator_registry[response.responder_index].open_branch_challenges` whose (`root`, `data_index`) match the (`root`, `data_index`) of the `response`. Verify that one such record exists (it is not possible for there to be more than one), call it `record`. +* Verify that `verify_merkle_branch(leaf=response.data, branch=response.branch, depth=record.depth, index=record.data_index, root=record.root)` is True. +* Verify that `get_current_epoch(state) >= record.inclusion_epoch + ENTRY_EXIT_DELAY`. +* Remove the `record` from `state.validator_registry[response.responder_index].open_branch_challenges` +* Determine the proposer `proposer_index = get_beacon_proposer_index(state, state.slot)` and set `state.validator_balances[proposer_index] += base_reward(state, index) // MINOR_REWARD_QUOTIENT`. + +#### Subkey reveals + +Verify that `len(block.body.early_subkey_reveals) <= MAX_EARLY_SUBKEY_REVEALS`. + +For each `reveal` in `block.body.early_subkey_reveals`: + +* Verify that `verify_custody_subkey_reveal(state.validator_registry[reveal.validator_index].pubkey, reveal.subkey, reveal.period, reveal.mask, state.validator_registry[reveal.revealer_index].pubkey)` returns `True`. +* Let `is_early_reveal = reveal.period > get_current_custody_period(state) or (reveal.period == get_current_custody_period(state) and state.validator_registry[reveal.validator_index].exit_epoch > get_current_epoch(state))` (ie. either the reveal is of a future period, or it's of the current period and the validator is still active) +* Verify that one of the following is true: + * (i) `is_early_reveal` is `True` + * (ii) `is_early_reveal` is `False` and `reveal.period == state.validator_registry[reveal.validator_index].next_subkey_to_reveal` (revealing a past subkey, or a current subkey for a validator that has exited) and `reveal.mask == ZERO_HASH` + +In case (i): + +* Verify that `state.validator_registry[reveal.validator_index].penalized_epoch > get_current_epoch(state). +* Run `penalize_validator(state, reveal.validator_index, reveal.revealer_index)`. +* Set `state.validator_balances[reveal.revealer_index] += base_reward(state, index) // MINOR_REWARD_QUOTIENT` + +In case (ii): + +* Determine the proposer `proposer_index = get_beacon_proposer_index(state, state.slot)` and set `state.validator_balances[proposer_index] += base_reward(state, index) // MINOR_REWARD_QUOTIENT`. +* Set `state.validator_registry[reveal.validator_index].next_subkey_to_reveal += 1` +* Set `state.validator_registry[reveal.validator_index].reveal_max_periods_late = max(state.validator_registry[reveal.validator_index].reveal_max_periods_late, get_current_period(state) - reveal.period)`. + +## Per-epoch processing + +Add the following loop immediately below the `process_ejections` loop: + +```python +def process_challenge_absences(state: BeaconState) -> None: + """ + Iterate through the validator registry + and penalize validators with balance that did not answer challenges. + """ + for index, validator in enumerate(state.validator_registry): + if len(validator.open_branch_challenges) > 0 and get_current_epoch(state) > validator.open_branch_challenges[0].inclusion_epoch + CHALLENGE_RESPONSE_DEADLINE: + penalize_validator(state, index, validator.open_branch_challenges[0].challenger_index) + if validator.challenge_data.challenger != VALIDATOR_NULL and get_current_epoch(state) > validator.challenge.deadline: + penalize_validator(state, index, validator.challenge_data.challenger_index) + if get_current_epoch(state) >= state.validator_registry[validator.now_challenging].withdrawal_epoch: + penalize_validator(state, index, validator.now_challenging) + penalize_validator(state, index, validator.challenge_data.challenger_index) +``` + +In `process_penalties_and_exits`, change the definition of `eligible` to the following (note that it is not a pure function because `state` is declared in the surrounding scope): + +```python +def eligible(index): + validator = state.validator_registry[index] + # Cannot exit if there are still open branch challenges + if len(validator.open_branch_challenges) > 0: + return False + # Cannot exit if you have not revealed all of your subkeys + elif validator.next_subkey_to_reveal <= epoch_to_custody_period(validator.exit_epoch): + return False + # Cannot exit if you already have + elif validator.withdrawable_epoch < FAR_FUTURE_EPOCH: + return False + # Return minimum time + else: + return current_epoch >= validator.exit_epoch + MIN_VALIDATOR_WITHDRAWAL_EPOCHS +``` + +## One-time phase 1 initiation transition + +Run the following on the fork block after per-slot processing and before per-block and per-epoch processing. + +For all `validator` in `ValidatorRegistry`, update it to the new format and fill the new member values with: + +```python + 'open_branch_challenges': [], + 'next_subkey_to_reveal': get_current_custody_period(state), + 'reveal_max_periods_late': 0, +``` + # Proof of custody interactive game ### Constants @@ -447,21 +815,3 @@ def process_branch_response(response: BranchResponse, penalize_validator(state, response.responder_index, challenge_data.challenger_index) state.validator_registry[challenge_data.challenger_index].now_challenging = VALIDATOR_NULL ``` - -Amend `process_challenge_absences` as follows: - -```python -def process_challenge_absences(state: BeaconState) -> None: - """ - Iterate through the validator registry - and penalize validators with balance that did not answer challenges. - """ - for index, validator in enumerate(state.validator_registry): - if len(validator.open_branch_challenges) > 0 and get_current_epoch(state) > validator.open_branch_challenges[0].inclusion_epoch + CHALLENGE_RESPONSE_DEADLINE: - penalize_validator(state, index, validator.open_branch_challenges[0].challenger_index) - if validator.challenge_data.challenger != VALIDATOR_NULL and get_current_epoch(state) > validator.challenge.deadline: - penalize_validator(state, index, validator.challenge_data.challenger_index) - if get_current_epoch(state) >= state.validator_registry[validator.now_challenging].withdrawal_epoch: - penalize_validator(state, index, validator.now_challenging) - penalize_validator(state, index, validator.challenge_data.challenger_index) -``` diff --git a/specs/simple-serialize.md b/specs/simple-serialize.md index c71654b677..6021619a60 100644 --- a/specs/simple-serialize.md +++ b/specs/simple-serialize.md @@ -24,11 +24,12 @@ deserializing objects and data types. - [bytesN](#bytesn-1) - [List/Vectors](#listvectors-1) - [Container](#container-1) - + [Tree Hash](#tree-hash) + + [Tree Hash](#tree-hash) - [`uint8`..`uint256`, `bool`, `bytes1`..`bytes32`](#uint8uint256-bool-bytes1bytes32) - [`uint264`..`uintN`, `bytes33`..`bytesN`](#uint264uintn-bytes33bytesn) - [List/Vectors](#listvectors-2) - [Container](#container-2) + + [Signed Roots](#signed-roots) * [Implementations](#implementations) ## About @@ -396,6 +397,14 @@ Recursively tree hash the values in the container in the same order as the field return merkle_hash([hash_tree_root_internal(getattr(x, field)) for field in value.fields]) ``` +### Signed roots + +Let `field_name` be a field name in an SSZ container `container`. We define `truncate(container, field_name)` to be the `container` with the fields from `field_name` onwards truncated away. That is, `truncate(container, field_name) = [getattr(container, field)) for field in value.fields[:i]]` where `i = value.fields.index(field_name)`. + +When `field_name` maps to a signature (e.g. a BLS12-381 signature of type `Bytes96`) the convention is that the corresponding signed message be `signed_root(container, field_name) = hash_tree_root(truncate(container, field_name))`. For example if `container = {"foo": sub_object_1, "bar": sub_object_2, "signature": bytes96, "baz": sub_object_3}` then `signed_root(container, "signature") = merkle_hash([hash_tree_root(sub_object_1), hash_tree_root(sub_object_2)])`. + +Note that this convention means that fields after the signature are _not_ signed over. If there are multiple signatures in `container` then those are expected to be signing over the fields in the order specified. If multiple signatures of the same value are expected the convention is that the signature field be an array of signatures. + ## Implementations | Language | Implementation | Description | diff --git a/specs/validator/0_beacon-chain-validator.md b/specs/validator/0_beacon-chain-validator.md index 2211e2e419..6fdeafb49c 100644 --- a/specs/validator/0_beacon-chain-validator.md +++ b/specs/validator/0_beacon-chain-validator.md @@ -186,7 +186,7 @@ epoch_signature = bls_sign( * Let `block_hash` be the block hash of the `ETH1_FOLLOW_DISTANCE`'th ancestor of the head of the canonical eth1.0 chain. * Let `deposit_root` be the deposit root of the eth1.0 deposit contract in the post-state of the block referenced by `block_hash` * If `D` is nonempty: - * Let `best_vote` be the member of `D` that has the highest `vote.eth1_data.vote_count`, breaking ties by favoring block hashes with higher associated block height. + * Let `best_vote` be the member of `D` that has the highest `vote.vote_count`, breaking ties by favoring block hashes with higher associated block height. * Let `block_hash = best_vote.eth1_data.block_hash`. * Let `deposit_root = best_vote.eth1_data.deposit_root`. * Set `block.eth1_data = Eth1Data(deposit_root=deposit_root, block_hash=block_hash)`.