Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleaner bitfields #1019

Closed
wants to merge 7 commits into from
Closed

Cleaner bitfields #1019

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
8e62927
Cleaner bitfields
JustinDrake May 1, 2019
10d38a2
Update 0_beacon-chain.md
JustinDrake May 1, 2019
c26983e
Update 0_beacon-chain.md
JustinDrake May 1, 2019
df8c41d
Update 0_beacon-chain.md
JustinDrake May 1, 2019
c789a42
Update 0_beacon-chain.md
JustinDrake May 1, 2019
a012dcf
Update 0_beacon-chain.md
JustinDrake May 1, 2019
5d7f9e4
Update 0_beacon-chain.md
JustinDrake May 1, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 36 additions & 72 deletions specs/core/0_beacon-chain.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,6 @@
- [Gwei values](#gwei-values)
- [Initial values](#initial-values)
- [Time parameters](#time-parameters)
- [State list lengths](#state-list-lengths)
- [Reward and penalty quotients](#reward-and-penalty-quotients)
- [Max operations per block](#max-operations-per-block)
- [Signature domains](#signature-domains)
Expand Down Expand Up @@ -79,8 +78,6 @@
- [`bytes_to_int`](#bytes_to_int)
- [`get_total_balance`](#get_total_balance)
- [`get_domain`](#get_domain)
- [`get_bitfield_bit`](#get_bitfield_bit)
- [`verify_bitfield`](#verify_bitfield)
- [`convert_to_indexed`](#convert_to_indexed)
- [`verify_indexed_attestation`](#verify_indexed_attestation)
- [`is_double_vote`](#is_double_vote)
Expand Down Expand Up @@ -209,17 +206,10 @@ These configurations are updated for releases, but may be out of sync during `de
| `PERSISTENT_COMMITTEE_PERIOD` | `2**11` (= 2,048) | epochs | 9 days |
| `MAX_CROSSLINK_EPOCHS` | `2**6` (= 64) | epochs | ~7 hours |
| `MIN_EPOCHS_TO_INACTIVITY_PENALTY` | `2**2` (= 4) | epochs | 25.6 minutes |
| `EPOCHS_PER_HISTORICAL_VECTOR` | `2**13` (= 8,192) | epochs | ~36 days |

* `MAX_CROSSLINK_EPOCHS` should be a small constant times `SHARD_COUNT // SLOTS_PER_EPOCH`

### State list lengths

| Name | Value | Unit | Duration |
| - | - | :-: | :-: |
| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days |
| `LATEST_ACTIVE_INDEX_ROOTS_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days |
| `LATEST_SLASHED_EXIT_LENGTH` | `2**13` (= 8,192) | epochs | ~36 days |

### Reward and penalty quotients

| Name | Value |
Expand Down Expand Up @@ -329,7 +319,7 @@ The types are defined topologically to aid in facilitating an executable version
# Attestation data
'data': AttestationData,
# Custody bit
'custody_bit': 'bool',
'custody_bit': 'bit',
}
```

Expand Down Expand Up @@ -401,7 +391,7 @@ The types are defined topologically to aid in facilitating an executable version
```python
{
# Attester aggregation bitfield
'aggregation_bitfield': 'bytes',
'aggregation_bitfield': ['bit'],
# Attestation data
'data': AttestationData,
# Inclusion slot
Expand Down Expand Up @@ -453,11 +443,11 @@ The types are defined topologically to aid in facilitating an executable version
```python
{
# Attester aggregation bitfield
'aggregation_bitfield': 'bytes',
'aggregation_bitfield': ['bit'],
# Attestation data
'data': AttestationData,
# Custody bitfield
'custody_bitfield': 'bytes',
'custody_bitfield': ['bit'],
# BLS aggregate signature
'signature': 'bytes96',
}
Expand Down Expand Up @@ -556,7 +546,7 @@ The types are defined topologically to aid in facilitating an executable version
'balances': ['uint64'],

# Randomness and committees
'latest_randao_mixes': ['bytes32', LATEST_RANDAO_MIXES_LENGTH],
'latest_randao_mixes': ['bytes32', EPOCHS_PER_HISTORICAL_VECTOR],
'latest_start_shard': 'uint64',

# Finality
Expand All @@ -566,7 +556,7 @@ The types are defined topologically to aid in facilitating an executable version
'current_justified_epoch': 'uint64',
'previous_justified_root': 'bytes32',
'current_justified_root': 'bytes32',
'justification_bitfield': 'uint64',
'justification_bitfield': ['bit', EPOCHS_PER_HISTORICAL_VECTOR],
'finalized_epoch': 'uint64',
'finalized_root': 'bytes32',

Expand All @@ -575,8 +565,8 @@ The types are defined topologically to aid in facilitating an executable version
'previous_crosslinks': [Crosslink, SHARD_COUNT],
'latest_block_roots': ['bytes32', SLOTS_PER_HISTORICAL_ROOT],
'latest_state_roots': ['bytes32', SLOTS_PER_HISTORICAL_ROOT],
'latest_active_index_roots': ['bytes32', LATEST_ACTIVE_INDEX_ROOTS_LENGTH],
'latest_slashed_balances': ['uint64', LATEST_SLASHED_EXIT_LENGTH], # Balances slashed at every withdrawal period
'latest_active_index_roots': ['bytes32', EPOCHS_PER_HISTORICAL_VECTOR],
'latest_slashed_balances': ['uint64', EPOCHS_PER_HISTORICAL_VECTOR], # Balances slashed at every withdrawal period
'latest_block_header': BeaconBlockHeader, # `latest_block_header.state_root == ZERO_HASH` temporarily
'historical_roots': ['bytes32'],

Expand Down Expand Up @@ -888,8 +878,8 @@ def get_randao_mix(state: BeaconState,
"""
Return the randao mix at a recent ``epoch``.
"""
assert get_current_epoch(state) - LATEST_RANDAO_MIXES_LENGTH < epoch <= get_current_epoch(state)
return state.latest_randao_mixes[epoch % LATEST_RANDAO_MIXES_LENGTH]
assert get_current_epoch(state) - EPOCHS_PER_HISTORICAL_VECTOR < epoch <= get_current_epoch(state)
return state.latest_randao_mixes[epoch % EPOCHS_PER_HISTORICAL_VECTOR]
```

### `get_active_index_root`
Expand All @@ -900,8 +890,8 @@ def get_active_index_root(state: BeaconState,
"""
Return the index root at a recent ``epoch``.
"""
assert get_current_epoch(state) - LATEST_ACTIVE_INDEX_ROOTS_LENGTH + ACTIVATION_EXIT_DELAY < epoch <= get_current_epoch(state) + ACTIVATION_EXIT_DELAY
return state.latest_active_index_roots[epoch % LATEST_ACTIVE_INDEX_ROOTS_LENGTH]
assert get_current_epoch(state) - EPOCHS_PER_HISTORICAL_VECTOR + ACTIVATION_EXIT_DELAY < epoch <= get_current_epoch(state) + ACTIVATION_EXIT_DELAY
return state.latest_active_index_roots[epoch % EPOCHS_PER_HISTORICAL_VECTOR]
```

### `generate_seed`
Expand Down Expand Up @@ -967,8 +957,8 @@ def get_attesting_indices(state: BeaconState,
"""
crosslink_committees = get_crosslink_committees_at_slot(state, attestation_data.slot)
crosslink_committee = [committee for committee, shard in crosslink_committees if shard == attestation_data.shard][0]
assert verify_bitfield(bitfield, len(crosslink_committee))
return sorted([index for i, index in enumerate(crosslink_committee) if get_bitfield_bit(bitfield, i) == 0b1])
assert len(bitfield) == len(crosslink_committee)
return sorted([index for position, index in enumerate(crosslink_committee) if bitfield[position]])
```

### `int_to_bytes1`, `int_to_bytes2`, ...
Expand Down Expand Up @@ -1006,34 +996,6 @@ def get_domain(state: BeaconState,
return bytes_to_int(fork_version + int_to_bytes4(domain_type))
```

### `get_bitfield_bit`

```python
def get_bitfield_bit(bitfield: bytes, i: int) -> int:
"""
Extract the bit in ``bitfield`` at position ``i``.
"""
return (bitfield[i // 8] >> (i % 8)) % 2
```

### `verify_bitfield`

```python
def verify_bitfield(bitfield: bytes, committee_size: int) -> bool:
"""
Verify ``bitfield`` against the ``committee_size``.
"""
if len(bitfield) != (committee_size + 7) // 8:
return False

# Check `bitfield` is padded with zero bits only
for i in range(committee_size, len(bitfield) * 8):
if get_bitfield_bit(bitfield, i) == 0b1:
return False

return True
```

### `convert_to_indexed`

```python
Expand Down Expand Up @@ -1209,9 +1171,9 @@ def slash_validator(state: BeaconState, slashed_index: ValidatorIndex, whistlebl
current_epoch = get_current_epoch(state)
initiate_validator_exit(state, slashed_index)
state.validator_registry[slashed_index].slashed = True
state.validator_registry[slashed_index].withdrawable_epoch = current_epoch + LATEST_SLASHED_EXIT_LENGTH
state.validator_registry[slashed_index].withdrawable_epoch = current_epoch + EPOCHS_PER_HISTORICAL_VECTOR
slashed_balance = state.validator_registry[slashed_index].effective_balance
state.latest_slashed_balances[current_epoch % LATEST_SLASHED_EXIT_LENGTH] += slashed_balance
state.latest_slashed_balances[current_epoch % EPOCHS_PER_HISTORICAL_VECTOR] += slashed_balance

proposer_index = get_beacon_proposer_index(state)
if whistleblower_index is None:
Expand Down Expand Up @@ -1256,7 +1218,7 @@ def get_genesis_beacon_state(genesis_validator_deposits: List[Deposit],
validator.activation_epoch = GENESIS_EPOCH

genesis_active_index_root = hash_tree_root(get_active_validator_indices(state, GENESIS_EPOCH))
for index in range(LATEST_ACTIVE_INDEX_ROOTS_LENGTH):
for index in range(EPOCHS_PER_HISTORICAL_VECTOR):
state.latest_active_index_roots[index] = genesis_active_index_root

return state
Expand Down Expand Up @@ -1402,34 +1364,36 @@ def process_justification_and_finalization(state: BeaconState) -> None:
# Process justifications
state.previous_justified_epoch = state.current_justified_epoch
state.previous_justified_root = state.current_justified_root
state.justification_bitfield = (state.justification_bitfield << 1) % 2**64
previous_epoch_matching_target_balance = get_attesting_balance(state, get_matching_target_attestations(state, previous_epoch))
if previous_epoch_matching_target_balance * 3 >= get_total_active_balance(state) * 2:
state.current_justified_epoch = previous_epoch
state.current_justified_root = get_block_root(state, state.current_justified_epoch)
state.justification_bitfield |= (1 << 1)
state.justification_bitfield[previous_epoch % EPOCHS_PER_HISTORICAL_VECTOR] = True
current_epoch_matching_target_balance = get_attesting_balance(state, get_matching_target_attestations(state, current_epoch))
if current_epoch_matching_target_balance * 3 >= get_total_active_balance(state) * 2:
state.current_justified_epoch = current_epoch
state.current_justified_root = get_block_root(state, state.current_justified_epoch)
state.justification_bitfield |= (1 << 0)
state.justification_bitfield[current_epoch % EPOCHS_PER_HISTORICAL_VECTOR] = True

# Process finalizations
bitfield = state.justification_bitfield
bit_0 = state.justification_bitfield[(current_epoch - 0) % EPOCHS_PER_HISTORICAL_VECTOR]
bit_1 = state.justification_bitfield[(current_epoch - 1) % EPOCHS_PER_HISTORICAL_VECTOR]
bit_2 = state.justification_bitfield[(current_epoch - 2) % EPOCHS_PER_HISTORICAL_VECTOR]
bit_3 = state.justification_bitfield[(current_epoch - 3) % EPOCHS_PER_HISTORICAL_VECTOR]
# The 2nd/3rd/4th most recent epochs are justified, the 2nd using the 4th as source
if (bitfield >> 1) % 8 == 0b111 and old_previous_justified_epoch == current_epoch - 3:
if bit_1 and bit_2 and bit_3 and old_previous_justified_epoch == current_epoch - 3:
state.finalized_epoch = old_previous_justified_epoch
state.finalized_root = get_block_root(state, state.finalized_epoch)
# The 2nd/3rd most recent epochs are justified, the 2nd using the 3rd as source
if (bitfield >> 1) % 4 == 0b11 and old_previous_justified_epoch == current_epoch - 2:
if bit_1 and bit_2 and old_previous_justified_epoch == current_epoch - 2:
state.finalized_epoch = old_previous_justified_epoch
state.finalized_root = get_block_root(state, state.finalized_epoch)
# The 1st/2nd/3rd most recent epochs are justified, the 1st using the 3rd as source
if (bitfield >> 0) % 8 == 0b111 and old_current_justified_epoch == current_epoch - 2:
if bit_0 and bit_1 and bit_2 and old_current_justified_epoch == current_epoch - 2:
state.finalized_epoch = old_current_justified_epoch
state.finalized_root = get_block_root(state, state.finalized_epoch)
# The 1st/2nd most recent epochs are justified, the 1st using the 2nd as source
if (bitfield >> 0) % 4 == 0b11 and old_current_justified_epoch == current_epoch - 1:
if bit_0 and bit_1 and old_current_justified_epoch == current_epoch - 1:
state.finalized_epoch = old_current_justified_epoch
state.finalized_root = get_block_root(state, state.finalized_epoch)
```
Expand Down Expand Up @@ -1577,12 +1541,12 @@ def process_slashings(state: BeaconState) -> None:
total_balance = get_total_balance(state, active_validator_indices)

# Compute `total_penalties`
total_at_start = state.latest_slashed_balances[(current_epoch + 1) % LATEST_SLASHED_EXIT_LENGTH]
total_at_end = state.latest_slashed_balances[current_epoch % LATEST_SLASHED_EXIT_LENGTH]
total_at_start = state.latest_slashed_balances[(current_epoch + 1) % EPOCHS_PER_HISTORICAL_VECTOR]
total_at_end = state.latest_slashed_balances[current_epoch % EPOCHS_PER_HISTORICAL_VECTOR]
total_penalties = total_at_end - total_at_start

for index, validator in enumerate(state.validator_registry):
if validator.slashed and current_epoch == validator.withdrawable_epoch - LATEST_SLASHED_EXIT_LENGTH // 2:
if validator.slashed and current_epoch == validator.withdrawable_epoch - EPOCHS_PER_HISTORICAL_VECTOR // 2:
penalty = max(
validator.effective_balance * min(total_penalties * 3, total_balance) // total_balance,
validator.effective_balance // MIN_SLASHING_PENALTY_QUOTIENT
Expand Down Expand Up @@ -1610,16 +1574,16 @@ def process_final_updates(state: BeaconState) -> None:
# Update start shard
state.latest_start_shard = (state.latest_start_shard + get_shard_delta(state, current_epoch)) % SHARD_COUNT
# Set active index root
index_root_position = (next_epoch + ACTIVATION_EXIT_DELAY) % LATEST_ACTIVE_INDEX_ROOTS_LENGTH
index_root_position = (next_epoch + ACTIVATION_EXIT_DELAY) % EPOCHS_PER_HISTORICAL_VECTOR
state.latest_active_index_roots[index_root_position] = hash_tree_root(
get_active_validator_indices(state, next_epoch + ACTIVATION_EXIT_DELAY)
)
# Set total slashed balances
state.latest_slashed_balances[next_epoch % LATEST_SLASHED_EXIT_LENGTH] = (
state.latest_slashed_balances[current_epoch % LATEST_SLASHED_EXIT_LENGTH]
state.latest_slashed_balances[next_epoch % EPOCHS_PER_HISTORICAL_VECTOR] = (
state.latest_slashed_balances[current_epoch % EPOCHS_PER_HISTORICAL_VECTOR]
)
# Set randao mix
state.latest_randao_mixes[next_epoch % LATEST_RANDAO_MIXES_LENGTH] = get_randao_mix(state, current_epoch)
state.latest_randao_mixes[next_epoch % EPOCHS_PER_HISTORICAL_VECTOR] = get_randao_mix(state, current_epoch)
# Set historical root accumulator
if next_epoch % (SLOTS_PER_HISTORICAL_ROOT // SLOTS_PER_EPOCH) == 0:
historical_batch = HistoricalBatch(
Expand Down Expand Up @@ -1674,7 +1638,7 @@ def process_randao(state: BeaconState, block: BeaconBlock) -> None:
# Verify that the provided randao value is valid
assert bls_verify(proposer.pubkey, hash_tree_root(get_current_epoch(state)), block.body.randao_reveal, get_domain(state, DOMAIN_RANDAO))
# Mix it in
state.latest_randao_mixes[get_current_epoch(state) % LATEST_RANDAO_MIXES_LENGTH] = (
state.latest_randao_mixes[get_current_epoch(state) % EPOCHS_PER_HISTORICAL_VECTOR] = (
xor(get_randao_mix(state, get_current_epoch(state)),
hash(block.body.randao_reveal))
)
Expand Down