Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added withdrawals with accumulator #354

Closed
wants to merge 11 commits into from
Closed

Added withdrawals with accumulator #354

Changes from 2 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
bb45425
Added withdrawals with accumulator
vbuterin Dec 22, 2018
1d0f440
Added min validator withdrawal time
vbuterin Dec 22, 2018
52f3a05
Added the general case for how accumulator values map to partial Merk…
vbuterin Dec 22, 2018
4b90656
Fixed as per Justin's comments
vbuterin Dec 22, 2018
14bb5c5
Fix ToC
hwwhww Dec 23, 2018
beed3a1
`acc` -> `accumulator`
hwwhww Dec 23, 2018
200aa86
Some refactoring
hwwhww Dec 23, 2018
5a26883
Fixed error: accumulator + 1 -> position + 1
vbuterin Dec 23, 2018
78a5c7e
Bugfix
hwwhww Dec 23, 2018
34bdf4e
Added WITHDRAWN check as per #353
vbuterin Dec 23, 2018
95dfab0
Fixes
vbuterin Dec 23, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 75 additions & 19 deletions specs/core/0_beacon-chain.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,9 +168,11 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
| `MAX_CASPER_VOTES` | `2**10` (= 1,024) | votes |
| `LATEST_BLOCK_ROOTS_LENGTH` | `2**13` (= 8,192) | block roots |
| `LATEST_RANDAO_MIXES_LENGTH` | `2**13` (= 8,192) | randao mixes |
| `MAX_WITHDRAWALS_PER_EPOCH` | 4 | [validators](#dfn-validator) |
| `EMPTY_SIGNATURE` | `[bytes48(0), bytes48(0)]` | - |

* For the safety of crosslinks `TARGET_COMMITTEE_SIZE` exceeds [the recommended minimum committee size of 111](https://vitalik.ca/files/Ithaca201807_Sharding.pdf); with sufficient active validators (at least `EPOCH_LENGTH * TARGET_COMMITTEE_SIZE`), the shuffling algorithm ensures committee sizes at least `TARGET_COMMITTEE_SIZE`. (Unbiasable randomness with a Verifiable Delay Function (VDF) will improve committee robustness and lower the safe minimum committee size.)
* 4 withdrawals per cycle means that eg. if a 1/3 subset of 10 million ETH equivocate to cause a safety failure and all withdraw immediately, they will be able to leave in `3333333 / 32 / 4 = 26042` epochs, or 10 million seconds ~= 4 months.
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the throttling of withdrawals be for EXITED_WITH_PENALTY only? That is, anyone EXITED_WITHOUT_PENALTY can withdraw immediately?


### Deposit contract

Expand All @@ -197,6 +199,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
| `MIN_ATTESTATION_INCLUSION_DELAY` | `2**2` (= 4) | slots | 24 seconds |
| `EPOCH_LENGTH` | `2**6` (= 64) | slots | 6.4 minutes |
| `POW_RECEIPT_ROOT_VOTING_PERIOD` | `2**10` (= 1,024) | slots | ~1.7 hours |
| `MIN_VALIDATOR_WITHDRAWAL_TIME` | `2**14` (= 16,384) | slots | ~27 hours |
| `SHARD_PERSISTENT_COMMITTEE_CHANGE_PERIOD` | `2**17` (= 131,072) | slots | ~9 days |
| `COLLECTIVE_PENALTY_CALCULATION_PERIOD` | `2**20` (= 1,048,576) | slots | ~73 days |
| `ZERO_BALANCE_VALIDATOR_TTL` | `2**22` (= 4,194,304) | slots | ~291 days |
Expand All @@ -222,6 +225,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
| `ACTIVE_PENDING_EXIT` | `2` |
| `EXITED_WITHOUT_PENALTY` | `3` |
| `EXITED_WITH_PENALTY` | `4` |
| `WITHDRAWN` | `6` |

### Max operations per block

Expand Down Expand Up @@ -400,6 +404,19 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
}
```

#### `WithdrawalData`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename to Withdrawal?


```python
{
# Withdrawal credentials
'withdrawal_credentials': 'hash32',
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
# Slot created
'slot_created': 'uint64',
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
# ETH to withdraw
'value': 'uint64'
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename value => balance

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we do that, we should do the same for DepositData. Should we?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have three data structures for deposits: Deposit, DepositData and DepositInput. Definitely an opportunity for simplification/cleanup :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe leave that for a separate patch at some point down the line? 😄

}
```

### Beacon chain blocks

#### `BeaconBlock`
Expand Down Expand Up @@ -466,6 +483,9 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
'validator_registry_latest_change_slot': 'uint64',
'validator_registry_exit_count': 'uint64',
'validator_registry_delta_chain_tip': 'hash32', # For light clients to track deltas
'validator_withdrawal_accumulator': 'hash32',
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
'validator_registry_withdrawal_count': 'uint64'
vbuterin marked this conversation as resolved.
Show resolved Hide resolved


# Randomness and committees
'latest_randao_mixes': ['hash32'],
Expand All @@ -488,7 +508,7 @@ Unless otherwise indicated, code appearing in `this style` is to be interpreted
# Recent state
'latest_crosslinks': [CrosslinkRecord],
'latest_block_roots': ['hash32'], # Needed to process attestations, older to newer
'latest_penalized_exit_balances': ['uint64'], # Balances penalized at every withdrawal period
'total_penalty_history': ['uint64'], # Balances penalized at every withdrawal period
'latest_attestations': [PendingAttestationRecord],
'batched_block_roots': ['hash32'],

Expand Down Expand Up @@ -950,6 +970,20 @@ def merkle_root(values):
return o[1]
```

#### `update_merkle_accumulator`

```python
def update_merkle_accumulator(acc, position, value):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename acc => accumulator

vbuterin marked this conversation as resolved.
Show resolved Hide resolved
h = value
j = 0
while (acc+1) % 2**(j+1) == 0:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spacing: +1 => + 1

h = hash(acc[j] + h)
j += 1
return acc[:j] + [h] + acc[j+1:]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spacing

```

**INVARIANT**: if `len(values) = 2**k` for integer `k < len(acc)`, then if you apply `for i, v in enumerate(values): acc = update_merkle_accumulator(acc, i, v)`, at the end `acc[k] == merkle_root(values)`.

#### `get_attestation_participants`

```python
Expand Down Expand Up @@ -1165,6 +1199,8 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit],
validator_registry_latest_change_slot=INITIAL_SLOT_NUMBER,
validator_registry_exit_count=0,
validator_registry_delta_chain_tip=ZERO_HASH,
validator_withdrawal_accumulator=[ZERO_HASH for _ in range(64)], # Max capacity 2**64 to match size of uint64
validator_registry_withdrawal_count=0,

# Randomness and committees
latest_randao_mixes=[ZERO_HASH for _ in range(LATEST_RANDAO_MIXES_LENGTH)],
Expand All @@ -1185,7 +1221,7 @@ def get_initial_beacon_state(initial_validator_deposits: List[Deposit],
# Recent state
latest_crosslinks=[CrosslinkRecord(slot=INITIAL_SLOT_NUMBER, shard_block_root=ZERO_HASH) for _ in range(SHARD_COUNT)],
latest_block_roots=[ZERO_HASH for _ in range(LATEST_BLOCK_ROOTS_LENGTH)],
latest_penalized_exit_balances=[],
total_penalty_history=[],
latest_attestations=[],
batched_block_roots=[],

Expand Down Expand Up @@ -1332,6 +1368,8 @@ def update_validator_status(state: BeaconState,
initiate_validator_exit(state, index)
if new_status in [EXITED_WITH_PENALTY, EXITED_WITHOUT_PENALTY]:
exit_validator(state, index, new_status)
if new_status == WITHDRAWN:
withdraw_validator(state, index)
```

The following are helpers and should only be called via `update_validator_status`:
Expand Down Expand Up @@ -1390,7 +1428,7 @@ def exit_validator(state: BeaconState,
validator.latest_status_change_slot = state.slot

if new_status == EXITED_WITH_PENALTY:
state.latest_penalized_exit_balances[state.slot // COLLECTIVE_PENALTY_CALCULATION_PERIOD] += get_effective_balance(state, index)
state.total_penalty_history[state.slot // COLLECTIVE_PENALTY_CALCULATION_PERIOD] += get_effective_balance(state, index)

whistleblower_index = get_beacon_proposer_index(state, state.slot)
whistleblower_reward = get_effective_balance(state, index) // WHISTLEBLOWER_REWARD_QUOTIENT
Expand Down Expand Up @@ -1418,6 +1456,20 @@ def exit_validator(state: BeaconState,
break
```

```python
def withdraw_validator(state: BeaconState,
index: int):
new_accumulator_object = WithdrawalData(
withdrawal_credentials=validator.withdrawal_credentials,
slot_created=state.slot,
value=state.validator_balances[index]
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
)
Copy link
Contributor

@hwwhww hwwhww Dec 25, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the third parameter of update_merkle_accumulator should be a Hash32 object?

    new_accumulator_object = hash_tree_root(
        WithdrawalData(
            credentials=validator.withdrawal_credentials,
            slot=state.slot,
            value=state.validator_balances[index],
        )
    )

state.validator_withdrawal_accumulator = update_accumulator(
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
state.validator_withdrawal_accumulator, state.validator_registry_withdrawal_count, new_accumulator_object
)
state.validator_registry_withdrawal_count += 1
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
```

## Per-slot processing

Below are the processing steps that happen at every slot.
Expand Down Expand Up @@ -1758,22 +1810,26 @@ def update_validator_registry(state: BeaconState) -> None:
update_validator_status(state, index, new_status=EXITED_WITHOUT_PENALTY)


# Calculate the total ETH that has been penalized in the last ~2-3 withdrawal periods
period_index = current_slot // COLLECTIVE_PENALTY_CALCULATION_PERIOD
total_penalties = (
(latest_penalized_exit_balances[period_index]) +
(latest_penalized_exit_balances[period_index - 1] if period_index >= 1 else 0) +
(latest_penalized_exit_balances[period_index - 2] if period_index >= 2 else 0)
)

# Calculate penalties for slashed validators
def to_penalize(index):
return state.validator_registry[index].status == EXITED_WITH_PENALTY
validators_to_penalize = filter(to_penalize, range(len(validator_registry)))
for index in validators_to_penalize:
state.validator_balances[index] -= get_effective_balance(state, index) * min(total_penalties * 3, total_balance) // total_balance

return validator_registry, latest_penalized_exit_balances, validator_registry_delta_chain_tip
# Withdraw validators
all_indices = range(len(state.validator_registry))
def eligible(index):
return state.validator_registry[x].status in (EXITED_WITHOUT_PENALTY, EXITED_WITH_PENALTY) and \
state.slot >= state.validator_registry[x].latest_status_change_slot >= MIN_VALIDATOR_WITHDRAWAL_TIME
eligible_indices = filter(eligible, all_indices)
sorted_indices = sorted(eligible_indices, filter=lambda index: state.validator_registry[index].exit_count)
removed = 0
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
for index in sorted_indices:
validator = state.validator_registry[index]
if validator.status == EXITED_WITH_PENALTY:
# Calculate and apply penalties for slashed validators
start_period = max(validator.latest_status_change_slot // COLLECTIVE_PENALTY_CALCULATION_PERIOD - 1, 0)
current_period = current_slot // COLLECTIVE_PENALTY_CALCULATION_PERIOD
vbuterin marked this conversation as resolved.
Show resolved Hide resolved
total_penalties = state.total_penalty_history[current_period] - state.total_penalty_history[start_period]
state.validator_balances[index] -= get_effective_balance(state, index) * min(total_penalties * 3, total_balance) // total_balance
update_validator_status(state, index, new_status=WITHDRAWN)
removed += 1
if removed >= MAX_WITHDRAWALS_PER_EPOCH:
break
```

Also perform the following updates:
Expand Down