Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history 
commit 8e2f7c5
Author: Michael Sproul <[email protected]>
Date:   Thu Feb 8 09:20:28 2024 +1100

    Delete unused epoch processing code (sigp#5170)

    * Delete unused epoch processing code

    * Compare total deltas

    * Remove unnecessary apply_pending

    * cargo fmt

    * Remove newline

commit ae6620e
Author: Michael Sproul <[email protected]>
Date:   Fri Feb 2 17:47:14 2024 +1100

    Delete `lighthouse db diff` (sigp#5171)

    * Delete `lighthouse db diff`

    * Fix help text

commit 25bcd2a
Author: Michael Sproul <[email protected]>
Date:   Wed Jan 31 10:13:18 2024 +1100

    Tree states v4.6.222-exp (sigp#5147)

commit 68a9a2e
Merge: 8e68926 0f345c7
Author: Michael Sproul <[email protected]>
Date:   Tue Jan 30 17:13:57 2024 +1100

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit 8e68926
Author: Michael Sproul <[email protected]>
Date:   Tue Jan 30 17:08:43 2024 +1100

    `fsync` during backfill to prevent DB corruption (sigp#5144)

commit 7862c71
Author: Michael Sproul <[email protected]>
Date:   Tue Jan 30 15:56:48 2024 +1100

    Fix tree-states sub-epoch diffs (sigp#5097)

commit 11461d8
Author: Michael Sproul <[email protected]>
Date:   Tue Jan 30 09:59:25 2024 +1100

    Fix new CLI tests for tree-states (sigp#5132)

commit 262e5f2
Merge: 6262be7 1be5253
Author: Michael Sproul <[email protected]>
Date:   Thu Jan 25 15:10:19 2024 +1100

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit 6262be7
Author: Michael Sproul <[email protected]>
Date:   Sun Jan 14 09:41:42 2024 +1100

    Don't error on inactive indices in att. rewards

commit 9cd9243
Author: Michael Sproul <[email protected]>
Date:   Fri Jan 12 10:50:34 2024 +1100

    Tree states release v4.6.111-exp

commit 664a778
Author: Michael Sproul <[email protected]>
Date:   Thu Jan 11 17:13:43 2024 +1100

    Add cache for parallel HTTP requests (sigp#4879)

commit 8db17da
Merge: c8dc082 2e8e160
Author: Michael Sproul <[email protected]>
Date:   Thu Jan 11 13:15:06 2024 +1100

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit c8dc082
Merge: 4741bf1 a3a3703
Author: Michael Sproul <[email protected]>
Date:   Thu Dec 14 09:59:43 2023 +1100

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit 4741bf1
Author: Michael Sproul <[email protected]>
Date:   Tue Dec 5 09:23:24 2023 +1100

    Remove stray println

commit cefe9fd
Author: Michael Sproul <[email protected]>
Date:   Mon Dec 4 17:15:25 2023 +1100

    Restore crash safety for database pruning (sigp#4975)

    * Add some DB sanity checks

    * Restore crash safety for database pruning

commit 66d30bc
Merge: e880d9d 44aaf13
Author: Michael Sproul <[email protected]>
Date:   Fri Dec 1 12:02:21 2023 +1100

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit e880d9d
Author: Michael Sproul <[email protected]>
Date:   Fri Dec 1 11:06:27 2023 +1100

    Fix cache initialization in block rewards API (sigp#4960)

commit 9cdc4b9
Merge: d36ebba 051c3e8
Author: Michael Sproul <[email protected]>
Date:   Sun Nov 12 22:19:07 2023 +0300

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit d36ebba
Author: Michael Sproul <[email protected]>
Date:   Thu Oct 26 18:09:28 2023 +1100

    Handle out-of-order forks in epoch cache (sigp#4881)

commit 1aca484
Author: Michael Sproul <[email protected]>
Date:   Fri Oct 20 12:31:41 2023 +1100

    Tree states release v4.5.444-exp

    - Update xdelta3 to remove dodgy build steps
    - Fix asset paths for draft release

commit 6b4f154
Author: Michael Sproul <[email protected]>
Date:   Thu Oct 19 15:57:11 2023 +1100

    Tree states release v4.5.333-exp

commit 0cb8fdf
Author: Michael Sproul <[email protected]>
Date:   Thu Oct 19 14:59:29 2023 +1100

    Various small tree-states fixes (sigp#4861)

    * Fix block backfill with genesis skip slots

    * Fix freezer upper limit

    * Fix: write post state in lcli skip-slots (sigp#4843)

    * Added CARGO_USE_GIT_CLI to the Dockerfile (sigp#4828)

    * chore: replace deprecated hub with gh for releases (sigp#4839)

    * Put schema version back to 24 (ignore Deneb)

    * Minimise diff

    ---------

    Co-authored-by: Joe Clapis <[email protected]>
    Co-authored-by: Dustin Brickwood <[email protected]>

commit 72d8c38
Merge: bb6675e 98cac2b
Author: Michael Sproul <[email protected]>
Date:   Thu Oct 19 12:07:35 2023 +1100

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit bb6675e
Author: Michael Sproul <[email protected]>
Date:   Fri Oct 13 16:45:56 2023 +1100

    Clean up progressive balance slashings further (sigp#4834)

    * Clean up progressive balance slashings further

    * Fix Rayon deadlock in test utils

    * Fix cargo-fmt

commit b121e69
Author: Michael Sproul <[email protected]>
Date:   Fri Oct 13 16:45:35 2023 +1100

    Fix cache logic for epoch boundary skips (sigp#4833)

commit b77de69
Author: Michael Sproul <[email protected]>
Date:   Wed Oct 11 14:37:46 2023 +1100

    Re-enable ARM builds

commit dfa3b43
Author: Michael Sproul <[email protected]>
Date:   Wed Oct 11 14:03:11 2023 +1100

    Fix Clippy for 1.73

commit 6ae4c22
Author: Michael Sproul <[email protected]>
Date:   Wed Oct 11 11:57:39 2023 +1100

    Fix merge snafu in tests

commit e63d02e
Merge: d4f87ef 4ad7e15
Author: Michael Sproul <[email protected]>
Date:   Wed Oct 11 11:52:39 2023 +1100

    Merge remote-tracking branch 'origin/deneb-free-blobs' into tree-states-deneb

commit d4f87ef
Author: Michael Sproul <[email protected]>
Date:   Wed Oct 11 11:18:22 2023 +1100

    Fix three consensus bugs!

commit ca1abfe
Author: Michael Sproul <[email protected]>
Date:   Wed Oct 11 10:43:49 2023 +1100

    Support iterables in compare_fields

commit e2a60a6
Merge: 9446fc8 203ac65
Author: Michael Sproul <[email protected]>
Date:   Fri Oct 6 11:11:36 2023 +1100

    Merge remote-tracking branch 'origin/deneb-free-blobs' into tree-states-deneb

commit 9446fc8
Author: Michael Sproul <[email protected]>
Date:   Tue Oct 3 16:07:25 2023 +1100

    Fix semantic Deneb <> tree-states conflicts

commit 109c4a5
Merge: f1f76f2 57edc0f
Author: Michael Sproul <[email protected]>
Date:   Fri Sep 29 16:34:29 2023 +1000

    Merge remote-tracking branch 'origin/deneb-free-blobs' into tree-states

commit f1f76f2
Author: Michael Sproul <[email protected]>
Date:   Tue Sep 26 12:23:28 2023 +1000

    Tree states release v4.5.222-exp (sigp#4782)

commit cae73a4
Merge: 364074d 441fc16
Author: Michael Sproul <[email protected]>
Date:   Tue Sep 26 11:21:44 2023 +1000

    Merge tag 'v4.5.0' into tree-states

    v4.5.0

commit 364074d
Author: Michael Sproul <[email protected]>
Date:   Fri Sep 22 15:52:23 2023 +1000

    Tree states release v4.5.111-exp (sigp#4769)

commit d24875f
Merge: cd23c89 69c39ad
Author: Michael Sproul <[email protected]>
Date:   Fri Sep 22 15:11:42 2023 +1000

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit cd23c89
Author: Michael Sproul <[email protected]>
Date:   Fri Sep 22 14:49:15 2023 +1000

    Improve state cache eviction and reduce mem usage (sigp#4762)

    * Improve state cache eviction and reduce mem usage

    * Fix epochs_per_state_diff tests

commit 1b4bc88
Author: Michael Sproul <[email protected]>
Date:   Thu Sep 14 10:07:26 2023 +1000

    Release v4.4.111-exp (sigp#4729)

commit 5cb2ed3
Author: Michael Sproul <[email protected]>
Date:   Wed Sep 13 14:43:02 2023 +1000

    Restore custom image for Cross

commit f7c6b7d
Author: Michael Sproul <[email protected]>
Date:   Wed Sep 13 14:00:28 2023 +1000

    Bump schema version to v24

commit 68f80cc
Author: Michael Sproul <[email protected]>
Date:   Wed Sep 13 13:56:53 2023 +1000

    Change default epochs-per-state-diff to 16

    This should make replaying diffs during non-finality a bit quicker.

commit 838e104
Author: Michael Sproul <[email protected]>
Date:   Wed Sep 13 13:54:03 2023 +1000

    Attempt to fix flaky test

commit d961d2c
Author: Michael Sproul <[email protected]>
Date:   Wed Sep 13 12:51:20 2023 +1000

    Disable ARM docker builds

commit b8e04ce
Merge: 1e4ee7a 35f47f4
Author: Michael Sproul <[email protected]>
Date:   Wed Sep 13 11:25:18 2023 +1000

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit 1e4ee7a
Author: Jimmy Chen <[email protected]>
Date:   Mon Sep 11 10:19:40 2023 +1000

    Tree states to support per-slot state diffs (sigp#4652)

    * Support per slot state diffs

    * Store HierarchyConfig on disk. Support storing hdiffs at per slot level.

    * Revert HierachyConfig change for testing.

    * Add validity check for the hierarchy config when opening the DB.

    * Update HDiff tests.

    * Fix `get_cold_state` panic when the diff for the slot isn't stored.

    * Use slots instead of epochs for storing snapshots in freezer DB.

    * Add snapshot buffer to `diff_buffer_cache` instead of loading it from db every time.

    * Add `hierarchy-exponents` cli flag to beacon node.

    * Add test for `StorageStrategy::ReplayFrom` and ignore a flaky test.

    * Drop hierarchy_config in tests for more frequent snapshot and fix an issue where hdiff wasn't stored unless it's a epoch boundary slot.

commit e373e9a
Author: Michael Sproul <[email protected]>
Date:   Wed Aug 9 19:42:14 2023 +1000

    Fix genesis state storage for genesis sync (sigp#4589)

commit bba1526
Author: Michael Sproul <[email protected]>
Date:   Tue Aug 8 13:57:05 2023 +1000

    Fix deadlock in finalization migration (sigp#4576)

commit 18e64e6
Author: Michael Sproul <[email protected]>
Date:   Tue Aug 8 11:25:26 2023 +1000

    Optimise mutations in single-pass epoch processing (sigp#4573)

    * Optimise mutations in single-pass epoch processing

    * Use safer Cow::make_mut

    * Update to upstream milhouse

commit 8423e9f
Merge: 5d2063d fc7f1ba
Author: Michael Sproul <[email protected]>
Date:   Wed Jul 19 11:23:52 2023 +1000

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit 5d2063d
Author: Michael Sproul <[email protected]>
Date:   Tue Jul 18 16:59:55 2023 +1000

    Single-pass epoch processing (sigp#4483)

commit 079cd67
Merge: 0291998 835fa70
Author: Michael Sproul <[email protected]>
Date:   Mon Jul 3 15:03:54 2023 +1000

    Merge remote-tracking branch 'origin/tree-states' into tree-states

commit 0291998
Merge: b414c32 46be05f
Author: Michael Sproul <[email protected]>
Date:   Mon Jul 3 15:01:21 2023 +1000

    Merge remote-tracking branch 'origin/unstable' into tree-states

commit b414c32
Author: Michael Sproul <[email protected]>
Date:   Mon Jul 3 12:03:14 2023 +1000

    Implement activation queue cache

commit 835fa70
Author: Michael Sproul <[email protected]>
Date:   Sat Jul 1 09:53:06 2023 +1000

    Fix EpochCache handling in ef-tests (sigp#4454)

commit f631b51
Author: Michael Sproul <[email protected]>
Date:   Fri Jun 30 22:57:36 2023 +1000

    Fix EpochCache handling in ef-tests

commit 2df714e
Author: Jimmy Chen <[email protected]>
Date:   Fri Jun 30 11:25:51 2023 +1000

    Tree states optimization using `EpochCache` (sigp#4429)

    * Relocate epoch cache to BeaconState

    * Optimize per block processing by pulling previous epoch & current epoch calculation up.

    * Revert `get_cow` change (no performance improvement)

    * Initialize `EpochCache` in epoch processing and load it from state when getting base rewards.

    * Initialize `EpochCache` at start of block processing if required.

    * Initialize `EpochCache` in `transition_blocks` if `exclude_cache_builds` is enabled

    * Fix epoch cache initialization logic

    * Remove FIXME comment.

    * Cache previous & current epochs in `consensus_context.rs`.

    * Move `get_base_rewards` from `ConsensusContext` to `BeaconState`.

    * Update Milhouse version

commit 160bbde
Author: Michael Sproul <[email protected]>
Date:   Fri Jun 30 10:29:34 2023 +1000

    Fix db-migration-period default (sigp#4441)

    * Fix db-migration-period default

    * Fix version regex

commit 6954de6
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 27 17:34:41 2023 +1000

    Tree states alpha release v4.2.990-exp

commit 8dc374e
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 27 17:33:25 2023 +1000

    Temporarily disable ARM builds

commit af5fb20
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 27 15:10:52 2023 +1000

    Tree states alpha release v4.2.99-exp

commit 56c7a52
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 27 16:52:24 2023 +1000

    Install Clang 5 in Cross builder image

commit 7c2eb96
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 27 15:06:43 2023 +1000

    Set epochs per migration to 1

    Workaround for sigp#4236

commit 88e30b6
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 27 15:04:39 2023 +1000

    Fix failing tests (sigp#4423)

    * Get tests passing

    * Get benchmarks compiling

    * Fix EF withdrawals test

    * Remove unused deps

    * Fix tree_hash panic in tests

    * Fix slasher compilation

    * Fix ssz_generic test

    * Get more tests passing

    * Fix EF tests for real

    * Fix local testnet scripts

commit ca412ab
Author: Michael Sproul <[email protected]>
Date:   Wed Jun 21 11:05:09 2023 +1000

    Use rebasing to minimise BeaconState mem usage (sigp#4416)

    * Use "rebasing" to minimise BeaconState mem usage

    * Update metastruct

    * Use upstream milhouse, update cargo lock

    * Rebase caches for extra memory savings

commit 6eb1513
Author: Michael Sproul <[email protected]>
Date:   Tue Jun 20 19:10:05 2023 +1000

    Configurable diff buffer cache size (sigp#4420)

commit d56cec8
Author: Paul Hauner <[email protected]>
Date:   Tue Jun 20 11:47:52 2023 +1000

    Address clippy lints in `tree-states` (sigp#4414)

    * Address some clippy lints

    * Box errors to fix error size lint

    * Add Default impl for Validator

    * Address more clippy lints

    * Re-implement `check_state_diff`

    * Fix misc test compile errors

commit 23db089
Author: Michael Sproul <[email protected]>
Date:   Mon Jun 19 10:14:47 2023 +1000

    Implement tree states & hierarchical state DB
  • Loading branch information
@dapplion
dapplion committed Feb 23, 2024
1 parent 20f53e7 commit 50a03dd
Show file tree
Hide file tree
Showing 21 changed files with 126 additions and 111 deletions.
54 changes: 36 additions & 18 deletions beacon_node/beacon_chain/src/attestation_rewards.rs
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use crate::{BeaconChain, BeaconChainError, BeaconChainTypes};
use eth2::lighthouse::attestation_rewards::{IdealAttestationRewards, TotalAttestationRewards};
use eth2::lighthouse::StandardAttestationRewards;
use participation_cache::ParticipationCache;
use safe_arith::SafeArith;
use serde_utils::quoted_u64::Quoted;
use slog::debug;
Expand All @@ -10,7 +9,7 @@ use state_processing::per_epoch_processing::altair::{
};
use state_processing::{
common::altair::BaseRewardPerIncrement,
per_epoch_processing::altair::{participation_cache, rewards_and_penalties::get_flag_weight},
per_epoch_processing::altair::rewards_and_penalties::get_flag_weight,
};
use std::collections::HashMap;
use store::consts::altair::{
Expand Down Expand Up @@ -134,8 +133,6 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
let spec = &self.spec;

// Calculate ideal_rewards
let participation_cache = ParticipationCache::new(&state, spec)
.map_err(|_| BeaconChainError::AttestationRewardsError)?;
process_justification_and_finalization(&state)?.apply_changes_to_state(&mut state);
process_inactivity_updates_slow(&mut state, spec)?;

Expand All @@ -147,14 +144,14 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
let weight = get_flag_weight(flag_index)
.map_err(|_| BeaconChainError::AttestationRewardsError)?;

let unslashed_participating_balance = participation_cache
.previous_epoch_flag_attesting_balance(flag_index)
.map_err(|_| BeaconChainError::AttestationRewardsError)?;
let unslashed_participating_balance = state
.progressive_balances_cache()
.previous_epoch_flag_attesting_balance(flag_index)?;

let unslashed_participating_increments =
unslashed_participating_balance.safe_div(spec.effective_balance_increment)?;

let total_active_balance = participation_cache.current_epoch_total_active_balance();
let total_active_balance = state.get_total_active_balance()?;

let active_increments =
total_active_balance.safe_div(spec.effective_balance_increment)?;
Expand Down Expand Up @@ -190,15 +187,15 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
let mut total_rewards: Vec<TotalAttestationRewards> = Vec::new();

let validators = if validators.is_empty() {
participation_cache.eligible_validator_indices().to_vec()
Self::all_eligible_validator_indices(&state, previous_epoch)?
} else {
Self::validators_ids_to_indices(&mut state, validators)?
};

for &validator_index in &validators {
// Return 0s for unknown/inactive validator indices. This is a bit different from stable
// where we error for unknown pubkeys.
let Ok(validator) = participation_cache.get_validator(validator_index) else {
let Ok(validator) = state.get_validator(validator_index) else {
debug!(
self.log,
"No rewards for inactive/unknown validator";
Expand All @@ -215,22 +212,25 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
});
continue;
};
let eligible = validator.is_eligible;
let previous_epoch_participation_flags = state
.previous_epoch_participation()?
.get(validator_index)
.ok_or(BeaconChainError::AttestationRewardsError)?;
let eligible = state.is_eligible_validator(previous_epoch, validator)?;
let mut head_reward = 0i64;
let mut target_reward = 0i64;
let mut source_reward = 0i64;
let mut inactivity_penalty = 0i64;

if eligible {
let effective_balance = validator.effective_balance;
let effective_balance = validator.effective_balance();

for flag_index in 0..PARTICIPATION_FLAG_WEIGHTS.len() {
let (ideal_reward, penalty) = ideal_rewards_hashmap
.get(&(flag_index, effective_balance))
.ok_or(BeaconChainError::AttestationRewardsError)?;
let voted_correctly = validator
.is_unslashed_participating_index(flag_index)
.map_err(|_| BeaconChainError::AttestationRewardsError)?;
let voted_correctly = !validator.slashed()
&& previous_epoch_participation_flags.has_flag(flag_index)?;
if voted_correctly {
if flag_index == TIMELY_HEAD_FLAG_INDEX {
head_reward += *ideal_reward as i64;
Expand All @@ -246,9 +246,9 @@ impl<T: BeaconChainTypes> BeaconChain<T> {

let penalty_numerator = effective_balance
.safe_mul(state.get_inactivity_score(validator_index)?)?;
let penalty_denominator = spec
.inactivity_score_bias
.safe_mul(spec.inactivity_penalty_quotient_for_state(&state))?;
let penalty_denominator = spec.inactivity_score_bias.safe_mul(
spec.inactivity_penalty_quotient_for_fork(state.fork_name_unchecked()),
)?;
inactivity_penalty =
-(penalty_numerator.safe_div(penalty_denominator)? as i64);
} else if flag_index == TIMELY_SOURCE_FLAG_INDEX {
Expand Down Expand Up @@ -314,6 +314,24 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
Ok(max_steps)
}

fn all_eligible_validator_indices(
state: &BeaconState<T::EthSpec>,
previous_epoch: Epoch,
) -> Result<Vec<usize>, BeaconChainError> {
state
.validators()
.iter()
.enumerate()
.filter_map(|(i, validator)| {
state
.is_eligible_validator(previous_epoch, validator)
.map(|eligible| eligible.then_some(i))
.map_err(BeaconChainError::BeaconStateError)
.transpose()
})
.collect()
}

fn validators_ids_to_indices(
state: &mut BeaconState<T::EthSpec>,
validators: Vec<ValidatorId>,
Expand Down
8 changes: 0 additions & 8 deletions beacon_node/http_api/src/attestation_performance.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ use eth2::lighthouse::{
AttestationPerformance, AttestationPerformanceQuery, AttestationPerformanceStatistics,
};
use state_processing::{
per_epoch_processing::altair::participation_cache::Error as ParticipationCacheError,
per_epoch_processing::EpochProcessingSummary, BlockReplayError, BlockReplayer,
};
use std::sync::Arc;
Expand All @@ -18,7 +17,6 @@ const BLOCK_ROOT_CHUNK_SIZE: usize = 100;
enum AttestationPerformanceError {
BlockReplay(#[allow(dead_code)] BlockReplayError),
BeaconState(#[allow(dead_code)] BeaconStateError),
ParticipationCache(#[allow(dead_code)] ParticipationCacheError),
UnableToFindValidator(#[allow(dead_code)] usize),
}

Expand All @@ -34,12 +32,6 @@ impl From<BeaconStateError> for AttestationPerformanceError {
}
}

impl From<ParticipationCacheError> for AttestationPerformanceError {
fn from(e: ParticipationCacheError) -> Self {
Self::ParticipationCache(e)
}
}

pub fn get_attestation_performance<T: BeaconChainTypes>(
target: String,
query: AttestationPerformanceQuery,
Expand Down
1 change: 0 additions & 1 deletion beacon_node/http_api/src/validator_inclusion.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,6 @@ pub fn global_validator_inclusion_data<T: BeaconChainTypes>(

Ok(GlobalValidatorInclusionData {
current_epoch_active_gwei: summary.current_epoch_total_active_balance(),
previous_epoch_active_gwei: summary.previous_epoch_total_active_balance(),
current_epoch_target_attesting_gwei: summary
.current_epoch_target_attesting_balance()
.map_err(convert_cache_error)?,
Expand Down
2 changes: 0 additions & 2 deletions common/eth2/src/lighthouse.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,6 @@ pub struct Peer<T: EthSpec> {
pub struct GlobalValidatorInclusionData {
/// The total effective balance of all active validators during the _current_ epoch.
pub current_epoch_active_gwei: u64,
/// The total effective balance of all active validators during the _previous_ epoch.
pub previous_epoch_active_gwei: u64,
/// The total effective balance of all validators who attested during the _current_ epoch and
/// agreed with the state about the beacon block at the first slot of the _current_ epoch.
pub current_epoch_target_attesting_gwei: u64,
Expand Down
6 changes: 5 additions & 1 deletion consensus/fork_choice/src/fork_choice.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -747,7 +747,11 @@ where
| BeaconBlockRef::Capella(_)
| BeaconBlockRef::Merge(_)
| BeaconBlockRef::Altair(_) => {
// FIXME(sproul): initialize progressive balances
// NOTE: Processing justification & finalization requires the progressive
// balances cache, but we cannot initialize it here as we only have an
// immutable reference. The state *should* have come straight from block
// processing, which initialises the cache, but if we add other `on_block`
// calls in future it could be worth passing a mutable reference.
per_epoch_processing::altair::process_justification_and_finalization(state)?
}
BeaconBlockRef::Base(_) => {
Expand Down
2 changes: 1 addition & 1 deletion consensus/state_processing/src/all_caches.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ impl<E: EthSpec> AllCaches for BeaconState<E> {
fn build_all_caches(&mut self, spec: &ChainSpec) -> Result<(), EpochCacheError> {
self.build_caches(spec)?;
initialize_epoch_cache(self, spec)?;
initialize_progressive_balances_cache(self, None, spec)?;
initialize_progressive_balances_cache(self, spec)?;
Ok(())
}

Expand Down
79 changes: 51 additions & 28 deletions consensus/state_processing/src/common/update_progressive_balances_cache.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,16 @@ use crate::metrics::{
PARTICIPATION_CURR_EPOCH_TARGET_ATTESTING_GWEI_PROGRESSIVE_TOTAL,
PARTICIPATION_PREV_EPOCH_TARGET_ATTESTING_GWEI_PROGRESSIVE_TOTAL,
};
use crate::per_epoch_processing::altair::ParticipationCache;
use crate::{BlockProcessingError, EpochProcessingError};
use lighthouse_metrics::set_gauge;
use std::borrow::Cow;
use types::{
is_progressive_balances_enabled, BeaconState, BeaconStateError, ChainSpec, Epoch,
EpochTotalBalances, EthSpec, ProgressiveBalancesCache,
EpochTotalBalances, EthSpec, ParticipationFlags, ProgressiveBalancesCache, Validator,
};

/// Initializes the `ProgressiveBalancesCache` cache using balance values from the
/// `ParticipationCache`. If the optional `&ParticipationCache` is not supplied, it will be computed
/// from the `BeaconState`.
/// Initializes the `ProgressiveBalancesCache` if it is unbuilt.
pub fn initialize_progressive_balances_cache<E: EthSpec>(
state: &mut BeaconState<E>,
maybe_participation_cache: Option<&ParticipationCache>,
spec: &ChainSpec,
) -> Result<(), BeaconStateError> {
if !is_progressive_balances_enabled(state)
Expand All @@ -26,29 +21,37 @@ pub fn initialize_progressive_balances_cache<E: EthSpec>(
return Ok(());
}

// FIXME(sproul): simplify the participation cache
let participation_cache = match maybe_participation_cache {
Some(cache) => Cow::Borrowed(cache),
None => {
state.build_total_active_balance_cache_at(state.current_epoch(), spec)?;
Cow::Owned(
ParticipationCache::new(state, spec)
.map_err(|e| BeaconStateError::ParticipationCacheError(format!("{e:?}")))?,
)
// Calculate the total flag balances for previous & current epoch in a single iteration.
// This calculates `get_total_balance(unslashed_participating_indices(..))` for each flag in
// the current and previous epoch.
let current_epoch = state.current_epoch();
let previous_epoch = state.previous_epoch();
let mut previous_epoch_cache = EpochTotalBalances::new(spec);
let mut current_epoch_cache = EpochTotalBalances::new(spec);
for ((validator, current_epoch_flags), previous_epoch_flags) in state
.validators()
.iter()
.zip(state.current_epoch_participation()?)
.zip(state.previous_epoch_participation()?)
{
// Exclude slashed validators. We are calculating *unslashed* participating totals.
if validator.slashed() {
continue;
}
};

let current_epoch = state.current_epoch();
let previous_epoch_cache = EpochTotalBalances {
total_flag_balances: participation_cache
.previous_epoch_participation
.total_flag_balances,
};
let current_epoch_cache = EpochTotalBalances {
total_flag_balances: participation_cache
.current_epoch_participation
.total_flag_balances,
};
// Update current epoch flag balances.
if validator.is_active_at(current_epoch) {
update_flag_total_balances(&mut current_epoch_cache, *current_epoch_flags, validator)?;
}
// Update previous epoch flag balances.
if validator.is_active_at(previous_epoch) {
update_flag_total_balances(
&mut previous_epoch_cache,
*previous_epoch_flags,
validator,
)?;
}
}

state.progressive_balances_cache_mut().initialize(
current_epoch,
Expand All @@ -61,6 +64,26 @@ pub fn initialize_progressive_balances_cache<E: EthSpec>(
Ok(())
}

/// During the initialization of the progressive balances for a single epoch, add
/// `validator.effective_balance` to the flag total, for each flag present in `participation_flags`.
///
/// Pre-conditions:
///
/// - `validator` must not be slashed
/// - the `participation_flags` must be for `validator` in the same epoch as the `total_balances`
fn update_flag_total_balances(
total_balances: &mut EpochTotalBalances,
participation_flags: ParticipationFlags,
validator: &Validator,
) -> Result<(), BeaconStateError> {
for (flag, balance) in total_balances.total_flag_balances.iter_mut().enumerate() {
if participation_flags.has_flag(flag)? {
balance.safe_add_assign(validator.effective_balance())?;
}
}
Ok(())
}

/// Updates the `ProgressiveBalancesCache` when a new target attestation has been processed.
pub fn update_progressive_balances_on_attestation<T: EthSpec>(
state: &mut BeaconState<T>,
Expand Down
2 changes: 1 addition & 1 deletion consensus/state_processing/src/per_block_processing.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ pub fn per_block_processing<T: EthSpec, Payload: AbstractExecPayload<T>>(

// Build epoch cache if it hasn't already been built, or if it is no longer valid
initialize_epoch_cache(state, spec)?;
initialize_progressive_balances_cache(state, None, spec)?;
initialize_progressive_balances_cache(state, spec)?;
state.build_slashings_cache()?;

let verify_signatures = match block_signature_strategy {
Expand Down
26 changes: 21 additions & 5 deletions consensus/state_processing/src/per_block_processing/altair/sync_committee.rs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ use crate::{signature_sets::sync_aggregate_signature_set, VerifySignatures};
use safe_arith::SafeArith;
use std::borrow::Cow;
use types::consts::altair::{PROPOSER_WEIGHT, SYNC_REWARD_WEIGHT, WEIGHT_DENOMINATOR};
use types::{BeaconState, ChainSpec, EthSpec, PublicKeyBytes, SyncAggregate, Unsigned};
use types::{
BeaconState, BeaconStateError, ChainSpec, EthSpec, PublicKeyBytes, SyncAggregate, Unsigned,
};

pub fn process_sync_aggregate<T: EthSpec>(
state: &mut BeaconState<T>,
Expand Down Expand Up @@ -47,20 +49,34 @@ pub fn process_sync_aggregate<T: EthSpec>(
// Apply participant and proposer rewards
let committee_indices = state.get_sync_committee_indices(&current_sync_committee)?;

let mut total_proposer_reward = 0;
let proposer_index = proposer_index as usize;
let mut proposer_balance = *state
.balances()
.get(proposer_index)
.ok_or(BeaconStateError::BalancesOutOfBounds(proposer_index))?;

for (participant_index, participation_bit) in committee_indices
.into_iter()
.zip(aggregate.sync_committee_bits.iter())
{
// FIXME(sproul): double-check this for Capella, proposer shouldn't have 0 effective balance
if participation_bit {
increase_balance(state, participant_index, participant_reward)?;
total_proposer_reward.safe_add_assign(proposer_reward)?;
// Accumulate proposer rewards in a temp var in case the proposer has very low balance, is
// part of the sync committee, does not participate and its penalties saturate.
if participant_index == proposer_index {
proposer_balance.safe_add_assign(participant_reward)?;
} else {
increase_balance(state, participant_index, participant_reward)?;
}
proposer_balance.safe_add_assign(proposer_reward)?;
} else if participant_index == proposer_index {
proposer_balance = proposer_balance.saturating_sub(participant_reward);
} else {
decrease_balance(state, participant_index, participant_reward)?;
}
}
increase_balance(state, proposer_index as usize, total_proposer_reward)?;

*state.get_balance_mut(proposer_index)? = proposer_balance;

Ok(())
}
Expand Down
9 changes: 0 additions & 9 deletions consensus/state_processing/src/per_block_processing/errors.rs
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
use super::signature_sets::Error as SignatureSetError;
use crate::per_epoch_processing::altair::participation_cache;
use crate::{ContextError, EpochCacheError};
use merkle_proof::MerkleTreeError;
use participation_cache::Error as ParticipationCacheError;
use safe_arith::ArithError;
use ssz::DecodeError;
use types::*;
Expand Down Expand Up @@ -91,7 +89,6 @@ pub enum BlockProcessingError {
found: Hash256,
},
WithdrawalCredentialsInvalid,
ParticipationCacheError(ParticipationCacheError),
}

impl From<BeaconStateError> for BlockProcessingError {
Expand Down Expand Up @@ -161,12 +158,6 @@ impl From<BlockOperationError<HeaderInvalid>> for BlockProcessingError {
}
}

impl From<ParticipationCacheError> for BlockProcessingError {
fn from(e: ParticipationCacheError) -> Self {
BlockProcessingError::ParticipationCacheError(e)
}
}

/// A conversion that consumes `self` and adds an `index` variable to resulting struct.
///
/// Used here to allow converting an error into an upstream error that points to the object that
Expand Down
Loading

0 comments on commit 50a03dd

Please sign in to comment.