Skip to content

Commit

Permalink
Squashed commit of #2404
Browse files Browse the repository at this point in the history
commit 410d751
Merge: 8e7466a 6a6d175
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Tue Aug 3 18:02:45 2021 +1000

    Merge remote-tracking branch 'origin/network-altair' into altair-vc

commit 8e7466a
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Tue Aug 3 18:02:21 2021 +1000

    Fix beacon chain tests

commit 6a6d175
Author: pawan <pawandhananjay@gmail.com>
Date:   Tue Aug 3 12:49:30 2021 +0530

    Address more review comments

commit 3266fcd
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Tue Aug 3 17:20:15 2021 +1000

    Fix clippy

commit 86b68a4
Merge: 84830a8 68357f3
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Tue Aug 3 17:13:52 2021 +1000

    Merge remote-tracking branch 'origin/network-altair' into altair-vc

commit 84830a8
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Tue Aug 3 16:53:51 2021 +1000

    Implement API for sync committee indices

commit 68357f3
Author: pawan <pawandhananjay@gmail.com>
Date:   Mon Aug 2 13:44:44 2021 +0530

    Remove sync_committee_subnet_count from ChainSpec

commit 940535c
Merge: 05fe793 c5786a8
Author: pawan <pawandhananjay@gmail.com>
Date:   Mon Aug 2 13:09:04 2021 +0530

    Merge branch 'unstable' into network-altair

commit 05fe793
Author: pawan <pawandhananjay@gmail.com>
Date:   Mon Aug 2 12:07:49 2021 +0530

    Fix issues from review

commit 7b3a0a1
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Aug 2 16:11:02 2021 +1000

    Tweak log levels

commit 8d6cdf9
Merge: d1bbf1a c5786a8
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Aug 2 14:46:21 2021 +1000

    Merge remote-tracking branch 'origin/unstable' into altair-vc

commit d1bbf1a
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Fri Jul 30 12:46:26 2021 +1000

    Add /v2/ API endpoints

commit b9d134e
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jul 26 12:37:49 2021 +1000

    Add target-peers and spec preset to local testnet

commit 0c25e92
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jul 26 12:34:34 2021 +1000

    Prevent bootnode adding its own ENR

commit a3fb7b3
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jul 26 12:34:11 2021 +1000

    Limit selection proof pre-computation

commit d0e2153
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Fri Jul 23 14:23:15 2021 +1000

    Delete the fork service

commit e63b5de
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Thu Jul 22 12:11:43 2021 +1000

    Altair validator client and API

commit 2b04ddf
Merge: 59a32a3 923486f
Author: pawan <pawandhananjay@gmail.com>
Date:   Wed Jul 28 14:45:57 2021 +0530

    Merge branch 'unstable' into network-altair

commit 59a32a3
Author: pawan <pawandhananjay@gmail.com>
Date:   Sat Jul 24 00:18:47 2021 +0530

    Fix minor issues

commit 0656056
Author: pawan <pawandhananjay@gmail.com>
Date:   Fri Jul 23 18:26:57 2021 +0530

    Change libp2p dependency to sigp fork

commit bed92c9
Author: realbigsean <seananderson33@gmail.com>
Date:   Tue Jul 13 12:21:56 2021 -0400

    fix incorrect method name and update test to ensure off-by-one is invalid

commit 6c76853
Merge: 2e35733 63923ea
Author: pawan <pawandhananjay@gmail.com>
Date:   Wed Jul 21 23:31:30 2021 +0530

    Merge discv5 upstream changes

commit 2e35733
Author: pawan <pawandhananjay@gmail.com>
Date:   Tue Jul 20 16:14:34 2021 +0530

    Maintain sync committee peers

commit 9f29788
Author: pawan <pawandhananjay@gmail.com>
Date:   Mon Jul 19 15:42:12 2021 +0530

    Fix subnet service test

commit 9f9c184
Merge: 3d42f06 9a8320b
Author: pawan <pawandhananjay@gmail.com>
Date:   Fri Jul 16 20:44:09 2021 +0530

    Merge network-1.5 changes

commit 3d42f06
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Jul 16 18:56:38 2021 +0530

    Network altair gossip (#2464)

    * small refactors

    * Update attestation/deposit processing

    * Implement `upgrade_to_altair`, genesis init test

    * fix slashings and implement sync committee processing

    * refactor common epoch processing methods

    * delete unused imports

    * Fix typo in sync_committee_updates

    * Update to v1.1.0-alpha.3

    * Fix remaining EF tests/handlers!

    * Fix beacon chain tests non-altair (#2305)

    * Use RefMut

    * Reset Cargo.lock to "stable" style

    * Tidy block tree hash

    * Temporarily hardcode base fork in op pool

    * Update block proposal & op pool

    * Fix bounds check on compute_sync_committee_indices

    * Tidy phase0 op pool

    * Remove global fork schedule (#2314)

    * Remove global fork schedule
    * Fix remaining EF tests for refactor
    * Get top-level compiling again
    * Bubble YAML config changes

    * Use superstruct partial getters

    This makes our Altair consensus logic compatible with future forks.

    * Lots of test refactors (#2315)

    * Fix beacon chain tests non-altair

    * fix op pool tests

    * fix merge

    * get block processing tests working

    * Refactor state transition vector tests, half are passing

    * cargo fmt

    * PR updates

    * Skip ahead to epoch 256 on beacon harness tests. Fix op pool tests.

    * clippy updates

    * More test cleanups and fixes

    * Use `upgrade_to_altair` in state transition

    * More test fixes

    * Begin adding pools related to sync committees

    * Add fork upgrade test, fix EF spec test

    * Sync committee pool refactors, attempt to get ssz tests working

    * Revert BitVector change

    * Update lcli and local testnet scripts

    * Fix `SyncCommitteeContribution` bit vector length, fix `SyncCommitteeSignature` field name

    * update docs and add tests for sync committee operation pool

    * More test updates, release-tests might pass now...

    * Nuke old store benchmark

    * Fix arbitrary instances

    * Participation bits fixes, clippy fixes

    * add BitVector intersection and union tests

    * Disable indexing in types & state processing

    * Delete tree hash and state proc benchmarks

    * Disable aggressive lints in tests

    * Appease udeps

    * Fix the RPC tests without hacks

    Thanks @pawanjay176

    * Fix double update in github cargo fmt

    * Use published superstruct

    * Try splitting beacon_chain tests out of CI

    * Fix cargo-fmt --all

    * Abstract over fork in beacon_chain tests

    Also consolidate persistence_tests into store tests to
    reduce the # of integration test binaries slightly.

    * Fix clippy

    * Maintain backwards compatibility with std config

    * Sync committee signature verification work

    * Add an `Arc` to the `current_sync_committee` field, refactor `sync_committee_verification` code

    * Clean up imports

    * Move `sync_committee_base_epoch` to `Epoch`

    * Fmt and fix some lints

    * Add tests for new observation pools

    * Some op pool work

    * Provide clarity about BeaconState leaf count (#2339)

    * Remove sync committee indices cache

    We could add back a cache at some point, but it would likely look
    completely different due to having to compute the indices "backwards"
    from the public keys. Dealing with all the cases involving rebuilding
    the cache efficiently, or what to do if it isn't built will be quite
    involved so I think it's better we wait to see if we need it before
    optimising prematurely.

    * - Add beacon harness methods for sync signature create
    - separate sync signature `SubnetId` and `SelecitonProof` types from attestations
    - update op pool persistence to include sync aggregates

    * merge with `altair`

    * Add tests for decoding state/block with bad slot (#2341)

    * Tidy SyncCommittee types

    * More sync committee type cleanups/fallout

    * merge with `altair`

    * Some sync verification fixes and test progress

    * Add hack to allow harness with altair spec

    * Add test for applying base block to altair chain

    * Use graffiti len more consistently (#2348)

    * Replace a test deleted during Altair update (#2351)

    * Apply comment fixes from Paul

    Co-authored-by: Paul Hauner <paul@paulhauner.com>

    * Add fork_name methods

    * Add fork checks and tests

    * Add tests for applying altair to base

    * Allow providing a custom spec to test harness

    * Fix compile error in network

    * Add tests for slot, epoch processing

    * Fix clippy lints

    * Fix release test

    * Finish op pool sync committee logic, add a couple tests.

    * Avoid slicing in PartialBeaconState (#2361)

    * [Altair] Make anonymous var-length SSZ decoding explicit (#2362)

    * Add explicit function for anon var length items

    * Remove unused import

    * Update comment

    * Use macros for naive aggregation pool tests.

    * small updates

    * [Altair] Rewards tests (#2349)

    * Rewards tests (base-only so far)

    * Altair rewards tests, clean-ups

    * Bump Rust version in Dockerfile

    * [Altair] Remove duplicate vars, panic on `u64::pow` overflow (#2350)

    * [Altair] Applying Base block to Altair chain (#2353)

    * Add hack to allow harness with altair spec

    * Add test for applying base block to altair chain

    * Add fork_name methods

    * Add fork checks and tests

    * Add tests for applying altair to base

    * Allow providing a custom spec to test harness

    * Fix compile error in network

    * Add tests for slot, epoch processing

    * Fix clippy lints

    * Fix release test

    * Fix test compile error

    * Fix test with inconsistent spec

    * Use fork_name_at_slot in genesis

    * [Altair] Add check for missed EF tests (#2346)

    * Rewards tests (base-only so far)

    * Altair rewards tests, clean-ups

    * Bump Rust version in Dockerfile

    * Add file access logging

    * Tidy

    * Fix clippy lint

    * Fix typo

    * Tidy, remove commented-out test exceptions

    * Improve naming in python script

    Co-authored-by: Michael Sproul <michael@sigmaprime.io>

    * Fix beacon chain harness bug in `make_sync_contributions`. Key the `ObservedSyncAggregators` pool on slot + subcommittee index

    * Accidentally removed a couple constants while merging

    * fix a typo in a const, and a bunch of lints

    * Restore lost CI YAML

    * Address review comments

    * Update comment

    * Return an error if total bal is 0 (#2366)

    * Fix and test selection proof `verify`

    * Fix get_sync_aggregate

    * update op pool tests to correctly use `get_sync_aggregate`

    * Minor changes to justification code (#2367)

    * [Altair] Optimization for `get_unslashed_participating_indices` (#2369)

    * Lift calculation of unslashed indices

    * Use HashSet for unslashed participating indices

    * Make Operation Pool Compile Again (MOPCA)

    * Add chainspec and genesis validators root to eth2_libp2p

    * Network altair rpc (#2301)

    * Add v2 messages to rpc decoder

    * Ugly hack

    * Pass chainspec and genesis_root to Rpc

    * Add context bytes while encoding

    * Add a ForkContext struct

    * Pass ForkContext to rpc

    * crate compiles

    * Extract ForkContext into separate file; add a current_fork field

    * Fix encoding/decoding

    * Fix tests

    * Remove fork_schedule from rebase

    * Fix ForkContext

    * Fix tests

    * Remove fork_schedule again

    * Add altair empty and full block limits

    * Fix panic in snappy decoding

    * Fix limits

    * Move wrapping of RPCRequests to handler

    * RpcRequestContainer only used in OutboundUpgrade

    * Add altair blocks in rpc end to end tests

    * same rpc limits for V1 and V2

    * V2 response decoding happens only for valid protocols

    * Add snappy response decoding tests

    * Add more snappy tests

    * Minor fixes

    * Appease clippy

    * to_context_bytes returns an Option

    * Add padding snappy message test for v2

    * Minor fixes; remove accidentally added file

    * lint

    * Add chainspec and genesis validators root to eth2_libp2p

    * Add altair gossip types

    * Subscribe to required gossip topics

    * Unsubscribe from pre fork topics

    * Compiles

    * Start adding sync committee attnets

    * Add Subnet enum to include SyncCommittee subnet_ids

    * update_enr_bitfield takes a Subnet instead of SubnetId

    * Fix network service

    * Add MetaData V2

    * Don't read context bytes for MetaData V2

    * Add sync committee topics to whitelist

    * Fix metadata encoding; fmt

    * Fix next_fork methods in chainspec

    * Minor fixes

    * Fix next_fork_update in network service; add next_unsubscribe delay

    * Accept peers which haven't updated next_fork_version/epoch

    * Add sync committee topic tests

    * Simplify required_gossip_digests; improve logging around fork boundary

    * Fix tests

    * Minor refactorings

    * Add sync contribution aggregation tests to the op pool and fix an op pool bug

    * Only advance state as necessary in sync committee verification tests. Fix compile error.

    * Delete `verify_sync_contributions.rs` and fix a comment.

    * Remove `Arc` from `current_sync_committee`

    * Add `max_capacity` to `ObservedAggregates` constructor

    * Update consensus code to v1.1.0-alpha.6

    * Make the `SyncAggregateId` a fixed-length struct

    * Add metrics and update docs

    * Address review comments

    * Sync committee subnets

    * Ripple spec updates

    * Revert "Remove `Arc` from `current_sync_committee`"

    This reverts commit bf6324e.

    * fix some pool tests

    * Updates to some docs, general cleanup

    * Fix default capacity for sync committee observed caches

    * Fix default capacity for sync committee observed caches

    * Fix default capacity for sync committee observed caches

    * Fix more docs and metrics

    * Remove `is_valid` methods from `SignedAggregateAndProof` and `SignedContributionAndProof`. The logic is duplicated in the signature set verification methods.

    * remove outdated `FIXME`

    * Fix docs

    * Fix docs

    * Implement preset <> config distinction

    * Update confusing name `SyncCommitteeSubnetSize` -> `SyncSubcommitteeSize`

    * Updates for latest spec merge

    * update client for `lighthouse/spec` endpoint

    * Add back accidentally deleted import

    * Return ConfigAndPreset from VC API

    * Don't panic if there aren't any aggregators

    Rustfmt also returned with a vengeance to rearrange this block.

    * Check state root in transition tests

    * Fix clippy

    * Fix Pyrmont & Prater configs. Delete Toledo.

    * Add compatibility shim for /config/spec

    * Fix fork_digest in Status message

    * Revert "update client for `lighthouse/spec` endpoint"

    This reverts commit 7a9b2ac.

    # Conflicts:
    #	validator_client/src/http_api/tests.rs

    * - Update `SYNC_COMMITTEE_SUBNET_COUNT` to alpha.6
    - rename the `sync_aggregator_selection_data.rs` file
    - include sync committee ssz static tests in the `check_all_file_accessed.py` checks
    - Update `ObservedAggregates` consts so they are specific to the type of object being cached

    * - Update `NaiveAggregationPool`'s default capacity so it is specific to the type of object being inserted

    * fix `observed_aggregates` tests

    * Add chainspec and genesis validators root to eth2_libp2p

    * Network altair rpc (#2301)

    * Add v2 messages to rpc decoder

    * Ugly hack

    * Pass chainspec and genesis_root to Rpc

    * Add context bytes while encoding

    * Add a ForkContext struct

    * Pass ForkContext to rpc

    * crate compiles

    * Extract ForkContext into separate file; add a current_fork field

    * Fix encoding/decoding

    * Fix tests

    * Remove fork_schedule from rebase

    * Fix ForkContext

    * Fix tests

    * Remove fork_schedule again

    * Add altair empty and full block limits

    * Fix panic in snappy decoding

    * Fix limits

    * Move wrapping of RPCRequests to handler

    * RpcRequestContainer only used in OutboundUpgrade

    * Add altair blocks in rpc end to end tests

    * same rpc limits for V1 and V2

    * V2 response decoding happens only for valid protocols

    * Add snappy response decoding tests

    * Add more snappy tests

    * Minor fixes

    * Appease clippy

    * to_context_bytes returns an Option

    * Add padding snappy message test for v2

    * Minor fixes; remove accidentally added file

    * lint

    * Use SyncSubnetId for sync committee subnets

    * Add worker tasks for sync committee gossip messages

    * Use correct type level constants

    * Decode altair gossip blocks

    * Add a SyncCommitteeSubscribe network event

    * PR updates

    * fix op pool tests

    * Minor cleanups

    * Calculate `SyncAggregate` on demand

    * Address review comments

    * Op pool migration + superstruct

    * Fix op pool test

    * Update to spec v1.1.0-alpha.7

    * Fix sync committee subscriptions

    * Add sync committee subscription test

    * Add chainspec and genesis validators root to eth2_libp2p

    * Network altair rpc (#2301)

    * Add v2 messages to rpc decoder

    * Ugly hack

    * Pass chainspec and genesis_root to Rpc

    * Add context bytes while encoding

    * Add a ForkContext struct

    * Pass ForkContext to rpc

    * crate compiles

    * Extract ForkContext into separate file; add a current_fork field

    * Fix encoding/decoding

    * Fix tests

    * Remove fork_schedule from rebase

    * Fix ForkContext

    * Fix tests

    * Remove fork_schedule again

    * Add altair empty and full block limits

    * Fix panic in snappy decoding

    * Fix limits

    * Move wrapping of RPCRequests to handler

    * RpcRequestContainer only used in OutboundUpgrade

    * Add altair blocks in rpc end to end tests

    * same rpc limits for V1 and V2

    * V2 response decoding happens only for valid protocols

    * Add snappy response decoding tests

    * Add more snappy tests

    * Minor fixes

    * Appease clippy

    * to_context_bytes returns an Option

    * Add padding snappy message test for v2

    * Minor fixes; remove accidentally added file

    * lint

    * - Fix sync committee verification on slot prior to sync committee period boundary
    - Add `Arc` to `next_sync_committee`
    - Other PR updates

    * Refactor how we retrieve `PublicKey` for sync signature verification

    * Key `observed_contributors` on `SlotSubcommitteeIndex`

    * Update mainnet preset YAML

    * Gossipsub message_id_fn changes

    * Add tests for sync committee verification at slot before sync committee period boundary

    * fix op pool tests

    * - Make `SyncSubnetId` a required argument for `verify_sync_signature_for_gossip`
    - Drop locks as soon as possible during metrics scrape

    * fix sync committee verification test macro

    * Fix beacon_chain test

    * rust 1.53.0 updates

    * add recursion limit to `simulator` crate

    * cargo fmt

    * `SyncCommitteSignature` -> `SyncCommitteeMessage`

    * Updates related to merge with `unstable`

    * More updates related to `SyncCommitteMessage` refactor

    * Couple more `unstable` merge fixes

    * Fix `SyncAggregate` field name

    * Add chainspec and genesis validators root to eth2_libp2p

    * Network altair rpc (#2301)

    * Add v2 messages to rpc decoder

    * Ugly hack

    * Pass chainspec and genesis_root to Rpc

    * Add context bytes while encoding

    * Add a ForkContext struct

    * Pass ForkContext to rpc

    * crate compiles

    * Extract ForkContext into separate file; add a current_fork field

    * Fix encoding/decoding

    * Fix tests

    * Remove fork_schedule from rebase

    * Fix ForkContext

    * Fix tests

    * Remove fork_schedule again

    * Add altair empty and full block limits

    * Fix panic in snappy decoding

    * Fix limits

    * Move wrapping of RPCRequests to handler

    * RpcRequestContainer only used in OutboundUpgrade

    * Add altair blocks in rpc end to end tests

    * same rpc limits for V1 and V2

    * V2 response decoding happens only for valid protocols

    * Add snappy response decoding tests

    * Add more snappy tests

    * Minor fixes

    * Appease clippy

    * to_context_bytes returns an Option

    * Add padding snappy message test for v2

    * Minor fixes; remove accidentally added file

    * lint

    * Change libp2p dependency for message_id_fn

    * appease clippy

    * Add sync committee related metrics

    * Expand and refactor fork functions

    * Simplify compute_subnets_for_sync_committee

    * Fix subnet service test

    * Add v2 protocols for OutboundRequests missed in the merge

    * Handle sync committee gossip errors

    * Remove delay for updating fork

    * fix the expected error in the sync contribution verification test at the sync period boundary

    * Updates related to merge with `unstable`

    * Couple more `unstable` merge fixes

    * Split rpc encode/decode into smaller functions

    * Import some relevant funcs from the VC

    Based on "Expand and refactor fork functions"

    * Fix issues with skip slots at fork boundaries

    * Fix clippy again

    * alpha.8 updates

    * Pop sync committee tasks from beacon processor

    * Fix some gossip scoring severity issue

    * Fix bug in peer_manager; Improve logging around subnet discoveries

    Co-authored-by: realbigsean <seananderson33@gmail.com>
    Co-authored-by: Michael Sproul <michael@sigmaprime.io>
    Co-authored-by: realbigsean <sean@sigmaprime.io>
    Co-authored-by: Paul Hauner <paul@paulhauner.com>

commit f37f267
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue May 11 10:30:43 2021 +0530

    Network altair rpc (#2301)

    * Add v2 messages to rpc decoder

    * Ugly hack

    * Pass chainspec and genesis_root to Rpc

    * Add context bytes while encoding

    * Add a ForkContext struct

    * Pass ForkContext to rpc

    * crate compiles

    * Extract ForkContext into separate file; add a current_fork field

    * Fix encoding/decoding

    * Fix tests

    * Remove fork_schedule from rebase

    * Fix ForkContext

    * Fix tests

    * Remove fork_schedule again

    * Add altair empty and full block limits

    * Fix panic in snappy decoding

    * Fix limits

    * Move wrapping of RPCRequests to handler

    * RpcRequestContainer only used in OutboundUpgrade

    * Add altair blocks in rpc end to end tests

    * same rpc limits for V1 and V2

    * V2 response decoding happens only for valid protocols

    * Add snappy response decoding tests

    * Add more snappy tests

    * Minor fixes

    * Appease clippy

    * to_context_bytes returns an Option

    * Add padding snappy message test for v2

    * Minor fixes; remove accidentally added file

    * lint

commit b477bdd
Author: pawan <pawandhananjay@gmail.com>
Date:   Mon Apr 5 19:30:48 2021 +0530

    Add chainspec and genesis validators root to eth2_libp2p
  • Loading branch information
paulhauner committed Aug 4, 2021
1 parent 47d0dff commit a30680f
Showing 85 changed files with 7,166 additions and 2,043 deletions.
92 changes: 48 additions & 44 deletions Cargo.lock

Large diffs are not rendered by default.

117 changes: 94 additions & 23 deletions beacon_node/beacon_chain/src/beacon_chain.rs
Original file line number Diff line number Diff line change
@@ -43,7 +43,7 @@ use crate::validator_pubkey_cache::ValidatorPubkeyCache;
use crate::BeaconForkChoiceStore;
use crate::BeaconSnapshot;
use crate::{metrics, BeaconChainError};
use eth2::types::{EventKind, SseBlock, SseChainReorg, SseFinalizedCheckpoint, SseHead};
use eth2::types::{EventKind, SseBlock, SseChainReorg, SseFinalizedCheckpoint, SseHead, SyncDuty};
use fork_choice::ForkChoice;
use futures::channel::mpsc::Sender;
use itertools::process_results;
@@ -1081,6 +1081,29 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
Ok(pubkey_cache.get_index(pubkey))
}

/// Return the validator indices of all public keys fetched from an iterator.
///
/// If any public key doesn't belong to a known validator then an error will be returned.
/// We could consider relaxing this by returning `Vec<Option<usize>>` in future.
pub fn validator_indices<'a>(
&self,
validator_pubkeys: impl Iterator<Item = &'a PublicKeyBytes>,
) -> Result<Vec<u64>, Error> {
let pubkey_cache = self
.validator_pubkey_cache
.try_read_for(VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT)
.ok_or(Error::ValidatorPubkeyCacheLockTimeout)?;

validator_pubkeys
.map(|pubkey| {
pubkey_cache
.get_index(pubkey)
.map(|id| id as u64)
.ok_or(Error::ValidatorPubkeyUnknown(*pubkey))
})
.collect()
}

/// Returns the validator pubkey (if any) for the given validator index.
///
/// ## Notes
@@ -1214,6 +1237,16 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.get_by_slot_and_root(slot, attestation_data_root)
}

/// Return an aggregated `SyncCommitteeContribution` matching the given `root`.
pub fn get_aggregated_sync_committee_contribution(
&self,
sync_contribution_data: &SyncContributionData,
) -> Option<SyncCommitteeContribution<T::EthSpec>> {
self.naive_sync_aggregation_pool
.read()
.get(sync_contribution_data)
}

/// Produce an unaggregated `Attestation` that is valid for the given `slot` and `index`.
///
/// The produced `Attestation` will not be valid until it has been signed by exactly one
@@ -1882,6 +1915,19 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
Ok(())
}

/// Attempt to obtain sync committee duties from the head.
pub fn sync_committee_duties_from_head(
&self,
epoch: Epoch,
validator_indices: &[u64],
) -> Result<Vec<Option<SyncDuty>>, Error> {
self.with_head(move |head| {
head.beacon_state
.get_sync_committee_duties(epoch, validator_indices, &self.spec)
.map_err(Error::SyncDutiesError)
})
}

/// Attempt to verify and import a chain of blocks to `self`.
///
/// The provided blocks _must_ each reference the previous block via `block.parent_root` (i.e.,
@@ -2624,6 +2670,22 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
let proposer_index = state.get_beacon_proposer_index(state.slot(), &self.spec)? as u64;
let voluntary_exits = self.op_pool.get_voluntary_exits(&state, &self.spec).into();

// Closure to fetch a sync aggregate in cases where it is required.
let get_sync_aggregate = || -> Result<SyncAggregate<_>, BlockProductionError> {
Ok(self
.op_pool
.get_sync_aggregate(&state)
.map_err(BlockProductionError::OpPoolError)?
.unwrap_or_else(|| {
warn!(
self.log,
"Producing block with no sync contributions";
"slot" => state.slot(),
);
SyncAggregate::new()
}))
};

let inner_block = match state {
BeaconState::Base(_) => BeaconBlock::Base(BeaconBlockBase {
slot,
@@ -2641,24 +2703,26 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
voluntary_exits,
},
}),
BeaconState::Altair(_) => BeaconBlock::Altair(BeaconBlockAltair {
slot,
proposer_index,
parent_root,
state_root: Hash256::zero(),
body: BeaconBlockBodyAltair {
randao_reveal,
eth1_data,
graffiti,
proposer_slashings: proposer_slashings.into(),
attester_slashings: attester_slashings.into(),
attestations,
deposits,
voluntary_exits,
// FIXME(altair): put a sync aggregate from the pool here (once implemented)
sync_aggregate: SyncAggregate::new(),
},
}),
BeaconState::Altair(_) => {
let sync_aggregate = get_sync_aggregate()?;
BeaconBlock::Altair(BeaconBlockAltair {
slot,
proposer_index,
parent_root,
state_root: Hash256::zero(),
body: BeaconBlockBodyAltair {
randao_reveal,
eth1_data,
graffiti,
proposer_slashings: proposer_slashings.into(),
attester_slashings: attester_slashings.into(),
attestations,
deposits,
voluntary_exits,
sync_aggregate,
},
})
}
};

let block = SignedBeaconBlock::from_block(
@@ -3324,14 +3388,21 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
// therefore use the genesis slot.
let slot = self.slot().unwrap_or(self.spec.genesis_slot);

self.spec.enr_fork_id(slot, self.genesis_validators_root)
self.spec
.enr_fork_id::<T::EthSpec>(slot, self.genesis_validators_root)
}

/// Calculates the `Duration` to the next fork, if one exists.
pub fn duration_to_next_fork(&self) -> Option<Duration> {
let epoch = self.spec.next_fork_epoch()?;
/// Calculates the `Duration` to the next fork if it exists and returns it
/// with it's corresponding `ForkName`.
pub fn duration_to_next_fork(&self) -> Option<(ForkName, Duration)> {
// If we are unable to read the slot clock we assume that it is prior to genesis and
// therefore use the genesis slot.
let slot = self.slot().unwrap_or(self.spec.genesis_slot);

let (fork_name, epoch) = self.spec.next_fork_epoch::<T::EthSpec>(slot)?;
self.slot_clock
.duration_to_slot(epoch.start_slot(T::EthSpec::slots_per_epoch()))
.map(|duration| (fork_name, duration))
}

pub fn dump_as_dot<W: Write>(&self, output: &mut W) {
2 changes: 2 additions & 0 deletions beacon_node/beacon_chain/src/errors.rs
Original file line number Diff line number Diff line change
@@ -87,6 +87,7 @@ pub enum BeaconChainError {
DuplicateValidatorPublicKey,
ValidatorPubkeyCacheFileError(String),
ValidatorIndexUnknown(usize),
ValidatorPubkeyUnknown(PublicKeyBytes),
OpPoolError(OpPoolError),
NaiveAggregationError(NaiveAggregationError),
ObservedAttestationsError(ObservedAttestationsError),
@@ -120,6 +121,7 @@ pub enum BeaconChainError {
state_epoch: Epoch,
shuffling_epoch: Epoch,
},
SyncDutiesError(BeaconStateError),
InconsistentForwardsIter {
request_slot: Slot,
slot: Slot,
39 changes: 31 additions & 8 deletions beacon_node/beacon_chain/src/test_utils.rs
Original file line number Diff line number Diff line change
@@ -151,7 +151,7 @@ pub fn test_spec<E: EthSpec>() -> ChainSpec {
pub struct BeaconChainHarness<T: BeaconChainTypes> {
pub validator_keypairs: Vec<Keypair>,

pub chain: BeaconChain<T>,
pub chain: Arc<BeaconChain<T>>,
pub spec: ChainSpec,
pub data_dir: TempDir,
pub shutdown_receiver: Receiver<ShutdownReason>,
@@ -229,6 +229,29 @@ impl<E: EthSpec> BeaconChainHarness<EphemeralHarnessType<E>> {
target_aggregators_per_committee: u64,
store_config: StoreConfig,
chain_config: ChainConfig,
) -> Self {
Self::new_with_mutator(
eth_spec_instance,
spec,
validator_keypairs,
target_aggregators_per_committee,
store_config,
chain_config,
|x| x,
)
}

/// Apply a function to beacon chain builder before building.
pub fn new_with_mutator(
eth_spec_instance: E,
spec: Option<ChainSpec>,
validator_keypairs: Vec<Keypair>,
target_aggregators_per_committee: u64,
store_config: StoreConfig,
chain_config: ChainConfig,
mutator: impl FnOnce(
BeaconChainBuilder<EphemeralHarnessType<E>>,
) -> BeaconChainBuilder<EphemeralHarnessType<E>>,
) -> Self {
let data_dir = tempdir().expect("should create temporary data_dir");
let mut spec = spec.unwrap_or_else(test_spec::<E>);
@@ -240,7 +263,7 @@ impl<E: EthSpec> BeaconChainHarness<EphemeralHarnessType<E>> {
let log = test_logger();

let store = HotColdDB::open_ephemeral(store_config, spec.clone(), log.clone()).unwrap();
let chain = BeaconChainBuilder::new(eth_spec_instance)
let builder = BeaconChainBuilder::new(eth_spec_instance)
.logger(log.clone())
.custom_spec(spec.clone())
.store(Arc::new(store))
@@ -260,13 +283,13 @@ impl<E: EthSpec> BeaconChainHarness<EphemeralHarnessType<E>> {
log.clone(),
1,
)))
.monitor_validators(true, vec![], log)
.build()
.expect("should build");
.monitor_validators(true, vec![], log);

let chain = mutator(builder).build().expect("should build");

Self {
spec: chain.spec.clone(),
chain,
chain: Arc::new(chain),
validator_keypairs,
data_dir,
shutdown_receiver,
@@ -311,7 +334,7 @@ impl<E: EthSpec> BeaconChainHarness<DiskHarnessType<E>> {

Self {
spec: chain.spec.clone(),
chain,
chain: Arc::new(chain),
validator_keypairs,
data_dir,
shutdown_receiver,
@@ -353,7 +376,7 @@ impl<E: EthSpec> BeaconChainHarness<DiskHarnessType<E>> {

Self {
spec: chain.spec.clone(),
chain,
chain: Arc::new(chain),
validator_keypairs,
data_dir,
shutdown_receiver,
18 changes: 13 additions & 5 deletions beacon_node/beacon_chain/tests/block_verification.rs
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@
extern crate lazy_static;

use beacon_chain::test_utils::{
AttestationStrategy, BeaconChainHarness, BlockStrategy, EphemeralHarnessType,
test_logger, AttestationStrategy, BeaconChainHarness, BlockStrategy, EphemeralHarnessType,
};
use beacon_chain::{BeaconSnapshot, BlockError, ChainConfig, ChainSegmentResult};
use slasher::{Config as SlasherConfig, Slasher};
@@ -830,17 +830,25 @@ fn block_gossip_verification() {

#[test]
fn verify_block_for_gossip_slashing_detection() {
let mut harness = get_harness(VALIDATOR_COUNT);

let slasher_dir = tempdir().unwrap();
let slasher = Arc::new(
Slasher::open(
SlasherConfig::new(slasher_dir.path().into()).for_testing(),
harness.logger().clone(),
test_logger(),
)
.unwrap(),
);
harness.chain.slasher = Some(slasher.clone());

let harness = BeaconChainHarness::new_with_mutator(
MainnetEthSpec,
None,
KEYPAIRS.to_vec(),
1 << 32,
StoreConfig::default(),
ChainConfig::default(),
|builder| builder.slasher(slasher.clone()),
);
harness.advance_slot();

let state = harness.get_current_state();
let (block1, _) = harness.make_block(state.clone(), Slot::new(1));
5 changes: 4 additions & 1 deletion beacon_node/eth2_libp2p/Cargo.toml
Original file line number Diff line number Diff line change
@@ -40,12 +40,15 @@ rand = "0.7.3"
directory = { path = "../../common/directory" }
regex = "1.3.9"
strum = { version = "0.20", features = ["derive"] }
superstruct = "0.2.0"

[dependencies.libp2p]
#version = "0.39.1"
#default-features = false
# TODO: Update once https://github.com/libp2p/rust-libp2p/pull/2103 and
# https://github.com/libp2p/rust-libp2p/pull/2137 are merged upstream.
git = "https://github.com/sigp/rust-libp2p"
rev = "323cae1d08112052740834aa1fb262ae43e6f783"
rev = "75fd53ec5407a58ae1ff600fd1c68ea49079364a"
features = ["websocket", "identify", "mplex", "yamux", "noise", "gossipsub", "dns-tokio", "tcp-tokio"]

[dev-dependencies]
Loading

0 comments on commit a30680f

Please sign in to comment.