Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ed/timeout #1794

Merged
merged 59 commits into from
Oct 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
56b6663
TimeoutCert trait and TimeoutVote2 trait done
elliedavidson Sep 18, 2023
9aa591b
TimeoutExchange wireframe
elliedavidson Sep 18, 2023
95f1aac
Add TimeoutExchangeType trait
elliedavidson Sep 18, 2023
1acb8a2
Compiles with timeout_exchange added to consensus task
elliedavidson Sep 18, 2023
2d55199
Rough addition of create_timeout_message on timeout_exchange
elliedavidson Sep 18, 2023
809a27a
Add Timeout events to networking task
elliedavidson Sep 18, 2023
14e5336
Consensus task properly recieves TimeoutVotes
elliedavidson Sep 18, 2023
d428557
Add timeout exchange and accumulator to consensus task, compiles
elliedavidson Sep 19, 2023
81dd019
Comments
elliedavidson Sep 19, 2023
112a404
Vote collection is receiving timeout votes correctly
elliedavidson Sep 19, 2023
f6968ce
Bug - proposals after timeout don't have timeout cert and have too lo…
elliedavidson Sep 19, 2023
a519b8e
Fix previous bug
elliedavidson Sep 19, 2023
5d3c145
view change location updated; need to handle update view sent for tim…
elliedavidson Sep 19, 2023
409e8ef
Some bugs fixed, still triggering view sync when it shouldn't
elliedavidson Sep 20, 2023
a3017eb
Memory network is not receiving DA proposals correctly
elliedavidson Sep 20, 2023
98c397d
Update timeout test to use libp2p
elliedavidson Sep 20, 2023
76e64cf
comments
elliedavidson Sep 21, 2023
5cdcff5
Fix test_consensus_task test after timeout logic update
elliedavidson Sep 21, 2023
ef68488
test_consensus_vote working
elliedavidson Sep 21, 2023
037de03
test_basic passes
elliedavidson Sep 21, 2023
89a14a3
All tests passing?
elliedavidson Sep 21, 2023
702dbd9
merge with main
elliedavidson Sep 21, 2023
70da94b
Tests pass?
elliedavidson Sep 22, 2023
86183cb
Cleaning up TODOs
elliedavidson Sep 22, 2023
a9eb94e
More todos
elliedavidson Sep 22, 2023
e9d1424
Fix lints
elliedavidson Sep 22, 2023
3fe3d22
rename functions
elliedavidson Sep 22, 2023
7767f5e
Lint imports
elliedavidson Sep 22, 2023
24c6b22
Additional cleanup
elliedavidson Sep 22, 2023
a8027c7
Additional cleanup
elliedavidson Sep 22, 2023
c248d2e
Additional cleanup
elliedavidson Sep 22, 2023
bb0eb5f
Fix lint bug
elliedavidson Sep 22, 2023
5259ca6
Fix lint bug
elliedavidson Sep 22, 2023
3d6fcb2
Fix fmt
elliedavidson Sep 22, 2023
f3ce354
Update view sync view update
elliedavidson Sep 22, 2023
539964d
Add 1 thread to catchup just command
elliedavidson Sep 22, 2023
de0744c
Compiles, tests do not pass
elliedavidson Oct 11, 2023
148937d
basic tests pass
elliedavidson Oct 11, 2023
3028fa0
Tests passing?
elliedavidson Oct 11, 2023
f192190
lints
elliedavidson Oct 11, 2023
50eb9cb
lints
elliedavidson Oct 11, 2023
e0b7371
lints
elliedavidson Oct 11, 2023
d7e8ebf
lints
elliedavidson Oct 11, 2023
fba4909
remove get_commit func
elliedavidson Oct 11, 2023
5e01be8
Fix subtraction overflow
elliedavidson Oct 11, 2023
f684c3e
Merge with develop
elliedavidson Oct 11, 2023
5eb3afa
Update state map even if no parent
elliedavidson Oct 11, 2023
cf095c4
Test timeout on both libp2p and web
elliedavidson Oct 12, 2023
721abe5
lint
elliedavidson Oct 12, 2023
b3fb79b
comments
elliedavidson Oct 12, 2023
1e7eb39
update metrics
elliedavidson Oct 12, 2023
bdd8012
metric updates
elliedavidson Oct 12, 2023
8acca28
Create own timeout cert validation function
elliedavidson Oct 12, 2023
944c98a
lints:
elliedavidson Oct 12, 2023
dff85fd
Comments
elliedavidson Oct 12, 2023
914e890
Comments
elliedavidson Oct 12, 2023
0fb23e3
lints
elliedavidson Oct 12, 2023
689f959
Add return if proposal is invliad
elliedavidson Oct 12, 2023
6fd0f40
lints
elliedavidson Oct 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions crates/hotshot-signature-key/src/bn254/bn254_priv.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ impl BLSPrivKey {
}
}

// #[allow(clippy::incorrect_partial_ord_impl_on_ord_type)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to get rid of this dead code?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kept this in because my local clippy needed it. I expect once CI's clippy is upgraded we'll need to uncomment these. :)

impl PartialOrd for BLSPrivKey {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
let self_bytes = &self.priv_key.to_string();
Expand Down
1 change: 1 addition & 0 deletions crates/hotshot-signature-key/src/bn254/bn254_pub.rs
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ pub struct BLSPubKey {
pub_key: VerKey,
}

// #[allow(clippy::incorrect_partial_ord_impl_on_ord_type)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also dead code here?

impl PartialOrd for BLSPubKey {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
let self_bytes = &self.pub_key.to_string();
Expand Down
1 change: 0 additions & 1 deletion crates/hotshot/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,6 @@ required-features = ["demo", "libp2p/rsa"]
path = "examples/web-server-da/multi-web-server.rs"

[dependencies]
# TODO ED We should upgrade ark libraries to 0.4
async-compatibility-layer = { workspace = true }
async-lock = { workspace = true }
async-trait = { workspace = true }
Expand Down
27 changes: 15 additions & 12 deletions crates/hotshot/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,9 @@ use hotshot_task::{
task_launcher::TaskRunner,
};
use hotshot_task_impls::{events::SequencingHotShotEvent, network::NetworkTaskKind};
use hotshot_types::{
certificate::TimeoutCertificate, traits::node_implementation::SequencingTimeoutEx,
};

use hotshot_types::{
block_impl::{VIDBlockPayload, VIDTransaction},
Expand Down Expand Up @@ -238,22 +241,14 @@ impl<TYPES: NodeType, I: NodeImplementation<TYPES>> SystemContext<TYPES, I> {
Ok(Self { inner })
}

/// "Starts" consensus by sending a `ViewChange` event
/// "Starts" consensus by sending a `QCFormed` event
pub async fn start_consensus(&self) {
self.inner
.internal_event_stream
.publish(SequencingHotShotEvent::ViewChange(TYPES::Time::new(1)))
.publish(SequencingHotShotEvent::QCFormed(either::Left(
QuorumCertificate::genesis(),
)))
.await;

// ED This isn't ideal...
// async_sleep(Duration::new(1, 0)).await;

// self.inner
// .internal_event_stream
// .publish(SequencingHotShotEvent::QCFormed(
// QuorumCertificate::genesis(),
// ))
// .await;
}

/// Marks a given view number as timed out. This should be called a fixed period after a round is started.
Expand Down Expand Up @@ -663,6 +658,14 @@ where
Commitment = Commitment<ViewSyncData<TYPES>>,
Membership = MEMBERSHIP,
> + 'static,
SequencingTimeoutEx<TYPES, I>: ConsensusExchange<
TYPES,
Message<TYPES, I>,
Proposal = QuorumProposal<TYPES, SequencingLeaf<TYPES>>,
Certificate = TimeoutCertificate<TYPES>,
Commitment = Commitment<TYPES::Time>,
Membership = MEMBERSHIP,
> + 'static,
{
fn consensus(&self) -> &Arc<RwLock<Consensus<TYPES, I::Leaf>>> {
&self.inner.consensus
Expand Down
25 changes: 22 additions & 3 deletions crates/hotshot/src/tasks/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,16 @@ use hotshot_task_impls::{
};
use hotshot_types::{
block_impl::{VIDBlockPayload, VIDTransaction},
certificate::ViewSyncCertificate,
certificate::{TimeoutCertificate, ViewSyncCertificate},
data::{ProposalType, QuorumProposal, SequencingLeaf},
event::Event,
message::{Message, Messages, SequencingMessage},
traits::{
election::{ConsensusExchange, Membership},
network::{CommunicationChannel, TransmitType},
network::{CommunicationChannel, ConsensusIntentEvent, TransmitType},
node_implementation::{
CommitteeEx, ExchangesType, NodeImplementation, NodeType, QuorumEx, ViewSyncEx,
CommitteeEx, ExchangesType, NodeImplementation, NodeType, QuorumEx,
SequencingTimeoutEx, ViewSyncEx,
},
state::ConsensusTime,
},
Expand Down Expand Up @@ -276,6 +277,13 @@ where
Certificate = DACertificate<TYPES>,
Commitment = Commitment<TYPES::BlockType>,
>,
SequencingTimeoutEx<TYPES, I>: ConsensusExchange<
TYPES,
Message<TYPES, I>,
Proposal = QuorumProposal<TYPES, SequencingLeaf<TYPES>>,
Certificate = TimeoutCertificate<TYPES>,
Commitment = Commitment<TYPES::Time>,
>,
{
let consensus = handle.hotshot.get_consensus();
let c_api: HotShotSequencingConsensusApi<TYPES, I> = HotShotSequencingConsensusApi {
Expand All @@ -290,6 +298,7 @@ where
cur_view: TYPES::Time::new(0),
block: Some(VIDBlockPayload::genesis()),
quorum_exchange: c_api.inner.exchanges.quorum_exchange().clone().into(),
timeout_exchange: c_api.inner.exchanges.timeout_exchange().clone().into(),
api: c_api.clone(),
committee_exchange: c_api.inner.exchanges.committee_exchange().clone().into(),
_pd: PhantomData,
Expand All @@ -302,6 +311,16 @@ where
id: handle.hotshot.inner.id,
qc: None,
};
consensus_state
.quorum_exchange
.network()
.inject_consensus_info(ConsensusIntentEvent::PollForCurrentProposal)
.await;
consensus_state
.quorum_exchange
.network()
.inject_consensus_info(ConsensusIntentEvent::PollForProposal(1))
.await;
let filter = FilterEvent(Arc::new(consensus_event_filter));
let consensus_name = "Consensus Task";
let consensus_event_handler = HandleEvent(Arc::new(
Expand Down
4 changes: 2 additions & 2 deletions crates/hotshot/src/traits/networking/combined_network.rs
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,7 @@ impl<TYPES: NodeType, I: NodeImplementation<TYPES>, MEMBERSHIP: Membership<TYPES
.broadcast_message(message.clone(), recipients.clone())
.await
{
Ok(_) => {
Ok(()) => {
self.primary_down.store(0, Ordering::Relaxed);
}
Err(e) => {
Expand Down Expand Up @@ -318,7 +318,7 @@ impl<TYPES: NodeType, I: NodeImplementation<TYPES>, MEMBERSHIP: Membership<TYPES
.direct_message(message.clone(), recipient.clone())
.await
{
Ok(_) => {
Ok(()) => {
self.primary_down.store(0, Ordering::Relaxed);
}
Err(e) => {
Expand Down
2 changes: 0 additions & 2 deletions crates/hotshot/src/traits/networking/web_server_network.rs
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,6 @@ struct Inner<M: NetworkMsg, KEY: SignatureKey, TYPES: NodeType> {
/// The last tx_index we saw from the web server
tx_index: Arc<RwLock<u64>>,

// TODO ED This should be TYPES::Time
// Theoretically there should never be contention for this lock...
/// Task map for quorum proposals.
proposal_task_map:
Arc<RwLock<HashMap<u64, UnboundedSender<ConsensusIntentEvent<TYPES::SignatureKey>>>>>,
Expand Down
2 changes: 1 addition & 1 deletion crates/hotshot/src/types/handle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,7 @@ impl<TYPES: NodeType, I: NodeImplementation<TYPES> + 'static> SystemContextHandl
if anchor_leaf.view_number == TYPES::Time::genesis() {
let leaf: I::Leaf = I::Leaf::from_stored_view(anchor_leaf);
let mut qc = QuorumCertificate::<TYPES, Commitment<I::Leaf>>::genesis();
qc.set_leaf_commitment(leaf.commit());
qc.leaf_commitment = leaf.commit();
let event = Event {
view_number: TYPES::Time::genesis(),
event: EventType::Decide {
Expand Down
3 changes: 3 additions & 0 deletions crates/libp2p-networking/src/network/node/handle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,9 @@ impl<S: Default + Debug> NetworkNodeHandle<S> {
///
/// Will panic if a handler is already spawned
#[allow(clippy::unused_async)]
// // Tokio and async_std disagree how this function should be linted
// #[allow(clippy::ignored_unit_patterns)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also dead code here?


pub async fn spawn_handler<F, RET>(self: &Arc<Self>, cb: F) -> impl Future<Output = ()>
where
F: Fn(NetworkEvent, Arc<NetworkNodeHandle<S>>) -> RET + Sync + Send + 'static,
Expand Down
2 changes: 1 addition & 1 deletion crates/orchestrator/src/config.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use hotshot_types::{ExecutionType, HotShotConfig};
use std::marker::PhantomData;
use std::{
marker::PhantomData,
net::{IpAddr, Ipv4Addr, SocketAddr},
num::NonZeroUsize,
time::Duration,
Expand Down
Loading