Skip to content

Commit

Permalink
Cosmos grpc fallbackprovider (#3139)
Browse files Browse the repository at this point in the history
### Description

Implements grpc fallback provider logic for cosmos

- initially tried implementing the fallback provider deprioritization
logic at middleware level like in the EVM. The difference between ethers
and cosmrs is that in the latter, middleware can only live at the
transport layer (`tower` crate level).
- based on this github
[issue](hyperium/tonic#733), that actually
doesn't look possible, because the http::Request type isn't `Clone` so
it can't be submitted to multiple providers
- ended up implementing the fallback provider at the application layer,
by keeping an array of grpc channels
- There is now a `call` method in `hyperlane_core::FallbackProvider`
which I'm actually really happy with. This method handles the
fallbackprovider-specific logic by taking in an async closure, running
it on each provider, and iterating providers if the closure call fails.
In `grpc.rs` you can see how this is slightly verbose but I think it's
quite manageable. The only part that bugs me is having to duplicate
`Pin::from(Box::from(future))`, but that's need afaict because the
regular closure returns an anonymous type
- adds `grpcUrls` and `customGrpcUrls` config items
- tests the cosmos fallback provider e2e

### Drive-by changes

<!--
Are there any minor or drive-by changes also included?
-->

### Related issues

- Fixes: hyperlane-xyz/issues#998

### Backward compatibility

<!--
Are these changes backward compatible? Are there any infrastructure
implications, e.g. changes that would prohibit deploying older commits
using this infra tooling?

Yes/No
-->

### Testing

<!--
What kind of testing have these changes undergone?

None/Manual/Unit Tests
-->
  • Loading branch information
daniel-savu authored Feb 5, 2024
1 parent 8bfa8c1 commit f2a0e92
Show file tree
Hide file tree
Showing 35 changed files with 766 additions and 311 deletions.
22 changes: 7 additions & 15 deletions rust/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion rust/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ hyper = "0.14"
hyper-tls = "0.5.0"
injective-protobuf = "0.2.2"
injective-std = "0.1.5"
itertools = "0.11.0"
itertools = "*"
jobserver = "=0.1.26"
jsonrpc-core = "18.0"
k256 = { version = "0.13.1", features = ["std", "ecdsa"] }
Expand Down
2 changes: 1 addition & 1 deletion rust/agents/relayer/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ tokio = { workspace = true, features = ["rt", "macros", "parking_lot"] }
tracing-futures.workspace = true
tracing.workspace = true

hyperlane-core = { path = "../../hyperlane-core", features = ["agent"] }
hyperlane-core = { path = "../../hyperlane-core", features = ["agent", "fallback-provider"] }
hyperlane-base = { path = "../../hyperlane-base" }
hyperlane-ethereum = { path = "../../chains/hyperlane-ethereum" }

Expand Down
2 changes: 1 addition & 1 deletion rust/agents/validator/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ tokio = { workspace = true, features = ["rt", "macros", "parking_lot"] }
tracing-futures.workspace = true
tracing.workspace = true

hyperlane-core = { path = "../../hyperlane-core", features = ["agent"] }
hyperlane-core = { path = "../../hyperlane-core", features = ["agent", "fallback-provider"] }
hyperlane-base = { path = "../../hyperlane-base" }
hyperlane-ethereum = { path = "../../chains/hyperlane-ethereum" }
hyperlane-cosmos = { path = "../../chains/hyperlane-cosmos" }
Expand Down
3 changes: 2 additions & 1 deletion rust/chains/hyperlane-cosmos/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ hyper = { workspace = true }
hyper-tls = { workspace = true }
injective-protobuf = { workspace = true }
injective-std = { workspace = true }
itertools = { workspace = true }
once_cell = { workspace = true }
protobuf = { workspace = true }
ripemd = { workspace = true }
Expand All @@ -38,4 +39,4 @@ tracing = { workspace = true }
tracing-futures = { workspace = true }
url = { workspace = true }

hyperlane-core = { path = "../../hyperlane-core" }
hyperlane-core = { path = "../../hyperlane-core", features = ["fallback-provider"]}
7 changes: 7 additions & 0 deletions rust/chains/hyperlane-cosmos/src/error.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use cosmrs::proto::prost;
use hyperlane_core::ChainCommunicationError;
use std::fmt::Debug;

/// Errors from the crates specific to the hyperlane-cosmos
/// implementation.
Expand Down Expand Up @@ -28,6 +29,9 @@ pub enum HyperlaneCosmosError {
/// Tonic error
#[error("{0}")]
Tonic(#[from] tonic::transport::Error),
/// Tonic codegen error
#[error("{0}")]
TonicGenError(#[from] tonic::codegen::StdError),
/// Tendermint RPC Error
#[error(transparent)]
TendermintError(#[from] tendermint_rpc::error::Error),
Expand All @@ -37,6 +41,9 @@ pub enum HyperlaneCosmosError {
/// Protobuf error
#[error("{0}")]
Protobuf(#[from] protobuf::ProtobufError),
/// Fallback providers failed
#[error("Fallback providers failed. (Errors: {0:?})")]
FallbackProvidersFailed(Vec<HyperlaneCosmosError>),
}

impl From<HyperlaneCosmosError> for ChainCommunicationError {
Expand Down
1 change: 1 addition & 0 deletions rust/chains/hyperlane-cosmos/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ mod multisig_ism;
mod payloads;
mod providers;
mod routing_ism;
mod rpc_clients;
mod signers;
mod trait_builder;
mod types;
Expand Down
4 changes: 2 additions & 2 deletions rust/chains/hyperlane-cosmos/src/payloads/aggregate_ism.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct VerifyRequest {
pub verify: VerifyRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct VerifyRequestInner {
pub metadata: String,
pub message: String,
Expand Down
2 changes: 1 addition & 1 deletion rust/chains/hyperlane-cosmos/src/payloads/general.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct EmptyStruct {}

#[derive(Serialize, Deserialize, Debug, Clone)]
Expand Down
12 changes: 6 additions & 6 deletions rust/chains/hyperlane-cosmos/src/payloads/ism_routes.rs
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
use super::general::EmptyStruct;
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct IsmRouteRequest {
pub route: IsmRouteRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct IsmRouteRequestInner {
pub message: String, // hexbinary
}
Expand All @@ -16,22 +16,22 @@ pub struct IsmRouteRespnose {
pub ism: String,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct QueryRoutingIsmGeneralRequest<T> {
pub routing_ism: T,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct QueryRoutingIsmRouteResponse {
pub ism: String,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct QueryIsmGeneralRequest<T> {
pub ism: T,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct QueryIsmModuleTypeRequest {
pub module_type: EmptyStruct,
}
Expand Down
20 changes: 10 additions & 10 deletions rust/chains/hyperlane-cosmos/src/payloads/mailbox.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,52 +3,52 @@ use serde::{Deserialize, Serialize};
use super::general::EmptyStruct;

// Requests
#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct GeneralMailboxQuery<T> {
pub mailbox: T,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct CountRequest {
pub count: EmptyStruct,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct NonceRequest {
pub nonce: EmptyStruct,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct RecipientIsmRequest {
pub recipient_ism: RecipientIsmRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct RecipientIsmRequestInner {
pub recipient_addr: String, // hexbinary
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct DefaultIsmRequest {
pub default_ism: EmptyStruct,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct DeliveredRequest {
pub message_delivered: DeliveredRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct DeliveredRequestInner {
pub id: String, // hexbinary
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct ProcessMessageRequest {
pub process: ProcessMessageRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct ProcessMessageRequestInner {
pub metadata: String,
pub message: String,
Expand Down
8 changes: 4 additions & 4 deletions rust/chains/hyperlane-cosmos/src/payloads/merkle_tree_hook.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,24 @@ use super::general::EmptyStruct;

const TREE_DEPTH: usize = 32;

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct MerkleTreeGenericRequest<T> {
pub merkle_hook: T,
}

// --------- Requests ---------

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct MerkleTreeRequest {
pub tree: EmptyStruct,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct MerkleTreeCountRequest {
pub count: EmptyStruct,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct CheckPointRequest {
pub check_point: EmptyStruct,
}
Expand Down
4 changes: 2 additions & 2 deletions rust/chains/hyperlane-cosmos/src/payloads/multisig_ism.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
use serde::{Deserialize, Serialize};

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct VerifyInfoRequest {
pub verify_info: VerifyInfoRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct VerifyInfoRequestInner {
pub message: String, // hexbinary
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,17 +2,17 @@ use serde::{Deserialize, Serialize};

use super::general::EmptyStruct;

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct GetAnnouncedValidatorsRequest {
pub get_announced_validators: EmptyStruct,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct GetAnnounceStorageLocationsRequest {
pub get_announce_storage_locations: GetAnnounceStorageLocationsRequestInner,
}

#[derive(Serialize, Deserialize, Debug)]
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct GetAnnounceStorageLocationsRequestInner {
pub validators: Vec<String>,
}
Expand Down
Loading

0 comments on commit f2a0e92

Please sign in to comment.