Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sdk)!: provide request execution information #2259

Merged
merged 18 commits into from
Oct 28, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
15 changes: 15 additions & 0 deletions packages/rs-dapi-client/src/address_list.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
//! Subsystem to manage DAPI nodes.

use chrono::Utc;
use dapi_grpc::tonic::codegen::http;
use dapi_grpc::tonic::transport::Uri;
use rand::{rngs::SmallRng, seq::IteratorRandom, SeedableRng};
use std::collections::HashSet;
Expand All @@ -20,6 +21,16 @@ pub struct Address {
uri: Uri,
}

impl FromStr for Address {
type Err = AddressListError;

fn from_str(s: &str) -> Result<Self, Self::Err> {
Uri::from_str(s)
.map(Address::from)
.map_err(AddressListError::from)
}
}

impl PartialEq<Self> for Address {
fn eq(&self, other: &Self) -> bool {
self.uri == other.uri
Expand Down Expand Up @@ -81,6 +92,9 @@ impl Address {
pub enum AddressListError {
#[error("address {0} not found in the list")]
AddressNotFound(#[cfg_attr(feature = "mocks", serde(with = "http_serde::uri"))] Uri),
#[error("unable parse address: {0}")]
#[cfg_attr(feature = "mocks", serde(skip))]
InvalidAddressUri(#[from] http::uri::InvalidUri),
}

/// A structure to manage DAPI addresses to select from
Expand Down Expand Up @@ -200,6 +214,7 @@ impl AddressList {
}
}

// TODO: Must be changed to FromStr
shumkov marked this conversation as resolved.
Show resolved Hide resolved
impl From<&str> for AddressList {
fn from(value: &str) -> Self {
let uri_list: Vec<Uri> = value
Expand Down
105 changes: 58 additions & 47 deletions packages/rs-dapi-client/src/dapi_client.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
//! [DapiClient] definition.

use backon::{ExponentialBuilder, Retryable};
use backon::{ConstantBuilder, Retryable};
use dapi_grpc::mock::Mockable;
use dapi_grpc::tonic::async_trait;
use std::fmt::Debug;
use std::sync::atomic::AtomicUsize;
use std::sync::{Arc, RwLock};
use std::time::Duration;
use tracing::Instrument;
Expand All @@ -12,19 +13,17 @@
use crate::connection_pool::ConnectionPool;
use crate::{
transport::{TransportClient, TransportRequest},
Address, AddressList, CanRetry, RequestSettings,
Address, AddressList, CanRetry, DapiRequestExecutor, ExecutionError, ExecutionResponse,

Check failure on line 16 in packages/rs-dapi-client/src/dapi_client.rs

View workflow job for this annotation

GitHub Actions / Rust packages (dash-sdk) / Linting

unresolved import `crate::DapiRequestExecutor`

error[E0432]: unresolved import `crate::DapiRequestExecutor` --> packages/rs-dapi-client/src/dapi_client.rs:16:37 | 16 | Address, AddressList, CanRetry, DapiRequestExecutor, ExecutionError, ExecutionResponse, | ^^^^^^^^^^^^^^^^^^^ no `DapiRequestExecutor` in the root | = help: consider importing this trait instead: crate::executor::DapiRequestExecutor

Check failure on line 16 in packages/rs-dapi-client/src/dapi_client.rs

View workflow job for this annotation

GitHub Actions / Rust packages (rs-dapi-client) / Linting

unresolved import `crate::DapiRequestExecutor`

error[E0432]: unresolved import `crate::DapiRequestExecutor` --> packages/rs-dapi-client/src/dapi_client.rs:16:37 | 16 | Address, AddressList, CanRetry, DapiRequestExecutor, ExecutionError, ExecutionResponse, | ^^^^^^^^^^^^^^^^^^^ no `DapiRequestExecutor` in the root | = help: consider importing this trait instead: crate::executor::DapiRequestExecutor
ExecutionResult, RequestSettings,
};

/// General DAPI request error type.
#[derive(Debug, thiserror::Error)]
#[cfg_attr(feature = "mocks", derive(serde::Serialize, serde::Deserialize))]
pub enum DapiClientError<TE: Mockable> {
/// The error happened on transport layer
#[error("transport error with {1}: {0}")]
Transport(
#[cfg_attr(feature = "mocks", serde(with = "dapi_grpc::mock::serde_mockable"))] TE,
Address,
),
#[error("transport error: {0}")]
Transport(#[cfg_attr(feature = "mocks", serde(with = "dapi_grpc::mock::serde_mockable"))] TE),
/// There are no valid DAPI addresses to use.
#[error("no available addresses to use")]
NoAvailableAddresses,
Expand All @@ -43,7 +42,7 @@
use DapiClientError::*;
match self {
NoAvailableAddresses => false,
Transport(transport_error, _) => transport_error.can_retry(),
Transport(transport_error) => transport_error.can_retry(),
AddressList(_) => false,
#[cfg(feature = "mocks")]
Mock(_) => false,
Expand Down Expand Up @@ -73,21 +72,6 @@
}
}

#[async_trait]
/// DAPI client executor trait.
pub trait DapiRequestExecutor {
/// Execute request using this DAPI client.
async fn execute<R>(
&self,
request: R,
settings: RequestSettings,
) -> Result<R::Response, DapiClientError<<R::Client as TransportClient>::Error>>
where
R: TransportRequest + Mockable,
R::Response: Mockable,
<R::Client as TransportClient>::Error: Mockable;
}

/// Access point to DAPI.
#[derive(Debug, Clone)]
pub struct DapiClient {
Expand Down Expand Up @@ -126,7 +110,7 @@
&self,
request: R,
settings: RequestSettings,
) -> Result<R::Response, DapiClientError<<R::Client as TransportClient>::Error>>
) -> ExecutionResult<R::Response, DapiClientError<<R::Client as TransportClient>::Error>>
where
R: TransportRequest + Mockable,
R::Response: Mockable,
Expand All @@ -140,24 +124,25 @@
.finalize();

// Setup retry policy:
let retry_settings = ExponentialBuilder::default()
let retry_settings = ConstantBuilder::default()
.with_max_times(applied_settings.retries)
// backon doesn't accept 1.0
.with_factor(1.001)
.with_min_delay(Duration::from_secs(0))
.with_max_delay(Duration::from_secs(0));
.with_delay(Duration::from_millis(10));

// Save dump dir for later use, as self is moved into routine
#[cfg(feature = "dump")]
let dump_dir = self.dump_dir.clone();
#[cfg(feature = "dump")]
let dump_request = request.clone();

let retries_counter_arc = Arc::new(AtomicUsize::new(0));
let retries_counter_arc_ref = &retries_counter_arc;

shumkov marked this conversation as resolved.
Show resolved Hide resolved
// Setup DAPI request execution routine future. It's a closure that will be called
// more once to build new future on each retry.
let routine = move || {
// Try to get an address to initialize transport on:
let retries_counter = Arc::clone(retries_counter_arc_ref);

// Try to get an address to initialize transport on:
let address_list = self
.address_list
.read()
Expand Down Expand Up @@ -192,30 +177,29 @@
async move {
// It stays wrapped in `Result` since we want to return
// `impl Future<Output = Result<...>`, not a `Result` itself.
let address = address_result?;
let address = address_result.map_err(|inner| ExecutionError {
inner,
retries: retries_counter.load(std::sync::atomic::Ordering::Acquire),
address: None,
})?;

let pool = self.pool.clone();

let mut transport_client = R::Client::with_uri_and_settings(
address.uri().clone(),
&applied_settings,
&pool,
)
.map_err(|e| {
DapiClientError::<<R::Client as TransportClient>::Error>::Transport(
e,
address.clone(),
)
.map_err(|error| ExecutionError {
inner: DapiClientError::Transport(error),
retries: retries_counter.load(std::sync::atomic::Ordering::Acquire),
address: Some(address.clone()),
})?;

let response = transport_request
.execute_transport(&mut transport_client, &applied_settings)
.await
.map_err(|e| {
DapiClientError::<<R::Client as TransportClient>::Error>::Transport(
e,
address.clone(),
)
});
.map_err(DapiClientError::Transport);

match &response {
Ok(_) => {
Expand All @@ -226,8 +210,14 @@
.write()
.expect("can't get address list for write");

address_list.unban_address(&address)
.map_err(DapiClientError::<<R::Client as TransportClient>::Error>::AddressList)?;
address_list.unban_address(&address).map_err(|error| {
ExecutionError {
inner: DapiClientError::AddressList(error),
retries: retries_counter
.load(std::sync::atomic::Ordering::Acquire),
address: Some(address.clone()),
}
})?;
}

tracing::trace!(?response, "received {} response", response_name);
Expand All @@ -240,16 +230,34 @@
.write()
.expect("can't get address list for write");

address_list.ban_address(&address)
.map_err(DapiClientError::<<R::Client as TransportClient>::Error>::AddressList)?;
address_list.ban_address(&address).map_err(|error| {
ExecutionError {
inner: DapiClientError::AddressList(error),
retries: retries_counter
.load(std::sync::atomic::Ordering::Acquire),
address: Some(address.clone()),
}
})?;
}
} else {
tracing::trace!(?error, "received error");
}
}
};

let retries = retries_counter.load(std::sync::atomic::Ordering::Acquire);

response
.map(|inner| ExecutionResponse {
inner,
retries,
address: address.clone(),
})
.map_err(|inner| ExecutionError {
inner,
retries,
address: Some(address),
})
}
};

Expand All @@ -258,11 +266,14 @@
let result = routine
.retry(retry_settings)
.notify(|error, duration| {
let retries_counter = Arc::clone(&retries_counter_arc);
retries_counter.fetch_add(1, std::sync::atomic::Ordering::AcqRel);

tracing::warn!(
?error,
"retrying error with sleeping {} secs",
duration.as_secs_f32()
)
);
})
.when(|e| e.can_retry())
.instrument(tracing::info_span!("request routine"))
Expand Down
66 changes: 66 additions & 0 deletions packages/rs-dapi-client/src/executor.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
use crate::transport::{TransportClient, TransportRequest};
use crate::{Address, CanRetry, DapiClientError, RequestSettings};
use dapi_grpc::mock::Mockable;
use dapi_grpc::tonic::async_trait;
use std::fmt::Debug;

#[async_trait]
/// DAPI client executor trait.
pub trait DapiRequestExecutor {
/// Execute request using this DAPI client.
async fn execute<R>(
&self,
request: R,
settings: RequestSettings,
) -> ExecutionResult<R::Response, DapiClientError<<R::Client as TransportClient>::Error>>
where
R: TransportRequest + Mockable,
R::Response: Mockable,
<R::Client as TransportClient>::Error: Mockable;
}

/// Error happened during request execution.
#[derive(Debug, Clone, thiserror::Error, Eq, PartialEq)]
#[error("{inner}")]
shumkov marked this conversation as resolved.
Show resolved Hide resolved
pub struct ExecutionError<E> {
/// The cause of error
pub inner: E,
/// How many times the request was retried
pub retries: usize,
/// The address of the node that was used for the request
pub address: Option<Address>,
}

impl<E> ExecutionError<E> {
/// Unwrap the error cause
pub fn into_inner(self) -> E {
self.inner
}
}

impl<E: CanRetry> CanRetry for ExecutionError<E> {
fn can_retry(&self) -> bool {
self.inner.can_retry()
}
}

/// Request execution response.
#[derive(Debug, Clone, Eq, PartialEq)]
pub struct ExecutionResponse<R> {
/// The response from the request
pub inner: R,
/// How many times the request was retried
pub retries: usize,
/// The address of the node that was used for the request
pub address: Address,
}
shumkov marked this conversation as resolved.
Show resolved Hide resolved

impl<R> ExecutionResponse<R> {
/// Unwrap the response
pub fn into_inner(self) -> R {
self.inner
}
}

/// Result of request execution
pub type ExecutionResult<R, E> = Result<ExecutionResponse<R>, ExecutionError<E>>;
11 changes: 6 additions & 5 deletions packages/rs-dapi-client/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
mod dapi_client;
#[cfg(feature = "dump")]
pub mod dump;
mod executor;
#[cfg(feature = "mocks")]
pub mod mock;
mod request_settings;
Expand All @@ -15,26 +16,26 @@
pub use address_list::Address;
pub use address_list::AddressList;
pub use connection_pool::ConnectionPool;
pub use dapi_client::DapiRequestExecutor;
pub use dapi_client::{DapiClient, DapiClientError};
use dapi_grpc::mock::Mockable;
#[cfg(feature = "dump")]
pub use dump::DumpData;
pub use executor::{ExecutionError, ExecutionResponse, ExecutionResult};
use futures::{future::BoxFuture, FutureExt};
pub use request_settings::RequestSettings;

/// A DAPI request could be executed with an initialized [DapiClient].
///
/// # Examples
/// ```
/// use rs_dapi_client::{RequestSettings, AddressList, mock::MockDapiClient, DapiClientError, DapiRequest};
/// use rs_dapi_client::{RequestSettings, AddressList, mock::MockDapiClient, DapiClientError, DapiRequest, ExecutionError};
/// use dapi_grpc::platform::v0::{self as proto};
///
/// # let _ = async {
/// let mut client = MockDapiClient::new();
/// let request: proto::GetIdentityRequest = proto::get_identity_request::GetIdentityRequestV0 { id: b"0".to_vec(), prove: true }.into();
/// let response = request.execute(&mut client, RequestSettings::default()).await?;
/// # Ok::<(), DapiClientError<_>>(())
/// # Ok::<(), ExecutionError<DapiClientError<_>>>(())
/// # };
/// ```
pub trait DapiRequest {
Expand All @@ -45,10 +46,10 @@

/// Executes the request.
fn execute<'c, D: DapiRequestExecutor>(
self,

Check failure on line 49 in packages/rs-dapi-client/src/lib.rs

View workflow job for this annotation

GitHub Actions / Rust packages (dash-sdk) / Linting

cannot find trait `DapiRequestExecutor` in this scope

error[E0405]: cannot find trait `DapiRequestExecutor` in this scope --> packages/rs-dapi-client/src/lib.rs:49:23 | 49 | fn execute<'c, D: DapiRequestExecutor>( | ^^^^^^^^^^^^^^^^^^^ not found in this scope | help: consider importing this trait | 16 + use crate::executor::DapiRequestExecutor; |

Check failure on line 49 in packages/rs-dapi-client/src/lib.rs

View workflow job for this annotation

GitHub Actions / Rust packages (rs-dapi-client) / Linting

cannot find trait `DapiRequestExecutor` in this scope

error[E0405]: cannot find trait `DapiRequestExecutor` in this scope --> packages/rs-dapi-client/src/lib.rs:49:23 | 49 | fn execute<'c, D: DapiRequestExecutor>( | ^^^^^^^^^^^^^^^^^^^ not found in this scope | help: consider importing this trait | 16 + use crate::executor::DapiRequestExecutor; |
dapi_client: &'c D,
settings: RequestSettings,
) -> BoxFuture<'c, Result<Self::Response, DapiClientError<Self::TransportError>>>
) -> BoxFuture<'c, ExecutionResult<Self::Response, DapiClientError<Self::TransportError>>>
where
shumkov marked this conversation as resolved.
Show resolved Hide resolved
Self: 'c;
}
Expand All @@ -60,10 +61,10 @@
type TransportError = <T::Client as transport::TransportClient>::Error;

fn execute<'c, D: DapiRequestExecutor>(
self,

Check failure on line 64 in packages/rs-dapi-client/src/lib.rs

View workflow job for this annotation

GitHub Actions / Rust packages (dash-sdk) / Linting

cannot find trait `DapiRequestExecutor` in this scope

error[E0405]: cannot find trait `DapiRequestExecutor` in this scope --> packages/rs-dapi-client/src/lib.rs:64:23 | 64 | fn execute<'c, D: DapiRequestExecutor>( | ^^^^^^^^^^^^^^^^^^^ not found in this scope | help: consider importing this trait | 16 + use crate::executor::DapiRequestExecutor; |

Check failure on line 64 in packages/rs-dapi-client/src/lib.rs

View workflow job for this annotation

GitHub Actions / Rust packages (rs-dapi-client) / Linting

cannot find trait `DapiRequestExecutor` in this scope

error[E0405]: cannot find trait `DapiRequestExecutor` in this scope --> packages/rs-dapi-client/src/lib.rs:64:23 | 64 | fn execute<'c, D: DapiRequestExecutor>( | ^^^^^^^^^^^^^^^^^^^ not found in this scope | help: consider importing this trait | 16 + use crate::executor::DapiRequestExecutor; |
dapi_client: &'c D,
settings: RequestSettings,
) -> BoxFuture<'c, Result<Self::Response, DapiClientError<Self::TransportError>>>
) -> BoxFuture<'c, ExecutionResult<Self::Response, DapiClientError<Self::TransportError>>>
where
Self: 'c,
{
Expand Down
Loading
Loading