Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(release): v1.0.7-beta #1936

Merged
merged 17 commits into from
Sep 8, 2023
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
d4d5add
feat(adex-cli): Set config file permissions to 660 in unix (#1913)
rozhkovdmitrii Jul 26, 2023
80f7e6f
feat(adex-cli): activation request types (#1912)
rozhkovdmitrii Jul 26, 2023
7b29553
fix(posv): fix missing n_time in posv transactions (#1925)
reddink Aug 2, 2023
e333635
fix(ibc-test): use latest relayer channel for tendermint test (#1929)
onur-ozkan Aug 2, 2023
867a01a
feat(adex-cli): add support for https connection (#1910)
rozhkovdmitrii Aug 2, 2023
54dce3c
fix(endpoints): update prices url (#1928)
smk762 Aug 3, 2023
92372cb
fix(nft): add log_index to history table and use in PK (#1926)
shamardy Aug 7, 2023
3cbb54d
chore(release): bump mm2 version to 1.0.7-beta (#1937)
shamardy Aug 15, 2023
483f04c
feat(trading-proto-upgrade): UTXO PoC + State machine refactor (#1927)
artemii235 Aug 23, 2023
9d5ab11
fix(cli): use the updated activation scheme (#1938)
rozhkovdmitrii Aug 24, 2023
410eda2
fix(kmd): use kmd rewards for fees if change + interest is below dust…
shamardy Aug 24, 2023
e4b091b
chore(contact info update): replace old github username (#1949)
onur-ozkan Aug 28, 2023
51c44f6
fix(hd-wallet): enable/withdraw using any account'/change/address_ind…
shamardy Aug 31, 2023
9a71744
fix(tests): ignore failing tests due to BCHD (#1955)
shamardy Sep 4, 2023
96a53ce
chore(mm2 binary): remove debug info from release binary
onur-ozkan Sep 4, 2023
1538564
feat(zcoin): allow ARRR to sync using a start date (#1922)
borngraced Sep 7, 2023
1b10a06
chore(release): add changelog entries for v1.0.7-beta (#1961)
shamardy Sep 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 75 additions & 0 deletions mm2src/adex_cli/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions mm2src/adex_cli/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,8 @@ derive_more = "0.99"
directories = "5.0"
env_logger = "0.7.1"
http = "0.2"
hyper = { version = "0.14.26", features = ["client", "http2", "tcp"] }
hyper-rustls = "^0.23.0"
gstuff = { version = "=0.7.4" , features = [ "nightly" ]}
inquire = "0.6"
itertools = "0.10"
Expand All @@ -23,14 +25,14 @@ mm2_net = { path = "../mm2_net" }
mm2_number = { path = "../mm2_number" }
mm2_rpc = { path = "../mm2_rpc"}
passwords = "3.1"
rpc = { path = "../mm2_bitcoin/rpc" }
rustls = { version = "^0.20.4", features = [ "dangerous_configuration" ] }
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @DeckerSU @Alrighttt

@rozhkovdmitrii why two diff versions? (we seem using a total of 3 diff across codebase, 0.19.1, 0.20.4 and 0.20.8)

cli lockfile:

[[package]]
name = "rustls"
version = "0.19.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35edb675feee39aec9c99fa5ff985081995a06d594114ae14cbe797ad7b7a6d7"
dependencies = [
 "base64 0.13.1",
 "log 0.4.17",
 "ring",
 "sct 0.6.1",
 "webpki 0.21.4",
]

[[package]]
name = "rustls"
version = "0.20.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fff78fc74d175294f4e83b28343315ffcfb114b156f0185e9741cb5570f50e2f"
dependencies = [
 "log 0.4.17",
 "ring",
 "sct 0.7.0",
 "webpki 0.22.0",
]

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jfyi: dangerous_configuration: this feature enables a dangerous() method on ClientConfig and ServerConfig that allows setting inadvisable options, such as replacing the certificate verification process. Applications requesting this feature should be reviewed carefully.

assume this is for self-signed / local cert handling? cc @shamardy

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

assume this is for self-signed / local cert handling?

Yes. To disable certificate verification from cli side dangerous_configuration has to be used

config
.dangerous()
.set_certificate_verifier(Arc::new(NoCertificateVerification {}));

P.S. dangerous_configuration is not used from the https server side in mm2.

Copy link

@rozhkovdmitrii rozhkovdmitrii Aug 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing it out )

Originally the version of rustls was not constrained and "0.20.8" was used in adex-cli.

Version "0.19.1" is used as subdependency of `mm2_net`
$ cargo tree --manifest-path mm2src/adex_cli/Cargo.toml -i [email protected]
rustls v0.19.1
├── adex-cli v0.1.0 (/home/rozhkov/sources/atomicDEX-API/mm2src/adex_cli)
└── futures-rustls v0.21.1
    └── mm2_core v0.1.0 (/home/rozhkov/sources/atomicDEX-API/mm2src/mm2_core)
        └── mm2_net v0.1.0 (/home/rozhkov/sources/atomicDEX-API/mm2src/mm2_net)
            └── adex-cli v0.1.0 (/home/rozhkov/sources/atomicDEX-API/mm2src/adex_cli)

On 03.08 I had to start using rustls as explicit dependency and I was oriented on using rustls 0.20.4. It was a version which of mm2 was dependent on. Perhaps I had to strongly tie adex-cli on 0.20.4 to be able to manage versions manually.

Now using both version "0.19.1" and "0.20.8" looks appropriate in my honest opinion


Concerning the question why dangerous_configuration feature was utilized - that was to make it able to connect to mm2 that issues self signed certificate.

Thank you

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Concerning the question why dangerous_configuration feature was utilized - that was to make it able to connect to mm2 that issues self signed certificate.

mm2 can be initialized using a certificate file too, it doesn't have to be self-signed but will be in most cases. It would be good to allow the cli user to disable certificate verification themselves like it's done in some other clients (e.g. postman provides this, it doesn't disable it by default). Please open an issue for this @rozhkovdmitrii and it can be done later as it's not urgent at all.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please open an issue for this @rozhkovdmitrii and it can be done later as it's not urgent at all.

done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just wanted to mention something about the versions of hyper-rustls and rustls. From what I understand, the version of rustls that we use should exactly match the version of rustls that hyper-rustls depends on. For example, if we are using hyper-rustls 0.23, we should also use rustls 0.20.8. This is important because if there is a version mismatch, such as using hyper-rustls 0.23 and rustls 0.21.7, we may encounter unexpected errors like below:

note: `ClientConfig` is defined in crate `rustls`
   --> /home/decker/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.21.7/src/client/client_conn.rs:128:1
    |
128 | pub struct ClientConfig {
    | ^^^^^^^^^^^^^^^^^^^^^^^
note: `rustls::client::client_conn::ClientConfig` is defined in crate `rustls`
   --> /home/decker/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.20.8/src/client/client_conn.rs:91:1
    |
91  | pub struct ClientConfig {
    | ^^^^^^^^^^^^^^^^^^^^^^^
    = note: perhaps two different versions of crate `rustls` are being used?

To avoid any potential compatibility issues, it may be better to specify the exact versions of the crates using the = symbol. This way, we can ensure that the versions of hyper-rustls and rustls are precisely matched, reducing the chances of encountering any compatibility problems. Perhaps I may have slightly overestimated the significance of the "issue", but I have personally encountered package version mismatches during some of my own tests. Anyway, JFYI.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for pointing it out 🙏, solved

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rozhkovdmitrii can you please make this fix and the one here #1936 (comment) in a seperate PR? The whole release will be blocked until #1932 is sec reviewed and QA tested otherwise.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

serde = "1.0"
serde_json = { version = "1", features = ["preserve_order", "raw_value"] }
sysinfo = "0.28"
tiny-bip39 = "0.8.0"
tokio = { version = "1.20", features = [ "macros" ] }
uuid = { version = "1.2.2", features = ["fast-rng", "serde", "v4"] }
rpc = { path = "../mm2_bitcoin/rpc" }

[target.'cfg(windows)'.dependencies]
winapi = { version = "0.3.3", features = ["processthreadsapi", "winnt"] }

14 changes: 12 additions & 2 deletions mm2src/adex_cli/src/activation_scheme_db/activation_scheme_impl.rs
Original file line number Diff line number Diff line change
@@ -1,19 +1,29 @@
use anyhow::{anyhow, bail, Result};
use log::{debug, error};
use serde_json::Value as Json;
use std::collections::HashMap;

use common::log::{debug, error};

use super::init_activation_scheme::get_activation_scheme_path;
use crate::helpers::read_json_file;
use crate::logging::{error_anyhow, error_bail};
use crate::rpc_data::ActivationRequest;

#[derive(Default)]
pub(crate) struct ActivationScheme {
scheme: HashMap<String, Json>,
}

impl ActivationScheme {
pub(crate) fn get_activation_method(&self, coin: &str) -> Option<&Json> { self.scheme.get(coin) }
pub(crate) fn get_activation_method(&self, coin: &str) -> Result<ActivationRequest> {
let method_json = self
.scheme
.get(coin)
.ok_or_else(|| error_anyhow!("Coin is not in activation scheme data: {}", coin))?;
let method: ActivationRequest = serde_json::from_value(method_json.clone())
.map_err(|error| error_anyhow!("Failed to deserialize json data: {:?}, error: {}", method_json, error))?;
Ok(method)
}

fn init(&mut self) -> Result<()> {
let mut scheme_source: Vec<Json> = Self::load_json_file()?;
Expand Down
10 changes: 9 additions & 1 deletion mm2src/adex_cli/src/adex_config.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ use std::path::{Path, PathBuf};

use crate::adex_proc::SmartFractPrecision;
use crate::helpers::rewrite_json_file;
#[cfg(unix)] use crate::helpers::set_file_permissions;
use crate::logging::{error_anyhow, warn_bail};

const PROJECT_QUALIFIER: &str = "com";
Expand All @@ -22,6 +23,8 @@ const VOLUME_PRECISION_MIN: usize = 2;
const VOLUME_PRECISION_MAX: usize = 5;
const VOLUME_PRECISION: SmartFractPrecision = (VOLUME_PRECISION_MIN, VOLUME_PRECISION_MAX);
const PRICE_PRECISION: SmartFractPrecision = (PRICE_PRECISION_MIN, PRICE_PRECISION_MAX);
#[cfg(unix)]
const CFG_FILE_PERM_MODE: u32 = 0o660;

pub(super) fn get_config() {
let Ok(adex_cfg) = AdexConfigImpl::from_config_path() else { return; };
Expand Down Expand Up @@ -151,7 +154,12 @@ impl AdexConfigImpl {
let adex_path_str = cfg_path
.to_str()
.ok_or_else(|| error_anyhow!("Failed to get cfg_path as str"))?;
rewrite_json_file(self, adex_path_str)
rewrite_json_file(self, adex_path_str)?;
#[cfg(unix)]
{
set_file_permissions(adex_path_str, CFG_FILE_PERM_MODE)?;
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this advised? Wouldn't it be better/safer to assume config file has correct permission setting as opposed to implementing a de-facto "chmod" into mm2?
Rly concerned over likely unneeded filestream ops and speaking general - against this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I advised him to this as it's creating the config JSON that is used by a separate process, mm2.

There could be a better solution, but without this, the seed will be readable by any user on the system.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my original comment
#1871 (comment)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it was adviced.

The command: adex-cli config set -u http://localhost:77873 -p creates the configuration file and setting permissions could look quite essential.

cc: @Alrighttt

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not totally necessary. Maybe a warning while creating this configuration would suffice since the target audience of this app is presumably power users or at least users familiar with a terminal.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
Ok(())
}

fn set_rpc_password(&mut self, rpc_password: String) { self.rpc_password.replace(rpc_password); }
Expand Down
6 changes: 2 additions & 4 deletions mm2src/adex_cli/src/adex_proc/adex_proc_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ use super::OrderbookConfig;
use crate::activation_scheme_db::get_activation_scheme;
use crate::adex_config::AdexConfig;
use crate::transport::Transport;
use crate::{error_anyhow, error_bail, warn_anyhow, warn_bail};
use crate::{error_anyhow, error_bail, warn_anyhow};

pub(crate) struct AdexProc<'trp, 'hand, 'cfg, T: Transport, H: ResponseHandler, C: AdexConfig + ?Sized> {
pub(crate) transport: Option<&'trp T>,
Expand All @@ -37,9 +37,7 @@ impl<T: Transport, P: ResponseHandler, C: AdexConfig + 'static> AdexProc<'_, '_,
info!("Enabling asset: {asset}");

let activation_scheme = get_activation_scheme()?;
let Some(activation_method) = activation_scheme.get_activation_method(asset) else {
warn_bail!("Asset is not known: {asset}")
};
let activation_method = activation_scheme.get_activation_method(asset)?;

let enable = Command::builder()
.flatten_data(activation_method)
Expand Down
10 changes: 10 additions & 0 deletions mm2src/adex_cli/src/helpers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ use serde::{Deserialize, Serialize};
use std::fs;
use std::io::Write;
use std::ops::Deref;
#[cfg(unix)] use std::os::unix::fs::PermissionsExt;
use std::path::Path;

use crate::error_anyhow;
Expand All @@ -22,6 +23,15 @@ where
writer
.write(&data)
.map_err(|error| error_anyhow!("Failed to write data into {file}: {error}"))?;

Ok(())
}

#[cfg(unix)]
pub(crate) fn set_file_permissions(file: &str, unix_mode: u32) -> Result<()> {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicate feedback from above:

is this advised? Wouldn't it be better/safer to assume config file has correct permission setting as opposed to implementing a de-facto "chmod" into mm2?
Rly concerned over likely unneeded filestream ops and speaking general - against this.

let mut perms = fs::metadata(file)?.permissions();
perms.set_mode(unix_mode);
fs::set_permissions(file, perms)?;
Ok(())
}

Expand Down
1 change: 1 addition & 0 deletions mm2src/adex_cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#[cfg(not(target_arch = "wasm32"))] mod cli;
#[cfg(not(target_arch = "wasm32"))] mod helpers;
mod logging;
#[cfg(not(target_arch = "wasm32"))] mod rpc_data;
#[cfg(not(target_arch = "wasm32"))] mod scenarios;
#[cfg(all(not(target_arch = "wasm32"), test))] mod tests;
#[cfg(not(target_arch = "wasm32"))] mod transport;
Expand Down
88 changes: 88 additions & 0 deletions mm2src/adex_cli/src/rpc_data.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,88 @@
//! Contains rpc data layer structures that are not ready to become a part of the mm2_rpc::data module
//!
//! *Note: it's expected that the following data types will be moved to mm2_rpc::data when mm2 is refactored to be able to handle them*
//!

use mm2_rpc::data::legacy::{ElectrumProtocol, GasStationPricePolicy, UtxoMergeParams};
use serde::ser::SerializeSeq;
use serde::{Deserialize, Serialize, Serializer};

#[derive(Debug, Deserialize, Serialize)]
#[serde(tag = "method", rename_all = "lowercase")]
pub(crate) enum ActivationRequest {
Enable(EnableRequest),
Electrum(ElectrumRequest),
}

#[derive(Debug, Deserialize, Serialize)]
pub(crate) struct EnableRequest {
coin: String,
#[serde(default, serialize_with = "serialize_urls", skip_serializing_if = "Vec::is_empty")]
urls: Vec<EnableUrl>,
#[serde(skip_serializing_if = "Option::is_none")]
swap_contract_address: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
fallback_swap_contract: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
gas_station_url: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
gas_station_decimals: Option<u8>,
#[serde(skip_serializing_if = "Option::is_none")]
gas_station_policy: Option<GasStationPricePolicy>,
#[serde(skip_serializing_if = "Option::is_none")]
mm2: Option<u8>,
#[serde(default)]
tx_history: bool,
#[serde(skip_serializing_if = "Option::is_none")]
required_confirmations: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
requires_notarization: Option<bool>,
#[serde(default)]
contract_supports_watchers: Option<bool>,
}

fn serialize_urls<S>(urls: &Vec<EnableUrl>, s: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let mut s_seq = s.serialize_seq(None)?;
for url in urls {
s_seq.serialize_element(url.url.as_str())?;
}
s_seq.end()
}

#[derive(Debug, Deserialize)]
struct EnableUrl {
url: String,
}

#[derive(Debug, Deserialize, Serialize)]
pub(crate) struct ElectrumRequest {
coin: String,
#[serde(skip_serializing_if = "Vec::is_empty")]
pub(super) servers: Vec<Server>,
#[serde(skip_serializing_if = "Option::is_none")]
mm2: Option<u8>,
#[serde(default)]
tx_history: bool,
#[serde(skip_serializing_if = "Option::is_none")]
required_confirmations: Option<u64>,
#[serde(skip_serializing_if = "Option::is_none")]
requires_notarization: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
swap_contract_address: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
fallback_swap_contract: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
utxo_merge_params: Option<UtxoMergeParams>,
}

#[derive(Debug, Deserialize, Serialize)]
pub(super) struct Server {
url: String,
#[serde(default)]
protocol: ElectrumProtocol,
#[serde(default)]
disable_cert_verification: bool,
}
9 changes: 5 additions & 4 deletions mm2src/adex_cli/src/tests/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ use crate::activation_scheme_db::{get_activation_scheme, get_activation_scheme_p
use crate::adex_config::AdexConfigImpl;
use crate::adex_proc::ResponseHandlerImpl;
use crate::cli::Cli;
use crate::rpc_data::ActivationRequest;

const FAKE_SERVER_COOLDOWN_TIMEOUT_MS: u64 = 10;
const FAKE_SERVER_WARMUP_TIMEOUT_MS: u64 = 100;
Expand Down Expand Up @@ -146,10 +147,10 @@ async fn test_activation_scheme() {
init_activation_scheme().await.unwrap();
let scheme = get_activation_scheme().unwrap();
let kmd_scheme = scheme.get_activation_method("KMD");
assert!(kmd_scheme.is_some());
let kmd_scheme = kmd_scheme.unwrap();
assert_eq!(kmd_scheme.get("method").unwrap().as_str().unwrap(), "electrum");
assert_ne!(kmd_scheme.get("servers").unwrap().as_array().unwrap().iter().count(), 0);
let Ok(ActivationRequest::Electrum(electrum)) = kmd_scheme else {
panic!("Failed to get electrum scheme")
};
assert_ne!(electrum.servers.len(), 0);
}

#[tokio::test]
Expand Down
Loading