Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identity E2E integration #755

Merged
merged 28 commits into from
May 21, 2024
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
0370df4
Get tests passing
neekolas May 19, 2024
94eb67e
Merge branch 'nm/group-updated-codec' into nm/end-to-end-integration
neekolas May 19, 2024
5412f0a
Use new group updated codec
neekolas May 19, 2024
9714d5b
More loose ends
neekolas May 20, 2024
679c7e1
Merge branch 'identity-release' into nm/end-to-end-integration
neekolas May 20, 2024
9c8eb36
More tests passing
neekolas May 20, 2024
649a305
Remove a few more todos
neekolas May 20, 2024
b6a9376
Fix up bindings
neekolas May 20, 2024
40422fc
Fix up bindings
neekolas May 20, 2024
20205fc
Fix up subscriptions tests
neekolas May 20, 2024
f75b4a1
Add hack for unexpected installations
neekolas May 21, 2024
d4e43d6
Fix issue with group creation
neekolas May 21, 2024
f8803fb
Fix lints
neekolas May 21, 2024
70d1353
Use specific docker image
neekolas May 21, 2024
39d8d92
Fix commit validation logic
neekolas May 21, 2024
90b8b5a
Update tests since we now have transcript messages
neekolas May 21, 2024
529dfa1
Unignore more tests
neekolas May 21, 2024
1eb5bd7
Lower concurrency
neekolas May 21, 2024
6050364
Change test-threads
neekolas May 21, 2024
b266c12
Lint
neekolas May 21, 2024
8fce7cc
Fix args
neekolas May 21, 2024
3593d81
Ignore failing test
neekolas May 21, 2024
1ef0b48
Update log line
neekolas May 21, 2024
c1702d1
Un-ignore test
neekolas May 21, 2024
b1c5f8d
Update docker image
neekolas May 21, 2024
2c49988
Lower number of test threads
neekolas May 21, 2024
ff8ce36
Lower number of test threads in bindings tests
neekolas May 21, 2024
b6e7366
Ignore flaky test
neekolas May 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 7 additions & 8 deletions bindings_ffi/src/mls.rs
Original file line number Diff line number Diff line change
Expand Up @@ -512,15 +512,14 @@ impl FfiGroup {
Ok(group.is_active()?)
}

// TODO: This should be `added_by_inbox_id`
pub fn added_by_address(&self) -> Result<String, GenericError> {
pub fn added_by_inbox_id(&self) -> Result<String, GenericError> {
let group = MlsGroup::new(
self.inner_client.context().clone(),
self.group_id.clone(),
self.created_at_ns,
);

Ok(group.added_by_address()?)
Ok(group.added_by_inbox_id()?)
}

pub fn group_metadata(&self) -> Result<Arc<FfiGroupMetadata>, GenericError> {
Expand Down Expand Up @@ -652,8 +651,8 @@ pub struct FfiGroupMetadata {

#[uniffi::export]
impl FfiGroupMetadata {
pub fn creator_account_address(&self) -> String {
self.inner.creator_account_address.clone()
pub fn creator_inbox_id(&self) -> String {
self.inner.creator_inbox_id.clone()
}

pub fn conversation_type(&self) -> String {
Expand Down Expand Up @@ -1249,13 +1248,13 @@ mod tests {

let bola_group = bola_groups.first().unwrap();

// Check Bola's group for the added_by_address of the inviter
let added_by_address = bola_group.added_by_address().unwrap();
// Check Bola's group for the added_by_inbox_id of the inviter
let added_by_inbox_id = bola_group.added_by_inbox_id().unwrap();

// // Verify the welcome host_credential is equal to Amal's
assert_eq!(
amal.siganture_request().unwrap(),
added_by_address,
added_by_inbox_id,
"The Inviter and added_by_address do not match!"
);
}
Expand Down
4 changes: 2 additions & 2 deletions dev/docker/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
services:
node:
image: xmtp/node-go:latest
platform: linux/amd64
image: xmtp/node-go:dev
# platform: linux/amd64
environment:
- GOWAKU-NODEKEY=8a30dcb604b0b53627a5adc054dbf434b446628d4bd1eccc681d223f0550ce67
command:
Expand Down
4 changes: 2 additions & 2 deletions examples/cli/serializable.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use xmtp_proto::xmtp::mls::message_contents::EncodedContent;

#[derive(Serialize, Debug)]
pub struct SerializableGroupMetadata {
creator_account_address: String,
creator_inbox_id: String,
policy: String,
}

Expand Down Expand Up @@ -37,7 +37,7 @@ impl<'a> From<&'a MlsGroup> for SerializableGroup {
group_id,
members,
metadata: SerializableGroupMetadata {
creator_account_address: metadata.creator_account_address.clone(),
creator_inbox_id: metadata.creator_inbox_id.clone(),
policy: permissions
.preconfigured_policy()
.expect("could not get policy")
Expand Down
3 changes: 3 additions & 0 deletions xmtp_mls/src/api/identity.rs
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,7 @@ where
&self,
account_addresses: Vec<String>,
) -> Result<AddressToInboxIdMap, WrappedApiError> {
log::info!("Asked for account addresses: {:?}", &account_addresses);
let result = self
.api_client
.get_inbox_ids(GetInboxIdsRequest {
Expand All @@ -119,6 +120,8 @@ where
})
.await?;

log::info!("Got result: {:?}", &result);

Ok(result
.responses
.into_iter()
Expand Down
41 changes: 23 additions & 18 deletions xmtp_mls/src/builder.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,24 +92,19 @@ where
.take()
.ok_or(ClientBuilderError::MissingParameter { parameter: "store" })?;
debug!("Initializing identity");
let mut identity = self
let identity = self
.identity_strategy
.initialize_identity(&api_client_wrapper, &store)
.await?;

// get sequence_id from identity updates
let updates = load_identity_updates(
// get sequence_id from identity updates loaded into the DB
load_identity_updates(
&api_client_wrapper,
&store.conn()?,
vec![identity.clone().inbox_id],
)
.await?;

let sequence_id = updates
.get(&identity.inbox_id)
.and_then(|updates| updates.last())
.map_or(0, |update| update.sequence_id);
identity.set_sequence_id(sequence_id);
Ok(Client::new(api_client_wrapper, identity, store))
}
}
Expand All @@ -118,6 +113,7 @@ where
mod tests {
use xmtp_api_grpc::grpc_api_helper::Client as GrpcClient;
use xmtp_cryptography::utils::generate_local_wallet;
use xmtp_id::associations::RecoverableEcdsaSignature;

use super::{ClientBuilder, IdentityStrategy};
use crate::{
Expand All @@ -132,6 +128,20 @@ mod tests {
.unwrap()
}

async fn register_client(client: &Client<GrpcClient>, owner: &impl InboxOwner) {
let mut signature_request = client.context.signature_request().unwrap();
let signature_text = signature_request.signature_text();
signature_request
.add_signature(Box::new(RecoverableEcdsaSignature::new(
signature_text.clone(),
owner.sign(&signature_text).unwrap().into(),
)))
.await
.unwrap();

client.register_identity(signature_request).await.unwrap();
}

impl ClientBuilder<GrpcClient> {
pub async fn local_grpc(self) -> Self {
self.api_client(get_local_grpc_client().await)
Expand All @@ -155,10 +165,8 @@ mod tests {
.build()
.await
.unwrap();
// TODO: Add signature
// let signature: Option<Vec<u8>> =
// client.get_signature_request().unwrap().add_signature();
// client.register_identity(signature).await.unwrap();

register_client(&client, owner).await;

client
}
Expand All @@ -172,7 +180,6 @@ mod tests {
}

#[tokio::test]
#[ignore]
async fn identity_persistence_test() {
let tmpdb = tmp_path();
let wallet = &generate_local_wallet();
Expand All @@ -192,11 +199,9 @@ mod tests {
.build()
.await
.unwrap();
// TODO: finish signature
// let signature: Option<Vec<u8>> = client_a
// .text_to_sign()
// .map(|text| wallet.sign(&text).unwrap().into());
// client_a.register_identity(signature).await.unwrap(); // Persists the identity on registration

register_client(&client_a, wallet).await;

let keybytes_a = client_a.installation_public_key();
drop(client_a);

Expand Down
93 changes: 43 additions & 50 deletions xmtp_mls/src/client.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
use std::{collections::HashMap, collections::HashSet, mem::Discriminant, sync::Arc};
use std::{
collections::{HashMap, HashSet},
mem::Discriminant,
sync::Arc,
};

use openmls::{
credentials::errors::BasicCredentialError,
Expand Down Expand Up @@ -28,18 +32,17 @@
use crate::{
api::{ApiClientWrapper, IdentityUpdate},
groups::{
validated_commit::CommitValidationError, AddressesOrInstallationIds, IntentError, MlsGroup,
PreconfiguredPolicies,
validated_commit::CommitValidationError, IntentError, MlsGroup, PreconfiguredPolicies,
},
identity::Identity,
identity_updates::{load_identity_updates, IdentityUpdateError},
identity::{parse_credential, Identity, IdentityError},
identity_updates::IdentityUpdateError,
storage::{
db_connection::DbConnection,
group::{GroupMembershipState, StoredGroup},
refresh_state::EntityKind,
EncryptedMessageStore, StorageError,
},
verified_key_package::{KeyPackageVerificationError, VerifiedKeyPackage},
verified_key_package_v2::{KeyPackageVerificationError, VerifiedKeyPackageV2},
xmtp_openmls_provider::XmtpOpenMlsProvider,
Fetch, XmtpApi,
};
Expand Down Expand Up @@ -101,6 +104,8 @@
},
#[error("invalid payload")]
InvalidPayload,
#[error(transparent)]
Identity(#[from] IdentityError),
#[error("openmls process message error: {0}")]
OpenMlsProcessMessage(#[from] openmls::prelude::ProcessMessageError),
#[error("merge pending commit: {0}")]
Expand Down Expand Up @@ -192,8 +197,8 @@
}

/// Get sequence id, may not be consistent with the backend
pub fn inbox_sequence_id(&self) -> u64 {
self.identity.sequence_id
pub fn inbox_sequence_id(&self, conn: &DbConnection) -> Result<i64, StorageError> {
self.identity.sequence_id(conn)
}

/// Integrators should always check the `signature_request` return value of this function before calling [`register_identity`](Self::register_identity).
Expand Down Expand Up @@ -226,11 +231,6 @@
}
}

pub fn account_address(&self) -> String {
// TODO:nm Remove this hack
self.inbox_id()
}

pub fn installation_public_key(&self) -> Vec<u8> {
self.context.installation_public_key()
}
Expand All @@ -240,8 +240,8 @@
}

/// Get sequence id, may not be consistent with the backend
pub fn inbox_sequence_id(&self) -> u64 {
self.context.inbox_sequence_id()
pub fn inbox_sequence_id(&self, conn: &DbConnection) -> Result<i64, StorageError> {
self.context.inbox_sequence_id(conn)
37ng marked this conversation as resolved.
Show resolved Hide resolved
}

pub fn store(&self) -> &EncryptedMessageStore {
Expand Down Expand Up @@ -271,7 +271,6 @@
self.context.clone(),
GroupMembershipState::Allowed,
permissions,
&self.context.inbox_id(),
)
.map_err(|e| {
ClientError::Storage(StorageError::Store(format!("group create error {}", e)))
Expand Down Expand Up @@ -343,13 +342,13 @@
signature_request: SignatureRequest,
) -> Result<(), ClientError> {
log::info!("registering identity");
let connection = self.store().conn()?;
let provider = self.mls_provider(connection.clone());
self.apply_signature_request(signature_request).await?;
load_identity_updates(&self.api_client, &connection, vec![self.inbox_id()]).await?;
let connection = self.store().conn()?;
let provider = self.mls_provider(connection);
self.identity()
.register(&provider, &self.api_client)
.await?;

Ok(())
}

Expand Down Expand Up @@ -452,46 +451,17 @@
})
}

pub(crate) async fn get_key_packages(
&self,
address_or_id: AddressesOrInstallationIds,
) -> Result<Vec<VerifiedKeyPackage>, ClientError> {
match address_or_id {
AddressesOrInstallationIds::AccountAddresses(addrs) => {
self.get_key_packages_for_account_addresses(addrs).await
}
AddressesOrInstallationIds::InstallationIds(ids) => {
self.get_key_packages_for_installation_ids(ids).await
}
}
}

// Get a flat list of one key package per installation for all the wallet addresses provided.
// Revoked installations will be omitted from the list
#[allow(dead_code)]
pub(crate) async fn get_key_packages_for_account_addresses(
&self,
account_addresses: Vec<String>,
) -> Result<Vec<VerifiedKeyPackage>, ClientError> {
let installation_ids = self
.get_all_active_installation_ids(account_addresses)
.await?;

self.get_key_packages_for_installation_ids(installation_ids)
.await
}

pub(crate) async fn get_key_packages_for_installation_ids(
&self,
installation_ids: Vec<Vec<u8>>,
) -> Result<Vec<VerifiedKeyPackage>, ClientError> {
) -> Result<Vec<VerifiedKeyPackageV2>, ClientError> {
let key_package_results = self.api_client.fetch_key_packages(installation_ids).await?;

let conn = self.store().conn()?;
let mls_provider = self.mls_provider(conn);
Ok(key_package_results
.values()
.map(|bytes| VerifiedKeyPackage::from_bytes(mls_provider.crypto(), bytes.as_slice()))
.map(|bytes| VerifiedKeyPackageV2::from_bytes(mls_provider.crypto(), bytes.as_slice()))
.collect::<Result<_, _>>()?)
}

Expand Down Expand Up @@ -534,6 +504,29 @@
Ok(groups)
}

/**
* Validates a credential against the given installation public key
*
* This will go to the network and get the latest association state for the inbox.
* It ensures that the installation_pub_key is in that association state
*/
pub async fn validate_credential_against_network(
&self,
conn: &DbConnection,
credential: &[u8],
installation_pub_key: Vec<u8>,
) -> Result<InboxId, ClientError> {
let inbox_id = parse_credential(credential)?;
let association_state = self.get_latest_association_state(conn, &inbox_id).await?;

match association_state.get(&installation_pub_key.clone().into()) {
Some(_) => {
return Ok(inbox_id);

Check warning on line 524 in xmtp_mls/src/client.rs

View workflow job for this annotation

GitHub Actions / workspace

unneeded `return` statement

warning: unneeded `return` statement --> xmtp_mls/src/client.rs:524:17 | 524 | return Ok(inbox_id); | ^^^^^^^^^^^^^^^^^^^ | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return = note: `#[warn(clippy::needless_return)]` on by default help: remove `return` | 524 - return Ok(inbox_id); 524 + Ok(inbox_id) |
}
None => return Err(IdentityError::InstallationIdNotFound(inbox_id).into()),

Check warning on line 526 in xmtp_mls/src/client.rs

View workflow job for this annotation

GitHub Actions / workspace

unneeded `return` statement

warning: unneeded `return` statement --> xmtp_mls/src/client.rs:526:21 | 526 | None => return Err(IdentityError::InstallationIdNotFound(inbox_id).into()), | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#needless_return help: remove `return` | 526 | None => Err(IdentityError::InstallationIdNotFound(inbox_id).into()), | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
}
}

/// Check whether an account_address has a key package registered on the network
///
/// Arguments:
Expand Down
2 changes: 1 addition & 1 deletion xmtp_mls/src/codecs/mod.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
pub mod group_updated;
pub mod membership_change;
pub mod text;
mod group_updated;

use thiserror::Error;

Expand Down
1 change: 0 additions & 1 deletion xmtp_mls/src/configuration.rs
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
use openmls::versions::ProtocolVersion;
use openmls_traits::types::Ciphersuite;

// TODO confirm ciphersuite choice
pub const CIPHERSUITE: Ciphersuite =
Ciphersuite::MLS_128_DHKEMX25519_CHACHA20POLY1305_SHA256_Ed25519;

Expand Down
Loading
Loading