-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add merkleization of DID documents #492
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So we build a custom merkle tree which contains only the keys of a DID?
Would it make sense to already include the whole DID document in here? If there are future use cases we would always need to come up with a custom merkle tree. I think I have a generalized DID Merkle Tree in mind where you could prove all info from the KILT chain.
@@ -45,13 +45,27 @@ impl<Identifier, Proof, Details> From<v1::IdentityProofAction<Identifier, Proof, | |||
} | |||
|
|||
#[derive(Encode, Decode, RuntimeDebug, Clone, Eq, PartialEq, TypeInfo)] | |||
pub enum VersionedIdentityProof<LeafKey, LeafValue> { | |||
pub enum VersionedIdentityProof<BlindedValue, Leaf> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe this should be marked as #[non_exhaustive]
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The idea is that consumers would try to convert their own version into this for message passing, and receivers would try to convert it back into a specific version. I took inspiration for how VersionedXcm works. Why do you think we need non_exhaustive
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking how a new version would be introduced. What happens if a VersionedIdentityProof::v2 get's send to someone that only supports v1? at the moment that would result in a decoding error. Adding non_exhaustive
wouldn't change anything here.
But if you use the dip
crate as a dependency and use the VersionedIdentityProof
having non_exhaustive
makes a difference. If VersionedIdentityProof
is not marked as non_exhaustive
, adding a new version is a breaking change (since match VersionedIdentityProof {}
doesn't cover this new case). Having non_exhaustive
makes adding new versions a non-breaking change since everyone is forced to already have code in place that covers new versions (and showing an error that this version is not supported).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. But isnt'a a compilation error what we actually want here? If you risk getting a versioned identity proof you don't support, you should probably return an error. Code won't need to be updated if you are using the old version, hence anything failing to decode will result in an error. Once you update the dependency, you should be at least aware that a new version could be around, and explicitly generate an error if it's not what you expect. Or am I still missing your point?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The non_exhaustive
is only for compile time. If we introduce a new version this would be a breaking change. Since it fails to compile.
At runtime this doesn't change anything since if we talk to a newer version that we don't know, it fails to decode and won't reach any match statement where we check versions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
343faf9. Let's see how this plays out 😄
9e24bbf
to
fc58382
Compare
Fixes KILTprotocol/ticket#2557 and fixes KILTprotocol/ticket#2556. This PR builds on top of the shell PR, and adds support for Merkle proof for DID documents. ## Merkle proof structure A DID merkle proof is, at the core, an order set of (key, value) pairs, on which proof-of-inclusion and proof-of-non-inclusion can be performed. This PR generates and validates merkle proofs where leaves are of two types: - a DID key reference leave, whose key is the tuple (key ID, key relationship) and the value is an empty tuple - a DID key details leave, whose key is the key ID and the value is the key details For each key reference leaf with a given key ID, the proof also has to contain a key details leaf whose key is the key ID. Multiple reference leaves can reference the same key details leaf, optimising the storage size. ## New runtime APIs There is a new runtime API which the DIP sender would expose, and that allows anyone to generate a merkle proof for a given DID identifier and set of key IDs. The result contains the merkle root (which must match what other chains have stored in their `pallet-dip-receiver` map), and a merkle proof, which includes blinded values and a set of key reference and key details leaves for the keys identified by the provided key IDs. ## How to test The setup flow is similar to that of #489. Specifically: - Set up the local Rococo network onboarding the sender and receiver chains with para IDs 2_000 and 2_001 respectively - Open an HRMP channel from sender 2_000 to receiver 2_001 - Create a DID on the sender chain, e.g., using the [kilt-did-utilities](https://github.com/KILTprotocol/kilt-did-utilities) tool ![Screenshot 2023-03-27 at 10 00 48](https://user-images.githubusercontent.com/6704504/227900994-41f0f355-84bd-4b8a-a2a8-3a9c74447e59.png) - Push the identity of the DID to the receiver chain via the `pallet-dip-sender` extrinsic ![Screenshot 2023-03-27 at 10 01 13](https://user-images.githubusercontent.com/6704504/227901150-7e8c9c9d-8aac-4739-8ad3-fad4ba6ff5f8.png) - Call the runtime API to generate a proof for the created DID with some keys revealed ![Screenshot 2023-03-27 at 10 01 40](https://user-images.githubusercontent.com/6704504/227901309-94b4dbc9-ca83-4541-820d-d1bec6adc6f0.png) - Use the generated proof to dispatch an extrinsic on the receiving chain ### How to use the runtime API with polkadot apps There is currently no support for the new runtime API in the public polkadot apps instance. To use the runtime APIs from UI, please use [our fork from the aa/dip-sender-template branch](https://github.com/KILTprotocol/polkadot-apps/tree/aa/dip-sender-template), by running `yarn && yarn build && yarn start`, then connecting to the sender node WS socket. For runtime augmentation within a Node script, please use our [SDK repo from the aa/dip-merkle-proof branch](https://github.com/KILTprotocol/sdk-js/tree/aa/dip-merkle-proof).
Fixes KILTprotocol/ticket#2557 and fixes KILTprotocol/ticket#2556. This PR builds on top of the shell PR, and adds support for Merkle proof for DID documents. ## Merkle proof structure A DID merkle proof is, at the core, an order set of (key, value) pairs, on which proof-of-inclusion and proof-of-non-inclusion can be performed. This PR generates and validates merkle proofs where leaves are of two types: - a DID key reference leave, whose key is the tuple (key ID, key relationship) and the value is an empty tuple - a DID key details leave, whose key is the key ID and the value is the key details For each key reference leaf with a given key ID, the proof also has to contain a key details leaf whose key is the key ID. Multiple reference leaves can reference the same key details leaf, optimising the storage size. ## New runtime APIs There is a new runtime API which the DIP sender would expose, and that allows anyone to generate a merkle proof for a given DID identifier and set of key IDs. The result contains the merkle root (which must match what other chains have stored in their `pallet-dip-receiver` map), and a merkle proof, which includes blinded values and a set of key reference and key details leaves for the keys identified by the provided key IDs. ## How to test The setup flow is similar to that of #489. Specifically: - Set up the local Rococo network onboarding the sender and receiver chains with para IDs 2_000 and 2_001 respectively - Open an HRMP channel from sender 2_000 to receiver 2_001 - Create a DID on the sender chain, e.g., using the [kilt-did-utilities](https://github.com/KILTprotocol/kilt-did-utilities) tool ![Screenshot 2023-03-27 at 10 00 48](https://user-images.githubusercontent.com/6704504/227900994-41f0f355-84bd-4b8a-a2a8-3a9c74447e59.png) - Push the identity of the DID to the receiver chain via the `pallet-dip-sender` extrinsic ![Screenshot 2023-03-27 at 10 01 13](https://user-images.githubusercontent.com/6704504/227901150-7e8c9c9d-8aac-4739-8ad3-fad4ba6ff5f8.png) - Call the runtime API to generate a proof for the created DID with some keys revealed ![Screenshot 2023-03-27 at 10 01 40](https://user-images.githubusercontent.com/6704504/227901309-94b4dbc9-ca83-4541-820d-d1bec6adc6f0.png) - Use the generated proof to dispatch an extrinsic on the receiving chain ### How to use the runtime API with polkadot apps There is currently no support for the new runtime API in the public polkadot apps instance. To use the runtime APIs from UI, please use [our fork from the aa/dip-sender-template branch](https://github.com/KILTprotocol/polkadot-apps/tree/aa/dip-sender-template), by running `yarn && yarn build && yarn start`, then connecting to the sender node WS socket. For runtime augmentation within a Node script, please use our [SDK repo from the aa/dip-merkle-proof branch](https://github.com/KILTprotocol/sdk-js/tree/aa/dip-merkle-proof).
Fixes KILTprotocol/ticket#2557 and fixes KILTprotocol/ticket#2556. This PR builds on top of the shell PR, and adds support for Merkle proof for DID documents. ## Merkle proof structure A DID merkle proof is, at the core, an order set of (key, value) pairs, on which proof-of-inclusion and proof-of-non-inclusion can be performed. This PR generates and validates merkle proofs where leaves are of two types: - a DID key reference leave, whose key is the tuple (key ID, key relationship) and the value is an empty tuple - a DID key details leave, whose key is the key ID and the value is the key details For each key reference leaf with a given key ID, the proof also has to contain a key details leaf whose key is the key ID. Multiple reference leaves can reference the same key details leaf, optimising the storage size. ## New runtime APIs There is a new runtime API which the DIP sender would expose, and that allows anyone to generate a merkle proof for a given DID identifier and set of key IDs. The result contains the merkle root (which must match what other chains have stored in their `pallet-dip-receiver` map), and a merkle proof, which includes blinded values and a set of key reference and key details leaves for the keys identified by the provided key IDs. ## How to test The setup flow is similar to that of #489. Specifically: - Set up the local Rococo network onboarding the sender and receiver chains with para IDs 2_000 and 2_001 respectively - Open an HRMP channel from sender 2_000 to receiver 2_001 - Create a DID on the sender chain, e.g., using the [kilt-did-utilities](https://github.com/KILTprotocol/kilt-did-utilities) tool ![Screenshot 2023-03-27 at 10 00 48](https://user-images.githubusercontent.com/6704504/227900994-41f0f355-84bd-4b8a-a2a8-3a9c74447e59.png) - Push the identity of the DID to the receiver chain via the `pallet-dip-sender` extrinsic ![Screenshot 2023-03-27 at 10 01 13](https://user-images.githubusercontent.com/6704504/227901150-7e8c9c9d-8aac-4739-8ad3-fad4ba6ff5f8.png) - Call the runtime API to generate a proof for the created DID with some keys revealed ![Screenshot 2023-03-27 at 10 01 40](https://user-images.githubusercontent.com/6704504/227901309-94b4dbc9-ca83-4541-820d-d1bec6adc6f0.png) - Use the generated proof to dispatch an extrinsic on the receiving chain ### How to use the runtime API with polkadot apps There is currently no support for the new runtime API in the public polkadot apps instance. To use the runtime APIs from UI, please use [our fork from the aa/dip-sender-template branch](https://github.com/KILTprotocol/polkadot-apps/tree/aa/dip-sender-template), by running `yarn && yarn build && yarn start`, then connecting to the sender node WS socket. For runtime augmentation within a Node script, please use our [SDK repo from the aa/dip-merkle-proof branch](https://github.com/KILTprotocol/sdk-js/tree/aa/dip-merkle-proof).
Feature branch for everything DIP. It will collect other PRs until we are happy with the features, and will add the DIP to some of our runtimes and merge this into `develop`. ## WIP Checklist for the open tasks for v1 - [x] Basic structure -> #489 - [x] Merkleization of DID Documents -> #492 - [x] `RuntimeCall` verification logic -> #502 - [x] DID signature verification -> #516 - [x] Add support for linked accounts and web3name -> #525 - [x] Configurable origin for `commit_identity` -> #526 - [x] Proper fee management -> #528 - [x] Update to Polkadot 0.9.43 -> c18a6ce - [x] Replace XCM with state proofs -> #543 - [x] Add support for relaychain consumer -> #553 (part of #543) - [x] Proper error handling -> #572 - [x] Add support for versioning -> #573 - [x] Take deposits for identity commitments -> #574 - [x] Expose common definitions usable by consumers -> #577 - [x] Change ensure_signed! to configurable origin also for the `dispatch_as` function -> #577 - [x] Proper benchmarking and weights -> #585 - [x] Comments and docs -> #584 - [x] Revert Dockerfile changes in #587 - [x] [OPTIONAL] Add support for Zombienet -> #587 - [x] [OPTIONAL] Add chain spec loading from file for template runtimes -> #587 - [x] Big, final review -> #494 (review) - [x] Improvements n.1 PR -> #591 - [x] Improvements n.2 PR -> #592 - [x] Add to Peregrine runtime -> #594 - [ ] Deploy on Peregrine - [ ] Unit tests - [ ] Add to Spiritnet runtime - [ ] Deploy on Spiritnet - [ ] [OPTIONAL] Move DIP-related stuff into its own repo --------- Co-authored-by: Adel Golghalyani <[email protected]> Co-authored-by: Chris Chinchilla <[email protected]> Co-authored-by: Albrecht <[email protected]>
Feature branch for everything DIP. It will collect other PRs until we are happy with the features, and will add the DIP to some of our runtimes and merge this into `develop`. ## WIP Checklist for the open tasks for v1 - [x] Basic structure -> KILTprotocol/kilt-node#489 - [x] Merkleization of DID Documents -> KILTprotocol/kilt-node#492 - [x] `RuntimeCall` verification logic -> KILTprotocol/kilt-node#502 - [x] DID signature verification -> KILTprotocol/kilt-node#516 - [x] Add support for linked accounts and web3name -> KILTprotocol/kilt-node#525 - [x] Configurable origin for `commit_identity` -> KILTprotocol/kilt-node#526 - [x] Proper fee management -> KILTprotocol/kilt-node#528 - [x] Update to Polkadot 0.9.43 -> KILTprotocol/kilt-node@c18a6ce - [x] Replace XCM with state proofs -> KILTprotocol/kilt-node#543 - [x] Add support for relaychain consumer -> KILTprotocol/kilt-node#553 (part of KILTprotocol/kilt-node#543) - [x] Proper error handling -> KILTprotocol/kilt-node#572 - [x] Add support for versioning -> KILTprotocol/kilt-node#573 - [x] Take deposits for identity commitments -> KILTprotocol/kilt-node#574 - [x] Expose common definitions usable by consumers -> KILTprotocol/kilt-node#577 - [x] Change ensure_signed! to configurable origin also for the `dispatch_as` function -> KILTprotocol/kilt-node#577 - [x] Proper benchmarking and weights -> KILTprotocol/kilt-node#585 - [x] Comments and docs -> KILTprotocol/kilt-node#584 - [x] Revert Dockerfile changes in KILTprotocol/kilt-node#587 - [x] [OPTIONAL] Add support for Zombienet -> KILTprotocol/kilt-node#587 - [x] [OPTIONAL] Add chain spec loading from file for template runtimes -> KILTprotocol/kilt-node#587 - [x] Big, final review -> KILTprotocol/kilt-node#494 (review) - [x] Improvements n.1 PR -> KILTprotocol/kilt-node#591 - [x] Improvements n.2 PR -> KILTprotocol/kilt-node#592 - [x] Add to Peregrine runtime -> KILTprotocol/kilt-node#594 - [ ] Deploy on Peregrine - [ ] Unit tests - [ ] Add to Spiritnet runtime - [ ] Deploy on Spiritnet - [ ] [OPTIONAL] Move DIP-related stuff into its own repo --------- Co-authored-by: Adel Golghalyani <[email protected]> Co-authored-by: Chris Chinchilla <[email protected]> Co-authored-by: Albrecht <[email protected]>
Feature branch for everything DIP. It will collect other PRs until we are happy with the features, and will add the DIP to some of our runtimes and merge this into `develop`. ## WIP Checklist for the open tasks for v1 - [x] Basic structure -> #489 - [x] Merkleization of DID Documents -> #492 - [x] `RuntimeCall` verification logic -> #502 - [x] DID signature verification -> #516 - [x] Add support for linked accounts and web3name -> #525 - [x] Configurable origin for `commit_identity` -> #526 - [x] Proper fee management -> #528 - [x] Update to Polkadot 0.9.43 -> c18a6ce - [x] Replace XCM with state proofs -> #543 - [x] Add support for relaychain consumer -> #553 (part of #543) - [x] Proper error handling -> #572 - [x] Add support for versioning -> #573 - [x] Take deposits for identity commitments -> #574 - [x] Expose common definitions usable by consumers -> #577 - [x] Change ensure_signed! to configurable origin also for the `dispatch_as` function -> #577 - [x] Proper benchmarking and weights -> #585 - [x] Comments and docs -> #584 - [x] Revert Dockerfile changes in #587 - [x] [OPTIONAL] Add support for Zombienet -> #587 - [x] [OPTIONAL] Add chain spec loading from file for template runtimes -> #587 - [x] Big, final review -> #494 (review) - [x] Improvements n.1 PR -> #591 - [x] Improvements n.2 PR -> #592 - [x] Add to Peregrine runtime -> #594 - [ ] Deploy on Peregrine - [ ] Unit tests - [ ] Add to Spiritnet runtime - [ ] Deploy on Spiritnet - [ ] [OPTIONAL] Move DIP-related stuff into its own repo --------- Co-authored-by: Adel Golghalyani <[email protected]> Co-authored-by: Chris Chinchilla <[email protected]> Co-authored-by: Albrecht <[email protected]>
Feature branch for everything DIP. It will collect other PRs until we are happy with the features, and will add the DIP to some of our runtimes and merge this into `develop`. ## WIP Checklist for the open tasks for v1 - [x] Basic structure -> #489 - [x] Merkleization of DID Documents -> #492 - [x] `RuntimeCall` verification logic -> #502 - [x] DID signature verification -> #516 - [x] Add support for linked accounts and web3name -> #525 - [x] Configurable origin for `commit_identity` -> #526 - [x] Proper fee management -> #528 - [x] Update to Polkadot 0.9.43 -> c18a6ce - [x] Replace XCM with state proofs -> #543 - [x] Add support for relaychain consumer -> #553 (part of #543) - [x] Proper error handling -> #572 - [x] Add support for versioning -> #573 - [x] Take deposits for identity commitments -> #574 - [x] Expose common definitions usable by consumers -> #577 - [x] Change ensure_signed! to configurable origin also for the `dispatch_as` function -> #577 - [x] Proper benchmarking and weights -> #585 - [x] Comments and docs -> #584 - [x] Revert Dockerfile changes in #587 - [x] [OPTIONAL] Add support for Zombienet -> #587 - [x] [OPTIONAL] Add chain spec loading from file for template runtimes -> #587 - [x] Big, final review -> #494 (review) - [x] Improvements n.1 PR -> #591 - [x] Improvements n.2 PR -> #592 - [x] Add to Peregrine runtime -> #594 - [ ] Deploy on Peregrine - [ ] Unit tests - [ ] Add to Spiritnet runtime - [ ] Deploy on Spiritnet - [ ] [OPTIONAL] Move DIP-related stuff into its own repo --------- Co-authored-by: Adel Golghalyani <[email protected]> Co-authored-by: Chris Chinchilla <[email protected]> Co-authored-by: Albrecht <[email protected]>
Fixes https://github.com/KILTprotocol/ticket/issues/2557 and fixes https://github.com/KILTprotocol/ticket/issues/2556.
This PR builds on top of the shell PR, and adds support for Merkle proof for DID documents.
Merkle proof structure
A DID merkle proof is, at the core, an order set of (key, value) pairs, on which proof-of-inclusion and proof-of-non-inclusion can be performed. This PR generates and validates merkle proofs where leaves are of two types:
For each key reference leaf with a given key ID, the proof also has to contain a key details leaf whose key is the key ID. Multiple reference leaves can reference the same key details leaf, optimising the storage size.
New runtime APIs
There is a new runtime API which the DIP sender would expose, and that allows anyone to generate a merkle proof for a given DID identifier and set of key IDs. The result contains the merkle root (which must match what other chains have stored in their
pallet-dip-receiver
map), and a merkle proof, which includes blinded values and a set of key reference and key details leaves for the keys identified by the provided key IDs.How to test
The setup flow is similar to that of #489.
Specifically:
pallet-dip-sender
extrinsicHow to use the runtime API with polkadot apps
There is currently no support for the new runtime API in the public polkadot apps instance. To use the runtime APIs from UI, please use our fork from the aa/dip-sender-template branch, by running
yarn && yarn build && yarn start
, then connecting to the sender node WS socket.For runtime augmentation within a Node script, please use our SDK repo from the aa/dip-merkle-proof branch.