SCP-002: Datastore P2P interface #4
Conversation
| @@ -0,0 +1,69 @@ | |||
| ## SCP-002: Datastore P2P interface | |||
There was a problem hiding this comment.
Right out of the gates I think we're gonna want to look at this one layer of abstraction upward:
This interface should be flexible enough to handle the internal needs of the datastores, as well as the external needs of whatever pubsub interface we end up using.
Internal requirements are defined as whatever this proposal ends up being, as well as whatever the current functionality is.
External means that eventually OrbitDB should be able to use MQTT, IPFS pubsub, whatever else we can think of
https://github.com/orbitdb/orbit-db-pubsub
There was a problem hiding this comment.
For example one possible external and internal requirement of the interface might be the use of a peerId in the pubsub messages to verify authenticity. However, if the pubsub like system being utilized does not support cryptographically signed peerIds and messages, then it might break certain functionality. I am not sure if MQTT has support for cryptographically signed messages and peerIds. If so we also need to make sure the formatting of these constructs is compatible, or can be transformed into a data type that is indifferent between systems.
| * The network: Cloud of peers all interconnected interested in the datastore in question | ||
| * Broadcast channel: Global pubsub channel of the corresponding datastore. | ||
| * Application specific P2P API: Interface diagrammed by higher level code implementing `orbit-db-store`; Defined by the store type. | ||
|
|
There was a problem hiding this comment.
We should diagram this. I might take a stab at it.
| ### Hypothetical model N1 | ||
| > Note all p2p traffic is over IPFS pubsub | ||
|
|
||
| **Flow**: |
There was a problem hiding this comment.
Note the prior work here:
- https://github.com/ipfs-shipyard/ipfs-pubsub-1on1/
- https://github.com/ipfs-shipyard/ipfs-pubsub-peer-monitor/
Ideally we'd appropriate these (or at least get commit access)
| **Flow**: | ||
|
|
||
| 1. Asking peer joins store broadcast channel. (Example: Asking peer is just starting up and gets connected to the network) | ||
| 2. Responding peer broadcasts a small message to all the other peers containing information regarding what diagrams/protocol it supports. Including Public key for encryption, if needed, and a forwarding pubsub channel ID (a channel where asking peers can send messages directly to the responding peer, derived from the peerId). The forwarding channel isn't designed to be a back and forth between a responding peer and an asking peer. Only a temporary channel for negotiating a direct peer-to-peer connection. |
There was a problem hiding this comment.
TODO: Research this at the IPFS level. I believe they have something like this now, or at least something experimental.
There was a problem hiding this comment.
There is some work being done for standardized data encryption as well as multikey, but it hasn't had much work done on it.
See
ipld/specs#257
ipfs-inactive/docs#87
ipfs/specs#58
More clarification would be helpful
| * Pubsub relays can help with routing packets to peers behind a firewall. | ||
|
|
||
| **Disadvantages** | ||
| * No P2P transport security out of the box. Higher chance of an adversory capturing P2P messages |
There was a problem hiding this comment.
At least we have handshake security with the Noise protocol. Wondering what else we can utilize. However, with the externally-pluggable requirement we should probably handle this at the orbit-db-pubsub level
There was a problem hiding this comment.
I am unaware the Noise protocol can be used over pubsub? Are you talking about noise protocol in regards to libp2p? Could you provide some more clarification?
| * Peer discovery/rendezvous might be an issue. | ||
| * Requires building custom systems to accomplish the same level of functionality as Model N2. | ||
|
|
||
| ### Hypothetical model N2 |
There was a problem hiding this comment.
At first read I like this approach in general. Let's keep iterating on it.
| Still relying on all peers to have a public/private key for the oplog entry signing. | ||
| * Querying other peers instead of searching local store. | ||
|
|
||
| Further discussion and comments is appreciated. |
There was a problem hiding this comment.
I would describe this as something similar to how you can define your own Index and Store classes when you extend orbit-db-store. It might be interesting to see it as a sibling class to those in a store implementation. Index, Store, and Messages, or some such thing
No description provided.