Shard Manager for watching multiple shards

[hwwhww]

• Enable regular stateless client to watch multiple shards at the same time.
• Once the validators detect that the they are sampled as the collator, they have to add the shard_id the watching list, and then validators have to run the stateless fast sync to sync the shard. The config looks like:

  {
      validator_address: "0x..." OR null,
      watching: [list of shard ids],
      ...
  }
  

• shard_id is similar to the concept of network_id, we use shard_id to identify different p2p networks.

[pipermerriam]

@hwwhww is this something that will live in the networking layer?

[hwwhww]

I hope the implementation would be mostly in transport layer and session layer.

1. Port mapping
   • Map different network_ids/shard_ids to different ports
2. Keep the same Node Discovery Protocol
3. Suggestions
   • The early step may be trying to connect to two non-sharding private networks concurrently. So we don’t need to update Wire Protocol at first.
   • Use different databases for different networks.
4. Update Wire Protocol for sharding
   • Option 1: Reuse the Status Message of Ethereum Wire Protocol (Transport Layer)
     • Status [+0x00: P, protocolVersion: P, networkId: P, td: P, bestHash: B_32, genesisHash: B_32] Inform a peer of its current ethereum state. This message should be sent after the initial handshake and prior to any ethereum related messages
       • protocolVersion: TBD
       • networkId: prefix 10 + shard_id
       • td: total score (number) of the best chain. Integer, as found in collation header.
       • bestHash: the hash of the best (i.e. highest score) known collation.
       • genesisHash: 32 bytes of 0x00
   • Option 2: Write new Sharding-only Status Message
5. I suppose ShardManager includes or triggers:
   1. Calling function in VMC utils (maybe use event filter) to detect whether they are sampled as the collator
   2. Maintaining watching list
   3. Modules like PeerManager in pydevp2p
      • connect / disconnect p2p network
   4. Modules like Synchronizer in pyethapp
      • handle (1) stateless fast sync or (2) full sync task
6. Dependency
   • Shard client needs to run a full node
     • Need to implement the equivalent of evm.p2p.peer.LESPeer for full node
     • Need command line interface