Skip to content

Nimbus Roadmap

Jump to bottom
zah edited this page Feb 22, 2020 · 17 revisions

This document attempts to specify a list of high-level components of the Nimbus project that can be developed and tested in isolation. It tries to identify the dependencies between the development tasks, so the implementation efforts can be scheduled in parallel where appropriate. Please keep in mind that the list is not exhaustive and further planning is necessary before all tasks are specified in the same level of detail.

Development tasks:

1. RLP (COMPLETED)

2. ETHASH (COMPLETED)

Remaining tasks:

  • GPU Mining
  • Mining with Stratum server support

3. Select an AES and SHA256 implementation. (COMPLETED)

The AES library should support both AES256 in CTR mode (as per the RLPx spec) and GCM mode (used in Whisper v6)

Library used: NimCrypto

4. Implement ECIES (COMPLETED)

Required for the encyprted handshake. Based on libsecp256k1.

Reference implementation:

Depends on:

5. Select Scrypt library (optional)

Both cpp-ethereum and Parity also use Scrypt in their local keystore code. It doesn't seem mandatory for the implementation.

6. Ethereum Bloom Filter (COMPLETED)

Bloom filters are used for efficient monitoring of the transaction logs/events, stored in the receipt trie of the blockchain, for topic selection in Whisper and some other places.

Reference implementation:

Existing Nim library:

7. Ethereum key file (PARTIALLY COMPLETED)

The key file stores the keys associated with one or more accounts in password-protected encrypted form.

Reference implementation:

Implementation:

Sub-tasks:

  • Safe storage (PBKDF2)
  • Safe storage (SCRYPT)
  • Key generation (PBKDF2)
  • Key generation (SCRYPT)
  • Password recovery (is it possible at all?)
  • Use native APIs on mobile platforms

Depends on:

8. JSON-RPC Server (PARTIALLY COMPLETED)

Development already started. This will be an ongoing effort and the list of supported calls should increase as other parts of the system are developed.

Reference information:

Optional components: [ ] WS-RCP Server [ ] IPC Server [ ] Password protection

9. Test suite

Sub-tasks:

  • CI tests for Genesis
  • CI tests for VM JSON tests
  • CI tests for storage backends
  • CI tests for stack functionality
  • CI tests for code stream
  • CI tests for VM memory
  • CI tests for VM opcodes
  • CI tests for GeneralStateTests JSON tests
  • CI tests for other libraries
  • Validate that the entire history of Ethereum can be executed by Nimbus
  • Check that we fit in a certain memory/compute budget?

10. Node discovery (COMPLETED)

This uses a modified and stripped down version of Kademlia. Kademlia us also used in Swarm for resource discovery and some of the logic for maintaining peer connection tables can be shared between the two implementations.

In the sharded blockchan, each shard is represented by a separate P2P network. This is implemented by running the same node discovery protocol, but with a different peer connection port:

In the future, Ethereum may support alternative transport protocols (such as lib2p2). To enable this, there is new version 5 of the node discovery protocol in development, which allow the nodes to advertise alternative connection methods (ENR):

Version 5 of the protocol also allows the discovery of nodes with certain capabilities:

Sub-tasks:

11. UPnP support

The standard Ethereum protocols use TCP to communicate between peers. Hosts with private IP addresses must set up port redirection on their routers. UPnP can be used to automate this.

Related tasks:

[X] Implement the nat options of the other Ethereum clients. The user can choose from any, none, upnp, extip (Geth aslo has pmp)

12. Authenticated handshake (COMPLETED)

https://github.com/ethereum/devp2p/blob/master/rlpx.md#encrypted-handshake

Sub-tasks:

  • Authenticate previously known nodes with an old session key
  • Authenticate newly discovered nodes
  • Support RLPx sub-protocol negotiation in the authentication message (capabilities)
  • Derive shared secrets from handshake

Depends on:

13. RLPx sub-protocols (COMPLETED)

The Ethereum wire protocols are transport-agnostic. Our framework should allow running them over the TCP connections established after node discovery or through connections established with libp2p or uTP in the future. It should also be possible to define unit tests involving multiple agents simulated within a single process (bypassing the network stack).

Issues:

None of the protocols currently supports using NAT traversal for directly establishing P2P connections. This may be important for supporting Video and Audio calls in Status. We can solve this by establishing such connections in an out-of-band protocol (e.g. by linking WebRTC or something similar) or by contributing EIPs for the existing protocols.

Sub-tasks:

  • SubProtocol registry

  • Active protocol map (shared objects)

  • Message dispatcher

  • Subprotocol definition DSL

  • Protocol negotiation

  • Per-subprotocol send queue

  • Chunking

  • Peer reputation tracking (with persistence)

  • Flow control (optional)

  • Pre-sending PoW priority increase

14. The new binary trie (COMPLETED)

Replaces the old hexary trie in the new blockchain shards.

Sub-tasks:

  • Add support for partial tries
  • Support for obtaining and verifying Merkle proofs

Reference implementations:

15. New parametric Blockchain definition (WORK STARTED)

There should be sufficient parametrization to allow:

The new shard chain uses a new account creation method:

... which requires 3 new opcodes to be added:

  • SIGHASH
  • PAYGAS
  • CREATE2

Current outstanding tasks:

  • Stateless and stateful execution with roll-back
  • Maintenance of a receipt trie
  • Block validation
  • Precompiled contracts
  • Transaction pool

Depends on:

Extra tasks:

  • VM execution tracing/stepping

16. Blockchain Sync Algorithms (WORK STARTED)

Fast sync uses the standard Ethereum sub-protocol.

Parity's warp sync is based on a custom server and protocol developed by the Parity team.

The sharding client features an even faster stateless sync protocol that aims to obtain only the latest block headers and to perform a randomized verification that the blockchain data is available upon request:

Depends on:

Related tasks: [ ] Blockchain init from a genesis config for private networks

17. Known contract abstraction / DApp framework (COMPLETED)

Make it easier to interact with known contracts such as Casper, the Sharding VMC, etc. Each such contract has a well-known address and a particular API that can be expressed in standard Nim.

Libraries such as web3.js and web.py offer similar functionality.

Sub-tasks:

  • Nim-types-to-ABI-types translation and contract FFI DSL.
  • Transaction creation logic
  • Event monitoring logic
  • Provide high-level information about the contract for the UI layer (NatSpec, etc)

Depends on:

18. Light Clients (WORK STARTED)

The light client operates without storing the full chain locally. Most operations depends on Asynchronous APIs that will downloads parts of the chain on-demand. Light clients can also execute DApps by monitoring the new blocks for events of interest.

Sub-tasks:

  • Events monitoring
  • State-fetching APIs
  • Flow control
  • Fetch queue

Depends on:

19. Stateless Clients

Sub-tasks:

  • Transaction state witness extraction
  • Stateless sync
  • State objects and "pure" VM execution
  • Track updates to all keys in the witness in an in-memory store.

Depends on:

20. Etehreum name service resolver

Sub-tasks:

  • Resolve names by recursive ENS contract application

Depends on:

22. Whisper (COMPLETED)

While Swarm seems a better fit for Status's needs, we may also implement the Whisper protocol, which is currently at version 6:

Overview information (previous versions):

Sub-tasks:

  • Envelope encryption
  • Mail Server
  • Filter table
  • Onion routing (Darkness)

Depends on:

23. Command-line interface, Configuration, Local directory structure, Build Targets, Web Interface

Sub-tasks:

  • Provide options for launching the various daemons and modes of Nimbus
  • Define a common scheme for specifying parameters (as command-line switches, as env vars, in configuration files, etc)
  • Provide abstractions over local file system usage (e.g. XDG, AppData, Android and iOS specific APIs)
  • Provide compile-time defines for turning off features of Nimbus (i.e. allow the lite clients to omit most of the code)
  • Blockchain export/inspection APIs and CLIs
  • Test example applications using Nimbus as a library
  • Define various UIs for Nimbus (Web UI, GUI, etc)
  • Android build / CI test suite
  • iOS build / CI test suite

24. Hardware-wallet support

Clone this wiki locally