NU5 Tracking: Pre-NU5 Spec work

[mpguerra]

This epic will track the minimum set of outstanding spec items, defined prior to NU5 and still active once NU5 is activated, which will need to be implemented in Zebra.

This tracking issue contains segments for the whole spec that includes Orchard changes for now. A checkbox means that that part of the specification describes something to implement.

A checked box or strikethrough means that we:

• have implemented that item
• are tracking that item in another Epic, or
• have otherwise resolved that item.

Our mandatory Canopy checkpoint significantly reduces the number of features that we have to implement.

Leaf nodes are marked by – and correspond to implementable items. They should be followed by issue references that track the status of that issue, or a short explanation.

• 1. Introduction – nothing implementable
• 2. Notation – nothing implementable
• 3. Concepts
  • 3.1. Payment Addresses and Keys (tracked under NU5 Tracking: Required ZIPs #1854 in Implement ZIP-32 Sapling and Orchard key derivation #267 )
  • 3.2. Notes – Sapling/Orchard note encryption implementation. #269 (will be done as part of the zebra-client work)
    • 3.2.1 Note Plaintexts and Memo Fields – Make Memo an enum type #250 (will be done as part of the zebra-client work)
  • 3.3. The Block Chain (done)
  • 3.4. Transactions and Treestates – Treestate type #290, Design: treestate (note commitment trees and nullifier sets) #958, Draft RFC: Treestate management #983
  • 3.5. JoinSplit Transfers and Descriptions (done)
  • 3.6. Spend Transfers, Output Transfers, and their Descriptions (done)
  • 3.7. Action Transfers and their Descriptions (tracked under NU5 Tracking: The Spec #1856)
  • 3.8. Note Commitment Trees – Implement incremental, sparse note commitment trees for Sapling and Orchard #1287
  • 3.10. Block Subsidy, Funding Streams, and Founders’ Reward Calc Block Subsidy and Funding Streams #338 (covered by ZIP-214/Calc Block Subsidy and Funding Streams #338 and tracked under NU5 Tracking: Required ZIPs #1854 in Tracking Issue: Validate coinbase transactions, including the dev fund #801)
  • 3.11. Coinbase Transactions(covered by Heartwood ZIP-213 and tracked under NU5 Tracking: Required ZIPs #1854 in ZIP-213: Validate Shielded Coinbase Outputs like other shielded outputs #608)
  • 3.12. Mainnet and Testnet(nothing to do here)
• 4. Abstract Protocol
  • 4.1. Abstract Cryptographic Schemes
    • 4.1.1. Hash Functions (pull as dependencies)
    • 4.1.2. Pseudo Random Functions (pull as dependencies)
    • 4.1.3. Symmetric Encryption (pull as dependencies? done?)
    • 4.1.4. Key Agreement (done for sprout and sapling)
    • 4.1.5. Key Derivation (done for sprout and sapling)
    • 4.1.6. Signature (RedJubJub and Ed25519)
    • 4.1.7. Commitment (done for sprout and sapling)
    • 4.1.8. Represented Group
    • 4.1.9. Hash Extractor (handled by JubJub lib)
    • 4.1.10. Group Hash (handled by *25519, JubJub libs)
    • 4.1.11. Represented Pairing
    • 4.1.12. Zero-Knowledge Proving System - Integrate async JoinSplit Groth16 verifier with transaction verifier #1835 and Integrate Spend and Output async groth verifiers with transaction verifier #1645
  • 4.2. Key Components
    • 4.2.1. Sprout Key Components (done)
    • 4.2.2. Sapling Key Components (done)
    • 4.2.3. Orchard Key Components (tracked under NU5 Tracking: The Spec #1856)
  • 4.3. JoinSplit Descriptions (done)
  • 4.4. Spend Descriptions (pre-NU5 work done)
  • 4.5. Output Descriptions (pre-NU5 work done)
  • 4.6. Action Descriptions (tracked under NU5 Tracking: The Spec #1856)
  • 4.7. Sending Notes
    • 4.7.1. Sending Notes (Sprout) (will be done as part of the zebra-client work)
    • 4.7.2. Sending Notes (Sapling and Orchard) (will be done as part of the zebra-client work in Sapling/Orchard note encryption implementation. #269)
  • 4.8. Dummy Notes - probably doesn't exist in the wild yet
    • 4.8.1. Dummy Notes (Sprout) (will be done as part of the zebra-client work)
    • 4.8.2. Dummy Notes (Sapling and Orchard) (will be done as part of the zebra-client work)
  • 4.9. Merkle Path Validity (pre-NU5 work done?)
  • 4.10. SIGHASH Transaction Hashing – Check sighash parameters in transaction verifier #1377
  • 4.11 Non-malleability (Sprout) - ?
  • 4.12. Balance (Sprout) (tracked under NU5 Tracking: Required ZIPs #1854 in ZIP-209)
  • 4.13. Balance and Binding Signature (Sapling)
  • 4.14. Balance and Binding Signature (Orchard) (tracked under NU5 Tracking: The Spec #1856)
  • 4.15. Spend Authorization Signature (Sapling and Orchard) (will be done as part of the zebra-client work in Implement spend authorization signing. #319)
  • 4.16 Note Commitments and Nullifiers (done for sprout and sapling)
  • 4.17. Zk-SNARK Statements
    • 4.17.1. JoinSplit Statement (Sprout) - Validate JoinSplit proofs #322
    • 4.17.2. Spend Statement (Sapling) - Validate Spend Proofs #323
    • 4.17.3. Output Statement (Sapling) - Validate Output Proofs #324
    • 4.17.4. Action Statement (Orchard) (tracked under NU5 Tracking: The Spec #1856)
  • 4.18. In-band secret distribution (Sprout) (will be done as part of the zebra-client work in Sprout Key Agreement and KDF #272)
    • 4.18.1. Encryption (Sprout) (will be done as part of the zebra-client work in Sprout note encryption/decryption implementation. #181)
    • 4.18.2. Decryption (Sprout) (will be done as part of the zebra-client work in Sprout note encryption/decryption implementation. #181)
  • 4.19. In-band secret distribution (Sapling and Orchard) (will be done as part of the zebra-client work in Sapling/Orchard Key Agreement and KDF #271)
    • 4.19.1. Encryption (Sapling and Orchard) (will be done as part of the zebra-client work in Sapling/Orchard note encryption implementation. #269 )
    • 4.19.2. Decryption using an Incoming Viewing Key (Sapling and Orchard) (will be done as part of the zebra-client work in Sapling/Orchard note encryption implementation. #269)
    • 4.19.3. Decryption using a Full Viewing Key (Sapling and Orchard) (will be done as part of the zebra-client work in Sapling/Orchard note encryption implementation. #269)
  • 4.20. Block Chain Scanning (Sprout) (will be done as part of the zebra-client work in Block Chain Scanning  #325)
  • 4.21. Block Chain Scanning (Sapling and Orchard) (will be done as part of the zebra-client work in Block Chain Scanning  #325)
• 5. Concrete Protocol (pre-NU5 work done)
• 6. Network Upgrades (tracked under NU5 Tracking: Required ZIPs #1854 in ZIP-200: Network Upgrade Mechanism #337)
• 7. Consensus Changes from Bitcoin
  • 7.1. Encoding of Transactions (already covered in other issues above)
  • 7.2. Encoding of JoinSplit Descriptions
  • 7.3. Encoding of Spend Descriptions
  • 7.4. Encoding of Output Descriptions
  • 7.5. Action Description Encoding and Consensus (tracked under NU5 Tracking: The Spec #1856)
  • 7.6. Block Header (already covered in other issues)
  • 7.7. Proof of Work
  • 7.8. Calculation of Block Subsidy, Funding Streams, and Founders’ Reward (covered by ZIP-214/Calc Block Subsidy and Funding Streams #338 and tracked under NU5 Tracking: Required ZIPs #1854)
  • 7.9. Payment of Founders’ Reward (obsoleted by Canopy checkpoint)
  • 7.10. Payment of Funding Streams (tracked under NU5 Tracking: Required ZIPs #1854 in Tracking Issue: Validate coinbase transactions, including the dev fund #801)
  • 7.11. Changes to the Script System
  • 7.12. Bitcoin Improvement Proposals

[mpguerra]

Added #267 under "3.1. Payment Addresses and Keys", I'm guessing this might change to only implementing Sapling key derivation?

[mpguerra]

Added #269 under "3.2. Notes"
Skipping #181 for the Sprout note encryption/decryption for now unless we really need to do this

[mpguerra]

Added #250 under "3.2.1 Note Plaintexts and Memo Fields"

[mpguerra]

Added #290 #958 and #983 to "3.4. Transactions and Treestates"

[mpguerra]

Added #1287 to " 3.8. Note Commitment Trees"

[mpguerra]

Added "3.10. Block Subsidy, Funding Streams, and Founders’ Reward" and related issue: #338

[mpguerra]

Added #1835 and #1645 to "4.1.12. Zero-Knowledge Proving System"

[mpguerra]

Added #181 to "4.7.1. Sending Notes (Sprout)"

[mpguerra]

Added #269 to "4.7.2. Sending Notes (Sapling and Orchard)"

[mpguerra]

Added #1377 to "4.10. SIGHASH Transaction Hashing"

[mpguerra]

Added #319 to "4.15. Spend Authorization Signature (Sapling and Orchard)"

[mpguerra]

@ZcashFoundation/zebra-team I think this list is now finished, I'm just missing some tickets for:

• 4.8.1. Dummy Notes (Sprout)
• 4.8.2. Dummy Notes (Sapling and Orchard)
• 4.9. Merkle Path Validity
• 4.11 Non-malleability (Sprout)
• 4.12. Balance (Sprout)
• 7.10. Payment of Funding Streams

[teor2345]

• 4.12. Balance (Sprout)

I think this is ZIP-209, but we also need to do Sapling and Orchard balances.

Also note that a non-negative transparent pool balance is a consensus rule as well.

• 7.10. Payment of Funding Streams

This is #801.

[mpguerra]

Removed a bunch of issues from the Epic relating to zebra-client/wallet support work (#250, #267, #269, #271, #272, #319, #320, #321, #325)

[teor2345]

I added #801 and #196 for transaction validation. Zebra parses pre-NU5 transactions correctly, but semantic and contextual transaction validation is incomplete.

[teor2345]

I added #477 for block header validation. Zebra parses pre-NU5 block headers correctly, but its semantic and contextual validation is incomplete.

[mpguerra]

Cleaning up this tracking issue to cut scope and remove tracking redundancies:

• removed Tracking Issue: Validate coinbase transactions, including the dev fund #801 as this is tracked under NU5 Tracking: Required ZIPs #1854
• removed Tracking: block header validation #477 as I believe this is now functionality complete (only things remaining are clean up and tests)

[mpguerra]

Removed #196 from the epic to cut scope and remove tracking redundancies. All issues under #196 that should be implemented are already tracked in this issue or #1854

[mpguerra]

I think we're all done here! 🎉