[Ready for review - Pending fix of upstream vectors] Update BLS signa…

…ture scheme to draft standard (#36)

* Rework the readme

* Prepare transitioning to the new IETF API

* Delete obsolete README, add Cipher suite ID (BLS_SIG_BLS12381G2-SHA256-SSWU-RO-_NUL_)

* Stash BLS keygen implementation

* Implement CoreSign, CoreVerify and signature aggregation

* Implement CoreAggregateVerify

* Implement message signing/verification, proof-of-possession generation/verification, aggregate verification/fast aggregate verification

* add aggregate verify. Enforce proper usage of proofs at the API level for signature validation

* Implement key pair generator

* Move old test vectors, add Eth2 v0.10.0 test vectors

* Fix paths for the moved old test suite

* Add the spec overload that don't enforce proof-of-possessions usage. Needed for test vectors

* Prevent non constant-time comparison of secret-key (and comparison altogether)

* Add IO + fix types/API issues

* IO: include to allow accessing private fields

* Add (failing) signing test vectors

* Add hex dumps - the hex dumps are the same! (but comparison fails for unknown reason)

* Fix sign test by exporting an `==` proc

* Factor out common part of the tests

* Properly handle tampered signature in the test vector

* Add signature aggregation tests

* Add fast aggregate verify test (failing to wrong test - ethereum/consensus-specs#1618)

* Switch coreAggregateVerify to a streaming API to accomodate AoS SoA and interleaving proofs of possessions

* Add AggregateVerify test for both SoA and AoS API

* Comment out internal tests, skip buggy upstream test, add to nimble

* don't export init/update

* Add yaml testing dependency to CI

* Use NimYAML fork (flyx/NimYAML#77)

* YAML fix was not fully applied

* NimYAML update lost in the rebase/force push

* Hopefully fix NimYAML build issues

* same NimYAML fix on POSIX

.appveyor.yml

@@ -33,9 +33,10 @@ install:
 build_script:
   - cd C:\projects\%APPVEYOR_PROJECT_SLUG%
   - nimble install -y
+  # testing only
+  - nimble install https://github.com/status-im/NimYAML@#head # Cannot clone upstream https://github.com/flyx/NimYAML/issues/77
 
 test_script:
   - nimble test
 
 deploy: off
-

.travis.yml

@@ -23,5 +23,6 @@ install:
 
 script:
   - nimble install -y
+  # testing only
+  - nimble install https://github.com/status-im/NimYAML@#head # Cannot clone upstream https://github.com/flyx/NimYAML/issues/77
   - nimble test
-

README.md

@@ -1,18 +1,14 @@
-# Nim BLS12-381 Curve implementation
+# BLS Signature Scheme over BLS12-381 pairing-friendly curve
 
 [![Build Status](https://travis-ci.org/status-im/nim-blscurve.svg?branch=master)](https://travis-ci.org/status-im/nim-blscurve)
 [![Build status](https://ci.appveyor.com/api/projects/status/6l1il60ljfbtxw3g/branch/master?svg=true)](https://ci.appveyor.com/project/nimbus/nim-blscurve/branch/master)
 [![Build Status: Azure](https://img.shields.io/azure-devops/build/nimbus-dev/0c305144-232d-4f3e-ba77-93e4e81182da/4/master?label=Azure%20%28Linux%2064-bit%2C%20Windows%2032-bit%2F64-bit%2C%20MacOS%2064-bit%29)](https://dev.azure.com/nimbus-dev/nim-blscurve/_build?definitionId=4&branchName=master)
 
+This library implements:
+- The BLS signature scheme (Boneh-Lynn-Shacham)
+- over the BLS12-381 (Barreto-Lynn-Scott) pairing-friendly curve
 
-This library uses sources from [AMCL](https://github.com/apache/incubator-milagro-crypto-c).
-
-Current curve supported:
-
-  - BLS12-381 (ZK-SNARKS)
-
-For signature and verification purposes, raw messages are first hashed with SHA256 (SHA2)
-following Ethereum requirements 2.0
+Cipher suite ID: BLS_SIG_BLS12381G2-SHA256-SSWU-RO-\_POP\_
 
 ## Installation
 
@@ -21,22 +17,15 @@ You can install the developement version of the library through nimble with the
 nimble install https://github.com/status-im/nim-blscurve
 ```
 
-## Keeping track of upstream
-
-To keep track of upstream:
-
-- Update the submodule.
-- Execute `nim e milagro.nims amcl blscurve/csources`
-- Test
-- Commit
-
 ## Implementation stability
 
 This repo follows Ethereum 2.0 requirements.
 
 Besides the standardization work described below, no changes are planned upstream
 for the foreseeable future.
 
+### Standardization
+
 Currently (Jun 2019) a cross-blockchain working group is working to standardize BLS signatures
 for the following blockchains:
 - Algorand
@@ -46,12 +35,31 @@ for the following blockchains:
 - Filecoin
 - Zcash Sapling
 
-Standardization work is led by:
-- Dan Boneh, Professor at Stanford University and co-author of BLS
-- Sergey Gorbunov, Assistant Professor at the University of Waterloo and working at Algorand
+#### Signature scheme
+
+- IETF draft submission: https://tools.ietf.org/html/draft-boneh-bls-signature-00
+- Repo for collaboration on the draft: https://github.com/cfrg/draft-irtf-cfrg-bls-signature
 
-IETF draft submission: https://tools.ietf.org/html/draft-boneh-bls-signature-00
-Repo for collaboration on the draft: https://github.com/pairingwg/bls_standard
+#### Hashing to curve
+
+- https://tools.ietf.org/html/draft-irtf-cfrg-hash-to-curve-05
+
+#### Curve implementation
+
+- https://tools.ietf.org/html/draft-irtf-cfrg-pairing-friendly-curves-00
+
+## Backend
+
+This library uses sources from [AMCL (Apache Milagro Crypto)](https://github.com/apache/incubator-milagro-crypto-c).
+
+### Keeping track of upstream
+
+To keep track of upstream AMCL:
+
+- Update the submodule.
+- Execute `nim e milagro.nims amcl blscurve/csources`
+- Test
+- Commit
 
 ## License
 

azure-pipelines.yml

@@ -141,12 +141,14 @@ steps:
       echo "PATH=${PATH}"
       nimble refresh
       nimble install -y --depsOnly
+      nimble install https://github.com/status-im/NimYAML@#head
     displayName: 'Building dependencies (Posix)'
     condition: ne(variables['Agent.OS'], 'Windows_NT')
   - powershell: |
       echo "PATH=$Env:Path"
       nimble refresh
       nimble install -y --depsOnly
+      nimble install https://github.com/status-im/NimYAML@#head
     displayName: 'Building dependencies (Windows)'
     condition: eq(variables['Agent.OS'], 'Windows_NT')
   - bash: |

blscurve.nim

@@ -6,5 +6,18 @@
 # at your option.
 # This file may not be copied, modified, or distributed except according to
 # those terms.
-import blscurve/[common, bls]
-export common, bls
+
+const BLS_USE_IETF_API* {.booldefine.} = true
+
+when BLS_USE_IETF_API:
+  import ./blscurve/bls_signature_scheme
+  export
+    SecretKey, PublicKey, Signature, ProofOfPossession,
+    `==`,
+    aggregate,
+    sign, verify, aggregateVerify, fastAggregateVerify,
+    keyGen,
+    fromHex, fromBytes, toHex
+else:
+  import ./blscurve/bls_old_spec
+  export bls_old_spec

blscurve.nimble

@@ -10,16 +10,26 @@ requires "nim >= 1.0.4",
          "stew"
 
 ### Helper functions
-proc test(path: string, lang = "c") =
+proc test(env, path: string, lang = "c") =
   if not dirExists "build":
     mkDir "build"
-  exec "nim " & lang & " --outdir:build -r --hints:off --warnings:off " & path
+  exec "nim " & lang & " " & env &
+    " --outdir:build -r --hints:off --warnings:off " & path
 
 ### tasks
 task test, "Run all tests":
   # Debug - test intermediate computations
-  test "blscurve/hkdf.nim"
-  test "blscurve/hash_to_curve.nim"
+  # test "", "blscurve/hkdf.nim"
+  # test "", "blscurve/hash_to_curve.nim"
 
-  # Public BLS API
-  test "tests/all_tests.nim"
+  # Internal+Public BLS API - pre Ethereum2.0 v0.10
+  # Those directly import the old API and internals
+  # and do not need the "-d:BLS_USE_IETF_API=false" flag
+  test "", "tests/old_spec/test_scheme.nim"
+  test "", "tests/old_spec/test_vectors.nim"
+
+  # Internal BLS API - IETF standard / post Ethereum2.0 v0.10
+  # test "", "tests/hash_to_curve.nim"
+
+  # Public BLS API - IETF standard / post Ethereum2.0 v0.10
+  test "-d:BLS_USE_IETF_API=true", "tests/eth2_vectors.nim"

blscurve/bls.nim → blscurve/bls_old_spec.nim

@@ -6,6 +6,12 @@
 # at your option.
 # This file may not be copied, modified, or distributed except according to
 # those terms.
+
+# Implementation of BLS signature scheme (Boneh-Lynn-Shacham)
+# pre-standardization. Target Ethereum 2.0 specification before v0.10.
+# To be removed other actors (testnet/interop, fuzzing)
+# switch to the IETF standard
+
 import nimcrypto/[sysrand, utils, hash, sha2]
 import stew/endians2
 import milagro, common

blscurve/bls_sig_io.nim

@@ -0,0 +1,48 @@
+# Nim-BLSCurve
+# Copyright (c) 2018 Status Research & Development GmbH
+# Licensed under either of
+#  * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
+#  * MIT license ([LICENSE-MIT](LICENSE-MIT))
+# at your option.
+# This file may not be copied, modified, or distributed except according to
+# those terms.
+
+# Implementation of IO routines to serialize to and from
+# the types defined in
+# - https://tools.ietf.org/html/draft-irtf-cfrg-bls-signature-00#section-5.5
+# - https://github.com/cfrg/draft-irtf-cfrg-bls-signature
+
+# This file should be included to have access to private fields
+# It is kept separated as it does not fall under the IETF BLS specification
+
+func fromHex*[T: SecretKey|PublicKey|Signature|ProofOfPossession](
+       obj: var T,
+       hexStr: string
+     ): bool {.inline.} =
+  ## Initialize a BLS signature scheme object from
+  ## its hex raw bytes representation.
+  ## Returns true on asuccess and false otherwise
+  when obj is SecretKey:
+    result = obj.intVal.fromHex(hexStr)
+  else:
+    result = obj.point.fromHex(hexStr)
+
+func fromBytes*[T: SecretKey|PublicKey|Signature|ProofOfPossession](
+       obj: var T,
+       raw: openarray[byte]
+      ): bool {.inline.} =
+  ## Initialize a BLS signature scheme object from
+  ## its raw bytes representation.
+  ## Returns true on success and false otherwise
+  when obj is SecretKey:
+    result = obj.intVal.fromBytes(hexStr)
+  else:
+    result = obj.point.fromBytes(hexStr)
+
+func toHex*(obj: SecretKey|PublicKey|Signature|ProofOfPossession): string =
+  ## Return the hex representation of a BLS signature scheme object
+  ## Signature and Proof-of-posessions are serialized in compressed form
+  when obj is SecretKey:
+    result = obj.intVal.toHex()
+  else:
+    result = obj.point.toHex()