arkworks integration (paritytech#13031)

* fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix * fix compression * fix compression * fix compression * fix compression * fix compression * fix std leak * fix std leak * fix std leak * merge master * merge master * cargo update * cargo update * cargo update * cargo update * cargo update * use serialize_result * cargo update * cargo update * cargo update * cargo update * reduce boilerplate code * remove host function muls * reduce boilerplate code * remove patches * uuse correct ark-substrate branch * reduce boilerplate code * cleanup * cleanup * proper error handling * derive serialize for error * proper error handling * proper error handling * proper error handling * derive Debug for PairingError * sp-arkworks path * cargo update * adopt tests to error handling * fix tests * cargo update * remove results * deserialize as G2Affine * cargo update * add codex index to PairingError * replace Vec<Vec<u8>> * replace Vec<Vec<u8>> * use into_iter for chunks * use chunks for scalars * fix ersialized_size * use into * collect as vec * collect as vec * no collect Vec * use into_iter * import AffineRepr * fix typo * cargo update * new serialization * fix typo * unwrap results * unwrap results * use correct deserialization * fix bugs, cleanup * correct len * vec without capacity * Revert "vec without capacity" This reverts commit 2b1cd00. * Revert "correct len" This reverts commit b85de86. * Revert "fix bugs, cleanup" This reverts commit eef4c77. * Revert "use correct deserialization" This reverts commit 9eacba9. * Revert "unwrap results" This reverts commit b0df1e1. * Revert "unwrap results" This reverts commit de3cfbd. * Revert "fix typo" This reverts commit c12045d. * Revert "new serialization" This reverts commit e56a088. * Revert "cargo update" This reverts commit 15898da. * Revert "fix typo" This reverts commit c89e963. * Revert "import AffineRepr" This reverts commit 5a103ac. * Revert "use into_iter" This reverts commit 2e31d91. * Revert "no collect Vec" This reverts commit db18dca. * Revert "collect as vec" This reverts commit dd3f809. * Revert "collect as vec" This reverts commit 9167d59. * Revert "use into" This reverts commit 344cfff. * Revert "fix ersialized_size" This reverts commit c6a7609. * Revert "use chunks for scalars" This reverts commit 67987ae. * Revert "use into_iter for chunks" This reverts commit 1ddd6b8. * Revert "replace Vec<Vec<u8>>" This reverts commit 4d3b13c. * cargo update * cargo update * Revert "replace Vec<Vec<u8>>" This reverts commit 4389714. * cargo update * add error * add error * add error * fix typo * fix imports * import coded * import codec * import PairingError * fix patches * sp-arkworks * sp-arkworks * use random values for multiplications * cargo update * fix imports * fix imports * add host functions * re-add mul impls * cargo update * cargo update * cargo update * cargo update * cargo update * cargo update * cargo update * PairingError -> () * remove PairingError * cargo update * cargo update * cargo update * reduce boilerplate code * cargo update * update comments * cargo update * optimize code quality * use ark_scale (paritytech#13954) * use ark_scale * fix tests * fix tests * cleanup & comments * use correct PR branch * hazmat * ed curves, use ArkScaleProjective * Achimcc/arkworks integration remove affine hostcalls (paritytech#13971) * remove affine host-calls * remove affine host-call impls, also in tests * cargo update * ark-substrate: use main branch * cargo update * Achimcc/arkworks integration bandersnatch (paritytech#13977) * use bandersnatch * bandersnatch * add abndersnatch sw msm * use correct PR branch * cargo update * cargo update * fix tests * cleanup * cleanup * fix tests * refactor tests * cargo update * cargo update * cargo update * refactor tests * cleanup & update tests * upgrade arkworks/algebra * cargo update * adopt tests * versioning ark-substrate * cargo update * remove patched deps * bump ark-scale * use crates-io deps * fix doc comments * Cargo.toml, linebreaks at end * reorgainze tests * sp-arkworks -> sp-crypto-ec-utils * move host functions to crypto-ec-utils * fmt * remove sp-ec-crypto-utils from io * remove unwrap from te msm * remove elliptic_curves references in test * elliptic_curves references in test * update doc comments * remove warn missing docs * fmt * cargo update * update doc comments * cargo update * cargo update, bump arkworks, codec versions * bump runtime version in sp-crypto-ec-utils * remove feature flag ec-utils-experimental * crypto-ec-utils -> crypto/ec-utils * tests/ -> test-data/ * update doc comments for signatures * update comments * update doc comments for signatures * fix doc comments * fix doc comments * fix doc comments * fix doc comments * fix doc comments * cleanup * fix doc comments * cargo update * fix doc comments * cargo update
nathanwhit · Jul 19, 2023 · 6006715 · 6006715
1 parent 12faec8
commit 6006715
Show file tree

Hide file tree

Showing 14 changed files with 1,530 additions and 285 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -199,6 +199,7 @@ members = [
 	"primitives/core",
 	"primitives/core/hashing",
 	"primitives/core/hashing/proc-macro",
+	"primitives/crypto/ec-utils",
 	"primitives/database",
 	"primitives/debug-derive",
 	"primitives/externalities",

diff --git a/primitives/crypto/ec-utils/Cargo.toml b/primitives/crypto/ec-utils/Cargo.toml
@@ -0,0 +1,61 @@
+[package]
+name = "sp-crypto-ec-utils"
+version = "0.4.0"
+authors = ["Parity Technologies <[email protected]>"]
+edition = "2021"
+license = "Apache-2.0"
+homepage = "https://substrate.io"
+repository = "https://github.com/paritytech/substrate/"
+
+[package.metadata.docs.rs]
+targets = ["x86_64-unknown-linux-gnu"]
+
+[dependencies]
+ark-serialize = { version = "0.4.2", default-features = false }
+ark-ff = { version = "0.4.2", default-features = false }
+ark-ec = { version = "0.4.2", default-features = false }
+ark-std = { version = "0.4.0", default-features = false }
+ark-bls12-377 = { version = "0.4.0", features = ["curve"], default-features = false }
+ark-bls12-381 = { version = "0.4.0", features = ["curve"], default-features = false }
+ark-bw6-761 = { version = "0.4.0", default-features = false }
+ark-ed-on-bls12-381-bandersnatch = { version = "0.4.0", default-features = false }
+ark-ed-on-bls12-377 = { version = "0.4.0", default-features = false }
+sp-std = { version = "8.0.0", path = "../../std", default-features = false }
+codec = { package = "parity-scale-codec", version = "3.5.0", default-features = false }
+ark-scale = { version = "0.0.3", features = ["hazmat"], default-features = false }
+sp-runtime-interface = { version = "17.0.0", default-features = false, path = "../../runtime-interface" }
+
+[dev-dependencies]
+sp-io = { path = "../../io", default-features = false }
+ark-algebra-test-templates = { version = "0.4.2", default-features = false }
+sp-ark-models = { version = "0.4.0-beta", default-features = false }
+sp-ark-bls12-377 = { version = "0.4.0-beta", default-features = false }
+sp-ark-bls12-381 = { version = "0.4.0-beta", default-features = false }
+sp-ark-bw6-761 = { version = "0.4.0-beta", default-features = false }
+sp-ark-ed-on-bls12-377 = { version = "0.4.0-beta", default-features = false }
+sp-ark-ed-on-bls12-381-bandersnatch = { version = "0.4.0-beta", default-features = false }
+
+[features]
+default = [ "std" ]
+std = [
+	"ark-serialize/std",
+	"ark-ff/std",
+	"ark-ec/std",
+	"ark-std/std",
+	"ark-bls12-377/std",
+	"ark-bls12-381/std",
+	"ark-bw6-761/std",
+	"ark-ed-on-bls12-381-bandersnatch/std",
+	"ark-ed-on-bls12-377/std",
+	"sp-std/std",
+	"codec/std",
+	"ark-scale/std",
+	"sp-runtime-interface/std",
+	"sp-io/std",
+	"ark-algebra-test-templates/std",
+	"sp-ark-bls12-377/std",
+	"sp-ark-bls12-381/std",
+    "sp-ark-bw6-761/std",
+    "sp-ark-ed-on-bls12-377/std",
+    "sp-ark-ed-on-bls12-381-bandersnatch/std",
+]
diff --git a/primitives/crypto/ec-utils/src/bls12_377.rs b/primitives/crypto/ec-utils/src/bls12_377.rs
@@ -0,0 +1,103 @@
+// This file is part of Substrate.
+
+// Copyright (C) 2017-2022 Parity Technologies (UK) Ltd.
+// SPDX-License-Identifier: Apache-2.0
+
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// 	http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Support functions for bls12_377 to improve the performance of
+//! multi_miller_loop, final_exponentiation, msm's and projective
+//! multiplications by host function calls
+
+use crate::utils::{
+	final_exponentiation_generic, msm_sw_generic, mul_projective_generic, multi_miller_loop_generic,
+};
+use ark_bls12_377::{g1, g2, Bls12_377};
+use sp_std::vec::Vec;
+
+/// Compute a multi miller loop through arkworks
+pub fn multi_miller_loop(a: Vec<u8>, b: Vec<u8>) -> Result<Vec<u8>, ()> {
+	multi_miller_loop_generic::<Bls12_377>(a, b)
+}
+
+/// Compute a final exponentiation through arkworks
+pub fn final_exponentiation(target: Vec<u8>) -> Result<Vec<u8>, ()> {
+	final_exponentiation_generic::<Bls12_377>(target)
+}
+
+/// Compute a multi scalar multiplication for short_weierstrass through
+/// arkworks on G1.
+pub fn msm_g1(bases: Vec<u8>, scalars: Vec<u8>) -> Result<Vec<u8>, ()> {
+	msm_sw_generic::<g1::Config>(bases, scalars)
+}
+
+/// Compute a multi scalar multiplication for short_weierstrass through
+/// arkworks on G2.
+pub fn msm_g2(bases: Vec<u8>, scalars: Vec<u8>) -> Result<Vec<u8>, ()> {
+	msm_sw_generic::<g2::Config>(bases, scalars)
+}
+
+/// Compute a projective scalar multiplication for short_weierstrass
+/// through arkworks on G1.
+pub fn mul_projective_g1(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+	mul_projective_generic::<g1::Config>(base, scalar)
+}
+
+/// Compute a projective scalar multiplication for short_weierstrass
+/// through arkworks on G2.
+pub fn mul_projective_g2(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+	mul_projective_generic::<g2::Config>(base, scalar)
+}
+
+#[cfg(test)]
+mod tests {
+	use super::*;
+	use ark_algebra_test_templates::*;
+	use sp_ark_bls12_377::{
+		Bls12_377 as Bls12_377Host, G1Projective as G1ProjectiveHost,
+		G2Projective as G2ProjectiveHost, HostFunctions,
+	};
+
+	#[derive(PartialEq, Eq)]
+	struct Host;
+
+	impl HostFunctions for Host {
+		fn bls12_377_multi_miller_loop(a: Vec<u8>, b: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_377_multi_miller_loop(a, b)
+		}
+		fn bls12_377_final_exponentiation(f12: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_377_final_exponentiation(f12)
+		}
+		fn bls12_377_msm_g1(bases: Vec<u8>, bigints: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_377_msm_g1(bases, bigints)
+		}
+		fn bls12_377_msm_g2(bases: Vec<u8>, bigints: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_377_msm_g2(bases, bigints)
+		}
+		fn bls12_377_mul_projective_g1(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_377_mul_projective_g1(base, scalar)
+		}
+		fn bls12_377_mul_projective_g2(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_377_mul_projective_g2(base, scalar)
+		}
+	}
+
+	type Bls12_377 = Bls12_377Host<Host>;
+	type G1Projective = G1ProjectiveHost<Host>;
+	type G2Projective = G2ProjectiveHost<Host>;
+
+	test_group!(g1; G1Projective; sw);
+	test_group!(g2; G2Projective; sw);
+	test_group!(pairing_output; ark_ec::pairing::PairingOutput<Bls12_377>; msm);
+	test_pairing!(pairing; super::Bls12_377);
+}
diff --git a/primitives/crypto/ec-utils/src/bls12_381.rs b/primitives/crypto/ec-utils/src/bls12_381.rs
@@ -0,0 +1,219 @@
+// This file is part of Substrate.
+
+// Copyright (C) 2017-2022 Parity Technologies (UK) Ltd.
+// SPDX-License-Identifier: Apache-2.0
+
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// 	http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+//! Support functions for bls12_381 to improve the performance of
+//! multi_miller_loop, final_exponentiation, msm's and projective
+//! multiplications by host function calls
+
+use crate::utils::{
+	final_exponentiation_generic, msm_sw_generic, mul_projective_generic, multi_miller_loop_generic,
+};
+use ark_bls12_381::{g1, g2, Bls12_381};
+use sp_std::vec::Vec;
+
+/// Compute a multi miller loop through arkworks
+pub fn multi_miller_loop(a: Vec<u8>, b: Vec<u8>) -> Result<Vec<u8>, ()> {
+	multi_miller_loop_generic::<Bls12_381>(a, b)
+}
+
+/// Compute a final exponentiation through arkworks
+pub fn final_exponentiation(target: Vec<u8>) -> Result<Vec<u8>, ()> {
+	final_exponentiation_generic::<Bls12_381>(target)
+}
+
+/// Compute a multi scalar multiplication for short_weierstrass through
+/// arkworks on G1.
+pub fn msm_g1(bases: Vec<u8>, scalars: Vec<u8>) -> Result<Vec<u8>, ()> {
+	msm_sw_generic::<g1::Config>(bases, scalars)
+}
+
+/// Compute a multi scalar multiplication for short_weierstrass through
+/// arkworks on G2.
+pub fn msm_g2(bases: Vec<u8>, scalars: Vec<u8>) -> Result<Vec<u8>, ()> {
+	msm_sw_generic::<g2::Config>(bases, scalars)
+}
+
+/// Compute a projective scalar multiplication for short_weierstrass
+/// through arkworks on G1.
+pub fn mul_projective_g1(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+	mul_projective_generic::<g1::Config>(base, scalar)
+}
+
+/// Compute a projective scalar multiplication for short_weierstrass
+/// through arkworks on G2.
+pub fn mul_projective_g2(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+	mul_projective_generic::<g2::Config>(base, scalar)
+}
+
+#[cfg(test)]
+mod tests {
+	use super::*;
+	use ark_algebra_test_templates::*;
+	use ark_ec::{AffineRepr, CurveGroup, Group};
+	use ark_ff::{fields::Field, One, Zero};
+	use ark_serialize::{CanonicalDeserialize, CanonicalSerialize, Compress, Validate};
+	use ark_std::{rand::Rng, test_rng, vec, UniformRand};
+	use sp_ark_bls12_381::{
+		fq::Fq, fq2::Fq2, fr::Fr, Bls12_381 as Bls12_381Host, G1Affine as G1AffineHost,
+		G1Projective as G1ProjectiveHost, G2Affine as G2AffineHost,
+		G2Projective as G2ProjectiveHost, HostFunctions,
+	};
+	use sp_ark_models::pairing::PairingOutput;
+
+	#[derive(PartialEq, Eq)]
+	struct Host;
+
+	impl HostFunctions for Host {
+		fn bls12_381_multi_miller_loop(a: Vec<u8>, b: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_381_multi_miller_loop(a, b)
+		}
+		fn bls12_381_final_exponentiation(f12: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_381_final_exponentiation(f12)
+		}
+		fn bls12_381_msm_g1(bases: Vec<u8>, bigints: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_381_msm_g1(bases, bigints)
+		}
+		fn bls12_381_msm_g2(bases: Vec<u8>, bigints: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_381_msm_g2(bases, bigints)
+		}
+		fn bls12_381_mul_projective_g1(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_381_mul_projective_g1(base, scalar)
+		}
+		fn bls12_381_mul_projective_g2(base: Vec<u8>, scalar: Vec<u8>) -> Result<Vec<u8>, ()> {
+			crate::elliptic_curves::bls12_381_mul_projective_g2(base, scalar)
+		}
+	}
+
+	type Bls12_381 = Bls12_381Host<Host>;
+	type G1Projective = G1ProjectiveHost<Host>;
+	type G2Projective = G2ProjectiveHost<Host>;
+	type G1Affine = G1AffineHost<Host>;
+	type G2Affine = G2AffineHost<Host>;
+
+	test_group!(g1; G1Projective; sw);
+	test_group!(g2; G2Projective; sw);
+	test_group!(pairing_output; PairingOutput<Bls12_381>; msm);
+	test_pairing!(ark_pairing; super::Bls12_381);
+
+	#[test]
+	fn test_g1_endomorphism_beta() {
+		assert!(sp_ark_bls12_381::g1::BETA.pow([3u64]).is_one());
+	}
+
+	#[test]
+	fn test_g1_subgroup_membership_via_endomorphism() {
+		let mut rng = test_rng();
+		let generator = G1Projective::rand(&mut rng).into_affine();
+		assert!(generator.is_in_correct_subgroup_assuming_on_curve());
+	}
+
+	#[test]
+	fn test_g1_subgroup_non_membership_via_endomorphism() {
+		let mut rng = test_rng();
+		loop {
+			let x = Fq::rand(&mut rng);
+			let greatest = rng.gen();
+
+			if let Some(p) = G1Affine::get_point_from_x_unchecked(x, greatest) {
+				if !<G1Projective as ark_std::Zero>::is_zero(&p.mul_bigint(Fr::characteristic())) {
+					assert!(!p.is_in_correct_subgroup_assuming_on_curve());
+					return
+				}
+			}
+		}
+	}
+
+	#[test]
+	fn test_g2_subgroup_membership_via_endomorphism() {
+		let mut rng = test_rng();
+		let generator = G2Projective::rand(&mut rng).into_affine();
+		assert!(generator.is_in_correct_subgroup_assuming_on_curve());
+	}
+
+	#[test]
+	fn test_g2_subgroup_non_membership_via_endomorphism() {
+		let mut rng = test_rng();
+		loop {
+			let x = Fq2::rand(&mut rng);
+			let greatest = rng.gen();
+
+			if let Some(p) = G2Affine::get_point_from_x_unchecked(x, greatest) {
+				if !<G2Projective as Zero>::is_zero(&p.mul_bigint(Fr::characteristic())) {
+					assert!(!p.is_in_correct_subgroup_assuming_on_curve());
+					return
+				}
+			}
+		}
+	}
+
+	// Test vectors and macro adapted from https://github.com/zkcrypto/bls12_381/blob/e224ad4ea1babfc582ccd751c2bf128611d10936/src/test-data/mod.rs
+	macro_rules! test_vectors {
+		($projective:ident, $affine:ident, $compress:expr, $expected:ident) => {
+			let mut e = $projective::zero();
+
+			let mut v = vec![];
+			{
+				let mut expected = $expected;
+				for _ in 0..1000 {
+					let e_affine = $affine::from(e);
+					let mut serialized = vec![0u8; e.serialized_size($compress)];
+					e_affine.serialize_with_mode(serialized.as_mut_slice(), $compress).unwrap();
+					v.extend_from_slice(&serialized[..]);
+
+					let mut decoded = serialized;
+					let len_of_encoding = decoded.len();
+					(&mut decoded[..]).copy_from_slice(&expected[0..len_of_encoding]);
+					expected = &expected[len_of_encoding..];
+					let decoded =
+						$affine::deserialize_with_mode(&decoded[..], $compress, Validate::Yes)
+							.unwrap();
+					assert_eq!(e_affine, decoded);
+
+					e += &$projective::generator();
+				}
+			}
+
+			assert_eq!(&v[..], $expected);
+		};
+	}
+
+	#[test]
+	fn g1_compressed_valid_test_vectors() {
+		let bytes: &'static [u8] = include_bytes!("test-data/g1_compressed_valid_test_vectors.dat");
+		test_vectors!(G1Projective, G1Affine, Compress::Yes, bytes);
+	}
+
+	#[test]
+	fn g1_uncompressed_valid_test_vectors() {
+		let bytes: &'static [u8] =
+			include_bytes!("test-data/g1_uncompressed_valid_test_vectors.dat");
+		test_vectors!(G1Projective, G1Affine, Compress::No, bytes);
+	}
+
+	#[test]
+	fn g2_compressed_valid_test_vectors() {
+		let bytes: &'static [u8] = include_bytes!("test-data/g2_compressed_valid_test_vectors.dat");
+		test_vectors!(G2Projective, G2Affine, Compress::Yes, bytes);
+	}
+
+	#[test]
+	fn g2_uncompressed_valid_test_vectors() {
+		let bytes: &'static [u8] =
+			include_bytes!("test-data/g2_uncompressed_valid_test_vectors.dat");
+		test_vectors!(G2Projective, G2Affine, Compress::No, bytes);
+	}
+}