Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: hello-world namespace support for ADVZ VID scheme #389

Merged
merged 56 commits into from
Nov 7, 2023
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
9ca622e
restore simple bytes-to-field from https://github.com/EspressoSystems…
ggutoski Oct 19, 2023
8f0d7f5
rename vid/mod.rs -> vid.rs
ggutoski Oct 23, 2023
f25b9e1
new assoc type VidScheme::Payload instead of Vec<u8>
ggutoski Oct 24, 2023
20ac68f
tidy: move boilerplate to the bottom
ggutoski Oct 24, 2023
a1070c9
new Namespacer trait skeleton
ggutoski Oct 25, 2023
ddccddb
wip: untested impl for namespace_verify
ggutoski Oct 25, 2023
f146fd2
port prove_namespace test to advz/namespaces.rs
ggutoski Oct 25, 2023
af98cb6
wip: untested implementation of data_proof
ggutoski Oct 26, 2023
75011f6
fix: use ifft instead of fft, debugging code for posterity
ggutoski Oct 27, 2023
efc81c5
remove debugging code
ggutoski Oct 27, 2023
2e65705
fix bug in modulus_byte_len, should refactor later
ggutoski Oct 27, 2023
b8f1505
test for data_proof
ggutoski Oct 27, 2023
29187f5
impl data_verify, fix bug fft -> ifft
ggutoski Oct 30, 2023
2cc8acb
refactor polynomial construction
ggutoski Oct 30, 2023
26bb9ec
refactor computation of block hash from polynomial commitments
ggutoski Oct 30, 2023
2453404
delete superfluous modulus_byte_len
ggutoski Oct 30, 2023
c753718
tidy index conversion utils
ggutoski Oct 30, 2023
08a536e
tidy
ggutoski Oct 30, 2023
7d36361
fix test build (my vscode settings has std feature enabled by default…
ggutoski Oct 31, 2023
293ed1a
update changelog (is this always worthwhile?)
ggutoski Oct 31, 2023
96e8f87
remove broken rustdoc links
ggutoski Oct 31, 2023
4dd89f4
add timer data to VID disperse, remove it temmporarily from KZG to re…
ggutoski Nov 1, 2023
58feac5
remove namespace_verify, add chunk_prove, chunk_verify
ggutoski Nov 2, 2023
daabb28
wip: stubs
ggutoski Nov 2, 2023
cc5e3c6
new chunk_proof_2 use Range instead of start, len
ggutoski Nov 3, 2023
7054ea4
wip ugly debugging for index gymnastics, forbid 0-length data
ggutoski Nov 3, 2023
76998f7
tighten checks in chunk_xxx2()
ggutoski Nov 3, 2023
373602b
tidy test
ggutoski Nov 3, 2023
59376a6
wip data_xxx2 methods, untested
ggutoski Nov 4, 2023
9bc5d33
test pass for data_verify2 (yay)
ggutoski Nov 5, 2023
0ea565c
remove old [data|chunk]_[proof|verify] methods
ggutoski Nov 5, 2023
054ae18
remove unneeded Payload struct
ggutoski Nov 5, 2023
6c1f161
replace Namespacer with 2 impls for PayloadProver
ggutoski Nov 5, 2023
4089612
rename files: namespace.rs -> payload_prover.rs
ggutoski Nov 5, 2023
0ae66d1
refactor common arg-checking code
ggutoski Nov 6, 2023
7b31498
clean up TODOs
ggutoski Nov 6, 2023
6ab15f3
fix bench
ggutoski Nov 6, 2023
5b19e06
tidy
ggutoski Nov 6, 2023
cd6e341
coeffs_iter -> evals_iter as per https://github.com/EspressoSystems/j…
ggutoski Nov 6, 2023
8f2240b
address https://github.com/EspressoSystems/jellyfish/pull/389#discuss…
ggutoski Nov 6, 2023
ff764b4
add commit arg to verify_share()
ggutoski Nov 6, 2023
89c9fea
add issue comment for https://github.com/EspressoSystems/jellyfish/pu…
ggutoski Nov 6, 2023
93e396e
clippy and fmt
ggutoski Nov 6, 2023
254c2e5
clarify rustdoc
ggutoski Nov 6, 2023
beeaea9
fix bench
ggutoski Nov 6, 2023
39d16e8
move bytes_to_field, etc from conversion.rs to new bytes_to_field.rs
ggutoski Nov 6, 2023
4b3a806
replace compile_time_checks with new elem_byte_capacity owned by advz
ggutoski Nov 6, 2023
7dcb84f
fmt and doctest
ggutoski Nov 6, 2023
a067833
timer tidy as per https://github.com/EspressoSystems/jellyfish/pull/3…
ggutoski Nov 6, 2023
b7e1480
fix rustdoc
ggutoski Nov 6, 2023
68453c6
fmt
ggutoski Nov 6, 2023
613a097
Merge branch 'main' into gg/vid-namespace
ggutoski Nov 6, 2023
f399a4f
new Statement type to simplify args for payload_verify as per https:/…
ggutoski Nov 7, 2023
4805bfb
fmt
ggutoski Nov 7, 2023
d51f76e
Merge branch 'main' into gg/vid-namespace
ggutoski Nov 7, 2023
eb8fde4
rename proof types, tidy, comment
ggutoski Nov 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ and follow [semantic versioning](https://semver.org/) for our releases.
- [#343](https://github.com/EspressoSystems/jellyfish/pull/343) Rescue parameter for `ark_bn254::Fq`
- [#362](https://github.com/EspressoSystems/jellyfish/pull/362) Derive Eq, Hash at a bunch of places
- [#381](https://github.com/EspressoSystems/jellyfish/pull/381) VID take iterator instead of slice
- [#389](https://github.com/EspressoSystems/jellyfish/pull/389) Hello-world namespace support for ADVZ VID scheme

### Changed

Expand Down
7 changes: 5 additions & 2 deletions primitives/benches/advz.rs
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,11 @@ mod feature_gated {
// run all benches for each payload_byte_lens
for len in payload_byte_lens {
// random payload data
let mut payload_bytes = vec![0u8; len];
rng.fill_bytes(&mut payload_bytes);
let payload_bytes = {
let mut payload_bytes = vec![0u8; len];
rng.fill_bytes(&mut payload_bytes);
payload_bytes
};

let benchmark_group_name =
|op_name| format!("advz_{}_{}_{}KB", pairing_name, op_name, len / KB);
Expand Down
17 changes: 10 additions & 7 deletions primitives/src/pcs/univariate_kzg/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,9 @@ impl<E: Pairing> PolynomialCommitmentScheme for UnivariateKzgPCS<E> {
poly: &Self::Polynomial,
) -> Result<Self::Commitment, PCSError> {
let prover_param = prover_param.borrow();
let commit_time =
start_timer!(|| format!("Committing to polynomial of degree {} ", poly.degree()));
// let commit_time =
chancharles92 marked this conversation as resolved.
Show resolved Hide resolved
// start_timer!(|| format!("Committing to polynomial of degree {} ",
// poly.degree()));

if poly.degree() > prover_param.powers_of_g.len() {
return Err(PCSError::InvalidParameters(format!(
Expand All @@ -105,15 +106,16 @@ impl<E: Pairing> PolynomialCommitmentScheme for UnivariateKzgPCS<E> {

let (num_leading_zeros, plain_coeffs) = skip_leading_zeros_and_convert_to_bigints(poly);

let msm_time = start_timer!(|| "MSM to compute commitment to plaintext poly");
// let msm_time = start_timer!(|| "MSM to compute commitment to plaintext
// poly");
let commitment = E::G1::msm_bigint(
&prover_param.powers_of_g[num_leading_zeros..],
&plain_coeffs,
)
.into_affine();
end_timer!(msm_time);
// end_timer!(msm_time);

end_timer!(commit_time);
// end_timer!(commit_time);
Ok(Commitment(commitment))
}

Expand Down Expand Up @@ -405,9 +407,10 @@ fn skip_leading_zeros_and_convert_to_bigints<F: PrimeField, P: DenseUVPolynomial
}

fn convert_to_bigints<F: PrimeField>(p: &[F]) -> Vec<F::BigInt> {
let to_bigint_time = start_timer!(|| "Converting polynomial coeffs to bigints");
// let to_bigint_time = start_timer!(|| "Converting polynomial coeffs to
// bigints");
let coeffs = p.iter().map(|s| s.into_bigint()).collect::<Vec<_>>();
end_timer!(to_bigint_time);
// end_timer!(to_bigint_time);
coeffs
}

Expand Down
86 changes: 46 additions & 40 deletions primitives/src/vid/mod.rs → primitives/src/vid.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,41 +7,10 @@
//! Trait and implementation for a Verifiable Information Retrieval (VID).
/// See <https://arxiv.org/abs/2111.12323> section 1.3--1.4 for intro to VID semantics.
use ark_serialize::{CanonicalDeserialize, CanonicalSerialize};
use ark_std::{borrow::Borrow, error::Error, fmt::Debug, hash::Hash, string::String, vec::Vec};
use ark_std::{error::Error, fmt::Debug, hash::Hash, string::String, vec::Vec};
use displaydoc::Display;
use serde::{Deserialize, Serialize};

pub mod advz;

/// The error type for `VidScheme` methods.
#[derive(Display, Debug)]
pub enum VidError {
/// invalid args: {0}
Argument(String),
/// internal error: {0}
Internal(anyhow::Error),
}

impl Error for VidError {}

/// Convenience wrapper to convert any error into a [`VidError`].
///
/// Private fn so as not to expose error conversion API outside this crate
/// as per [stackoverflow](https://stackoverflow.com/a/70057677).
///
/// # No-std support
/// `no_std` mode requires `.map_err(vid)` to convert from a non-`anyhow` error
/// as per [`anyhow` docs](https://docs.rs/anyhow/latest/anyhow/index.html#no-std-support),
fn vid<E>(e: E) -> VidError
where
E: ark_std::fmt::Display + Debug + Send + Sync + 'static,
{
VidError::Internal(anyhow::anyhow!(e))
}

/// Convenience [`Result`] wrapper for [`VidError`].
pub type VidResult<T> = Result<T, VidError>;

/// VID: Verifiable Information Dispersal
pub trait VidScheme {
/// Payload commitment.
Expand All @@ -54,25 +23,27 @@ pub trait VidScheme {
type Common: CanonicalSerialize + CanonicalDeserialize + Clone + Eq + PartialEq + Sync; // TODO https://github.com/EspressoSystems/jellyfish/issues/253

/// Compute a payload commitment
fn commit_only<I>(&self, payload: I) -> VidResult<Self::Commit>
fn commit_only<B>(&self, payload: B) -> VidResult<Self::Commit>
where
I: IntoIterator,
I::Item: Borrow<u8>;
B: AsRef<[u8]>;

/// Compute shares to send to the storage nodes
fn disperse<I>(&self, payload: I) -> VidResult<VidDisperse<Self>>
fn disperse<B>(&self, payload: B) -> VidResult<VidDisperse<Self>>
where
I: IntoIterator,
I::Item: Borrow<u8>;
B: AsRef<[u8]>;

/// Verify a share. Used by both storage node and retrieval client.
/// Why is return type a nested `Result`? See <https://sled.rs/errors>
/// Returns:
/// - VidResult::Err in case of actual error
/// - VidResult::Ok(Result::Err) if verification fails
/// - VidResult::Ok(Result::Ok) if verification succeeds
fn verify_share(&self, share: &Self::Share, common: &Self::Common)
-> VidResult<Result<(), ()>>;
fn verify_share(
&self,
share: &Self::Share,
common: &Self::Common,
commit: &Self::Commit,
) -> VidResult<Result<(), ()>>;

/// Recover payload from shares.
/// Do not verify shares or check recovered payload against anything.
Expand Down Expand Up @@ -105,3 +76,38 @@ pub struct VidDisperse<V: VidScheme + ?Sized> {
/// VID payload commitment.
pub commit: V::Commit,
}

pub mod payload_prover;

pub mod advz; // instantiation of `VidScheme`

// BOILERPLATE: error handling

/// The error type for `VidScheme` methods.
#[derive(Display, Debug)]
pub enum VidError {
/// invalid args: {0}
Argument(String),
/// internal error: {0}
Internal(anyhow::Error),
}

impl Error for VidError {}

/// Convenience wrapper to convert any error into a [`VidError`].
///
/// Private fn so as not to expose error conversion API outside this crate
/// as per [stackoverflow](https://stackoverflow.com/a/70057677).
///
/// # No-std support
/// `no_std` mode requires `.map_err(vid)` to convert from a non-`anyhow` error
/// as per [`anyhow` docs](https://docs.rs/anyhow/latest/anyhow/index.html#no-std-support),
fn vid<E>(e: E) -> VidError
where
E: ark_std::fmt::Display + Debug + Send + Sync + 'static,
{
VidError::Internal(anyhow::anyhow!(e))
}

/// Convenience [`Result`] wrapper for [`VidError`].
pub type VidResult<T> = Result<T, VidError>;
Loading
Loading