assignments using our basic gate

* WIP: currently `gates::flex_gate` is updated and passes basic test

check that the indicator witness was equal to 1 when non-zero.
* Previously the constraint ensured that `ind[i] = 0` when `idx != i`
  however `ind[idx]` could be anything!!!

* this function was not used anywhere

and there is a break point in that overlap
* we need to copy a cell between columns to fix

* In keygen mode, the `CircuitBuilder` will clone the `ThreadBuilder`
  instead of `take`ing it because the same circuit is used for both vk
gen and pk gen. This could lead to more memory usage for pk gen.

compatibility after halo2-base update

* add multi-thread witness assignment support for `variable_base_msm`
  and `fixed_base_msm`
* batch size 100 MSM witness generation went from 500ms -> 100ms

* Sync with updates in `halo2_proofs_axiom`
* `assign_advice` not longer returns `Result` so no more `unwrap`

* remove `size_hint` in `inner_product_simple`

* change other uses of `size_hint` to follow with `assert_eq!` instead
  of `debug_assert_eq!`

* fix: change `debug_assert` in `decompose_u64_digits_limbs` to restrict
`bit_len < 64` and `decompose_biguint` to `64 <= bit_len < 128`
* add more comprehensive tests for above two functions

* chore: clippy --fix

* Feat/add readme (#4)

* feat: add README

* feat: re-enable `secp256k1` module with updated tests

* chore: fix result println

* chore: update Cargo halo2_proofs_axiom to axiom/dev branch

* compatibility update with `halo2_proofs_axiom`

Co-authored-by: Matthias Seitz <[email protected]>

* chore: expose gate_builder.unknown

* feat: `GateThreadBuilder::assign_all` takes assigned_{advices,constants}
    as input instead of new hashmap, in case we want to constrain equalities
    for cells not belonging to this builder

* chore: update halo2-pse tag

* fix: `GateThreadBuilder::assign_all` now returns `HashMap`s of
    assigned cells for external equality constraints (e.g., instance cells,
    `AssignedCells` from chips not using halo2-lib).

* fix: `assign_all` was not assigning constants as desired: it was
    assigning a new constant per context. This leads to confusion and
    possible undesired consequences down the line.

*fix(BUG): `GateChip::idx_to_indicator` still had soundness bug where at index
`idx` the value could be 0 or 1 (instead of only 1)

* feat: add some function documentation

* test(idx_to_indicator): add comprehensive tests
* both positive and negative tests

* chore: update halo2-ecc version to 0.3.0

* fix(BUG): `FpChip::assert_equal` had `a` instead of `b` typo

* feat: add tests for `FpChip::assert_eq`
* positive and negative tests

* feat: move `GateCircuitBuilder::synthesize` to `sub_synthesize` function
which also returns the assigned advices.

* reduces code duplication between `GateCircuitBuilder::synthesize` and
  `RangeCircuitBuilder::synthesize` and also makes it easier to assign
public instances elsewhere (e.g., snark-verifier)

* feat: remove `Mutex` to prevent non-deterministism

* In variable and fixed base `msm_par` functions, remove use of
  `Mutex<GateThreadBuilder>` because even the `Mutex` is not thread-
  safe in the sense that: if you let `Mutex` decide order
  that `GateThreadBuilder` is unlocked, you may still add Contexts to
  the builder in a non-deterministic order.

* fix: `fixed_base::msm_par` loading new zeros

* In `msm_par` each parallelized context was loading a new zero via
  `ctx.load_zero()`
* This led to using more cells than the non-parallelized version
* In `fixed_base_msm_in`, the if statement depending on
  `rayon::current_number_threads` leads to inconsistent proving keys
between different machines. This has been removed and now uses a fixed
number `25`.

* Allow `assign_all` also if `witness_gen_only = true`

* adds draft documentation for range.rs

* draft docs for lib.rs, utiils.rs, builder.rs

* fix: add suggested doc edits for range.rs

* docs: add draft documentation for flex_gate.rs

* fix: range.rs doc capitalization error

* fix: suggested edits for utils.rs docs

* fix: resolve comments for range.rs docs

* fix: resolve comments on flex_gate.rs docs

* fix: resolve comments for lib.rs, util.rs docs

* fix: resolve comments for builder.rs docs

* chore: use `info!` instead of `println` for params

* Allow `assign_all` also if `witness_gen_only = true`

* Fix: `inner_product_left_last` size hint (#25)

* docs: minor fixes

---------

Co-authored-by: PatStiles <[email protected]>

* feat: smart `RangeCircuitBuilder`

Allow `RangeCircuitBuilder` to not create lookup table if it detects
that there's nothing to look up.

* feat: add `RangeWithInstanceCircuitBuilder`

* Moved from `snark-verifier-sdk`
* Also made this circuit builder smart so it doesn't load lookup table
  if not necessary
    * In particular this can also be used as a
      `GateWithInstanceCircuitBuilder`

* chore: derive Eq for CircuitBuilderStage

* fix: RangeConfig should not unwrap LOOKUP_BITS

* feat: add flex_gate_test.rs and pos add() test

* feat: add pos sub() test

* feat: add pos neg() test

* feat: add pos mul() test

* feat: add pos mul_add() test

* feat: add pos mul_not() test

* feat: add pos assert_bit

* feat: add pos div_unsafe() test

* feat: add pos assert_is_const test

* feat: add pos inner_product() test

* feat: add pos inner_product_left_last() test

* feat: add pos inner_product_with_sums test

* feat: add pos sum_products_with_coeff_and_var test

* feat: add pos and() test

* feat: add pos not() test

* feat: add pos select() test

* feat: add pos or_and() test

* feat: add pos bits_to_indicator() test

* feat: add pos idx_to_indicator() test

* feat: add pos select_by_indicator() test

* feat: add pos select_from_idx() test

* feat: add pos is_zero() test

* feat: add pos is_equal() test

* feat: add pos num_to_bits() test

* feat: add pos lagrange_eval() test

* feat: add pos get_field_element() test

* feat: add pos range_check() tests

* feat: add pos check_less_than() test

* feat: add pos check_less_than_safe() test

* feat: add pos check_big_less_than_safe() test

* feat: add pos is_less_than() test

* feat: add pos is_less_than_safe() test

* feat: add pos is_big_less_than_safe() test

* feat: add pos div_mod() test

* feat: add pos get_last_bit() test

* feat: add pos div_mod_var() test

* fix: pass slices into test functions not arrays

* feat: Add pos property tests for flex_gate

* feat: Add positive property tests for flex_gate

* feat: add pos property tests for range_check.rs

* feat: add neg pranking test for idx_to_indicator

* fix: change div_mod_var test values

* feat(refactor): refactor property tests

* fix: fix neg test, assert_const, assert_bit

* fix: failing prop tests

* feat: expand negative testing is_less_than_failing

* fix: Circuit overflow errors on neg tests

* fix: prop_test_mul_not

* fix: everything but get_last_bit & lagrange

* fix: clippy

* fix: set LOOKUP_BITS in range tests, make range check neg test more

robust

* fix: neg_prop_tests cannot prank inputs

Inputs have many copy constraints; pranking initial input will cause all
copy constraints to fail

* fix: test_is_big_less_than_safe, 240 bits max

* Didn't want to change current `is_less_than` implementation, which in
  order to optimize lookups for smaller bits, only works when inputs
have at most `(F::CAPACITY // lookup_bits - 1) * lookup_bits` bits

* fix: inline doc for lagrange_and_eval

* Remove proptest for lagrange_and_eval and leave as todo

* tests: add readme about serial execution

---------

Co-authored-by: Jonathan Wang <[email protected]>

NOTE: current ecdsa requires `r, s` to be given as proper CRT integers

TODO: newtypes to guard this assumption

)

fix: guard `ScalarField` to be little-endian

2 embarassing errors:
* Witness gen for last bit was wrong (used xor instead of &)
* `ctx.get` was called after `range_check` so it was getting the wrong
  cell

feat: add documentation for all debug_asserts

Add `divide` that checks denomintor is nonzero.
Add documentation in cases where `divide_unsafe` is used.

* feat: add new types `ProperUint` and `ProperCrtUint`

To guard around assumptions about big integer representations

* fix: remove unused `FixedAssignedCRTInteger`

* feat: use new types for bigint and field chips

New types now guard for different assumptions on non-native bigint
arithmetic. Distinguish between:
- Overflow CRT integers
- Proper BigUint with native part derived from limbs
- Field elements where inequality < modulus is checked

Also add type to help guard for inequality check in
ec_add_unequal_strict

Rust traits did not play so nicely with references, so I had to switch
many functions to move inputs instead of borrow by reference. However to
avoid writing `clone` everywhere, we allow conversion `From` reference
to the new type via cloning.

* feat: use `ProperUint` for `big_less_than`

* feat(ecc): add fns for assign private witness points

that constrain point to lie on curve

* fix: unnecessary lifetimes

* chore: remove clones

* feat: allow `msm_par` to return identity point

* feat: handle point at infinity

`multi_scalar_multiply` and `multi_exp_par` now handle point at infinity
completely

Add docs for `ec_add_unequal, ec_sub_unequal, ec_double_and_add_unequal`
to specify point at infinity leads to undefined behavior

* `msm` implementations now always use `ec_{add,sub}_unequal` in strict
mode for safety
* Add docs to `scalar_multiply` and a flag to specify when it's safe to
  turn off some strict assumptions

Multi-threading of witness generation is tricky because one has to
ensure the circuit column assignment order stays deterministic. To
ensure good developer experience / avoiding pitfalls, we provide a new
helper function for this.

Co-authored-by: Jonathan Wang <[email protected]>

We still require fixed base points to be non-identity, but now handle
the case when scalars may be zero or the final MSM value is identity
point.

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* chore: sync with release-0.3.0 and update CI

Co-authored-by: yulliakot <[email protected]>
Co-authored-by: yuliakot <[email protected]>

cannot multi-thread tests involving lookups due to environment variables

This test doesn't run any prover so the input must satisfy range check
assumption. More serious coverage is provided by
`prop_test_neg_is_less_than_safe`

* feat: add halo2-base readme

* fix: readme formatting

* fix: readme edits

* fix: grammer

* fix: use relative links and formatting

* fix: formatting

* feat: add RangeCircuitBuilder description

* feat: rewording and small edits

---------

Co-authored-by: PatStiles <[email protected]>

* More ecdsa tests

* Update mod.rs

* Update tests.rs

* Update ecdsa.rs

* Update ecdsa.rs

* Update ecdsa.rs

* msm tests

* Update mod.rs

* Update msm_sum_infinity.rs

* fix: ec_sub_strict was panicing when output is identity

* affects the MSM functions: right now if the answer is identity, there
  will be a panic due to divide by 0 instead of just returning 0
* there could be a more optimal solution, but due to the traits for
  EccChip, we just generate a random point solely to avoid divide by 0
in the case of identity point

* Fix/fb msm zero (#77)

* fix: fixed_base scalar multiply for [-1]P

* feat: use `multi_scalar_multiply` instead of `scalar_multiply`

* to reduce code maintanence / redundancy

* fix: add back scalar_multiply using any_point

* feat: remove flag from variable base `scalar_multiply`

* feat: add scalar multiply tests for secp256k1

* fix: variable scalar_multiply last select

* Fix/msm tests output identity (#75)

* fixed base msm tests for output infinity

* fixed base msm tests for output infinity

---------

Co-authored-by: yulliakot <[email protected]>

* feat: add tests and update CI

---------

Co-authored-by: yuliakot <[email protected]>
Co-authored-by: yulliakot <[email protected]>

---------

Co-authored-by: yulliakot <[email protected]>
Co-authored-by: yuliakot <[email protected]>

* Add SafeType

* Refactor & add testing

* Add doc comment

* Refactor SafeChip

* Move gen_proof/check_proof to utils

* Fix merge issues

* feat: add SafeAddress and SafeUint160

* fix incorrect byte size

* feat(base): range_check 0 bits by asserting is zero

* chore: add range_check 0 bits test

* feat(ecc): `FpChip::range_check` now works with `max_bits < n * (k-1)`

* feat: stop using env var to pass around FLEX_GATE_CONFIG_PARAMS and

LOOKUP_BITS. Bad for testing (multi-threaded issues). Now we use
thread_local to have a global static for these config params that can be
passed around.

* chore: make utils folder and move some files

* Fix halo2 base tests naming (#76)

* feat: `BaseConfig` to switch between `FlexGateConfig` and `RangeConfig`

- `RangeCircuitBuilder` now uses `BaseConfig` to auto-decide whether to
  create lookup table or not.
    - In the future this should be renamed `BaseCircuitBuilder` or just
      `CircuitBuilder`, but for backwards compatibility we leave the name for now.
- `GateCircuitBuilder` no longer implements `Circuit` because we want to
  switch to having everyone just use `RangeCircuitBuilder`.
- Tests won't compile now because we still need to refactor

* feat: refactored halo2-base tests to use new test suite

* feat: remove use of env var in halo2-ecc

CI now can just run `cargo test`

* feat: remove use of env var from zkevm-keccak

* Add zkevm-keccak test to CI

* chore: fix CI

* chore: add lint to CI

* chore: make Baseconfig fns public

* fix(test): zkevm-keccak test should have `first_pass = SKIP_FIRST_PASS`

Currently with `first_pass = true`, it skips the first pass, but when
feature "halo2-axiom" is used, there is only one pass of `synthesize` so
the whole thing gets skipped. Mea culpa!

---------

Co-authored-by: Xinding Wei <[email protected]>

* feat: add debugging functions

Functions only available for testing:
* `ctx.debug_assert_false` for debug break point to search for other
  constrain failures in mock prover
* `assigned_value.debug_prank(prank_value)` to prank witness values for
  negative tests

* chore: code pretty

* wip: change import to ff v0.13

* feat: remove `GateInstructions::get_field_element`

halo2curves now has `bn256-table` which creates table of small field
elements at compile time, so we should just use `F::from` always. This
also improves readability.

* chore: fix syntax and imports after update

* chore: add asm feature

* chore: workspace.resolver = 2

* chore: update ethers-core

* chore: add jemallocator feature to zkevm-keccak crate

* test: add bigger test case to keccak prover

* feat: use `configure_with_params`

remove `thread_local!` usage

* chore: bump zkevm-keccak version to 0.1.1

* feat: add `GateThreadBuilder::from_stage` for convenience

* chore: fixes

* fix: removed `lookup_bits` from `GateThreadBuilder::config`

* fix: debug_assert_false should load witness for debugging

* chore: use unreachable to document that Circuit::configure is never used

* chore: fix comment

* feat(keccak): use configure_with_params

* chore: fix halo2-pse errors

* chore: change halo2_proofs to main

* Add `sub_mul` to GateInstructions

* Add `sub_mul` prop test

* Add Poseidon chip

* chore: minor fixes

* test(poseidon): add compatbility tests

Cherry-picked from #98

Co-authored-by: Antonio Mejías Gil <[email protected]>

* chore: minor refactor to more closely match snark-verifier

https://github.com/axiom-crypto/snark-verifier/blob/main/snark-verifier/src/util/hash/poseidon.rs

---------

Co-authored-by: Xinding Wei <[email protected]>
Co-authored-by: Jonathan Wang <[email protected]>
Co-authored-by: Antonio Mejías Gil <[email protected]>

* feat: add VariableByteArray

* fix: correct type in panic msg

* feat: make MAX_VAR_LEN const generic

* feat: add `SafeBool` and `SafeByte` types

These are very common so we have separate wrapper to avoid the extra length 1
vector heap allocation.

* wip: add VarLenBytes

* Refactor VarLenBytes
Add VarLenBytesVec and FixLenBytes
Fix tests

* Add unsafe methods for bytes
Address NITs

---------

Co-authored-by: Jonathan Wang <[email protected]>
Co-authored-by: Xinding Wei <[email protected]>

Co-authored-by: Xinding Wei <[email protected]>

feat(base): add `select_array_by_indicator` to `GateInstructions`

* cleanup: use test-utils for benching

* feat: add `{gen,check}_proof_with_instances`

* feat: add initial `bench_builder` cmd to `BaseTester`

* fix: cargo fmt

* Add Poseidon hasher

* Fix test/lint

* Fix nits

* Fix lint

* Fix nits & add comments

* Add prover test

* Fix CI

* chore: rename crate zkevm-keccak to zkevm-hashes

* fix: add `input_len` back to `KeccakTable`

* chore: move keccak specific constants to `keccak_packed_multi/util`

* Fix test

---------

Co-authored-by: Jonathan Wang <[email protected]>

Fix CI for zkevm hashes

Split keccak implementation into multiple files

feat: constant visibility changes

* Replace raw keccak output instead of RLCOC

* Fix lint

* Add comments & improve expression performance

* feat: add `VirtualRegionManager` trait

Create `CopyConstraintManager` to manage global copy constraints.

* wip: separate `SinglePhaseGateManager` and `CopyConstraintManager`

`GateThreadBuilder` was very messy before

Todo:
- Split out lookup functionality

* wip: add `LookupAnyManager`

* wip: `RangeChip` with `LookupAnyManager`

reorg:
- previous `builder/threads` moved to `flex_gate/threads` because
it is all part of `FlexGateConfig` advice assignment logic
- `builder` moved to `range/circuit/builder.rs` as it is part of the
  assignment logic of `RangeCircuitBuilder`

* feat: working `BaseCircuitBuilder` backwards compatible

- `GateThreadBuilder` has become `MultiPhaseCoreManager`
- Some of the functionality has been moved into `BaseCircuitBuilder`,
  which is a generalization of `RangeCircuitBuilder`
- Some fixes on virtual managers because keygen calls `synthesize` twice
  (once for vk, once for pk) so can't drop

* fix: update halo2-ecc and sort `constant_equalities`

Sort `constant_equalities` to ensure deterministism.
Update `halo2-ecc` (mostly the tests) with new circuit builder format.

* fix: `LookupAnyManager` drop check `Arc` strong_count

* feat: add back single column lookup with selector

Special case: if only single advice column that you need to lookup, you
can create a selector and enable lookup on that column.
This means you add 1 selector column, instead of 1 advice column.

Only using this for `RangeConfig` and not generalizing it for now.

* feat: add example of dynamic lookup memory table

* Bump versions to 0.4.0

* chore: re-enable poseidon and safe_types

`Drop` for managers no longer panics because rust `should_panic` test
cannot handle non-unwinding panics.

* chore: remove `row_offset` from `assigned_advices`

This PR was merged: privacy-scaling-explorations/halo2#192

* chore: move `range::circuit` to `gates::circuit`

* nits: address review comments

* feat: add `num_instance_columns` to `BaseCircuitParams`

No longer a const generic

* chore(CI): use larger runner

* chore: switch `halo2_proofs` branch to `main`

* chore: use `raw_assign_{advice,fixed}` in keccak

* PoseidonHasher supports multiple inputs in compact format

* Add comments

* Remove unnecessary uses

* Expose Keccack raw inputs in bytes instead of input RLCs

* Fix column name in comments

* Add comments

* Compress 8 bytes of inputs into a single witness

* chore: add some comments

* Rewrite gates

* Fix comments & typos

* Fix naming

* Add comments

* Selector improvement

* Remove unused

---------

Co-authored-by: Jonathan Wang <[email protected]>

* feat: make `single_phase::assign_with_constraints` generic

Use const generic for max rotations accessed by the vertical gate.
This way we can re-use the code for RLC gate.

* chore: make single_phase pub

* feat: add safety check for overlapping gates

* chore: add convenience function to `BaseConfig`

* feat: add `deep_clone` to `BaseCircuitBuilder`

We sometimes want to clone `BaseCircuitBuilder` completely (for example
to re-run witness generation). The derived clone only clones the shared
references, instead of the underlying objects.

Because the thread will default to phase 0.

* feat: add `clear` function to circuit builder and managers

* feat: add `BaseConfig::initialize`

* fix: break points for multiphase

* fix: clear should not change phase

* chore: remove dbg

Backend implementation of `constrain_equal` depends on the order in
which you add equality constraints, so it is not thread-safe...