chunking in o1js #45
chunking in o1js #45
Conversation
|
|
||
| ## Drawbacks | ||
|
|
||
| Proving larger circuits will increase proving time and memory consumption. Given the memory limit in most browsers and WASM, this could cause problems for users trying to prove very large circuits. |
|
|
||
| Chunking working in [OCaml and Pickles](https://github.com/MinaProtocol/mina/blob/develop/src/lib/pickles/test/chunked_circuits/test_chunked_circuits.ml) directly. | ||
|
|
||
| ## Unresolved questions |
There was a problem hiding this comment.
It looks like some additional work may be required in pickles to resolve these issues. If so, we should add these to the Technical Implementation section above to account for all the work. The added implementation items can simply give a brief overview of the neceessary fix and refer to this section (Unresolved questions) for further info
|
|
||
| In order to make chunking work in o1js, we have to modify the `Pickles.compile` to accept the number of chunks that are required in order to construct the proof. The Pickles [test](https://github.com/MinaProtocol/mina/blob/develop/src/lib/pickles/test/chunked_circuits/test_chunked_circuits.ml) serves as an example on how to modify the `Pickles.compile` function to prove larger circuits and specify the amount of chunks. | ||
|
|
||
| Calculating how many chunks are needed to prove a circuit depends on the amount of constraints used in a circuit. Before passing the parameter to the function and compiling the circuit, we have to calculate the amount of chunks needed. Luckily, o1js already has a function that can inspect methods and count the amount of constraints, we get this information by calling `.analyzeMethods()` on the `ZkProgram` or `SmartContract`. |
There was a problem hiding this comment.
It would be nice to write out the formula for computing the chunks.
Note that Pickles adds a small amount of overhead circuit which isn't captured by analyzeMethods(). I'm not sure if it's necessary to include that in the estimate for the number of chunks.
But it does mean that there's an edge case in which the automatic approach fails: If the Pickles overhead slightly pushes the circuit size over the next power of two and makes it require more chunks than estimated.
To handle a failure like that, I think it's necessary to allow users to override numChunks through the compile() interfaces
| 2 What is the maximum allowed number chunks? From my understanding, setting iteration to `2^20` and `num_chunks:16` requires that the wrap domain must be `16`, but it only accepts `N0 | N1 | N2`. Setting the wrap domain to anything `<= N2` yields an error. How do we deal with large number chunks and constraints? `override_wrap_domain` is of type `Pickles_base.Proofs_verified.t` which only accepts `N0 | N1 | N2`. | ||
| See `large_wrap_domain.ml` for this case. `This circuit was compiled for proofs using the wrap domain of size 15, but the actual wrap domain size for the circuit has size 16` | ||
|
|
||
| 3 Running a small MVP of Chunking in o1js through the bindings yields the following error: `there are not enough random rows to achieve zero-knowledge (expected: 3, got: 2)`. It seems like some values aren't passed correctly. This also needs to be resolved during implementation. |
There was a problem hiding this comment.
Might this be related to the hack/workaround present in the ocaml chunking examples, which is not done in pickles_bindings.ml?
(* We must now appease the permutation argument gods, to ensure
that the 7th permuted column has polynomial degree larger
than 2^16, and thus that its high chunks are non-zero.
Suckiness of linearization strikes again!
*)There was a problem hiding this comment.
It might, although I did try different configurations with different circuits. If this is something that pops up often, we would need a way to better predict when it happens and a way to communicate that to the user if we cant fix it automatically
There was a problem hiding this comment.
note: I did write the circuit in o1js directly and passed the values manually to Pickles.compile, but good point: I will try and run the example via the bindings directly from OCaml to see if the issue persists
There was a problem hiding this comment.
I meant that maybe we just need to add that workaround to pickles_bindings.ml
There was a problem hiding this comment.
I don't understand what it does, but AFAICT it adds a generic gate with a special kind of witness layout which is different from anything that would be added in a normal circuit
There was a problem hiding this comment.
yea, but I also wanna make sure that the circuits I test via the bindings are the same as in the OCaml test, so I'll run that same circuit via the OCaml bindings just to check if the problem is fixed by the workaround or if it's something deeper in the bindings
There was a problem hiding this comment.
I did run the original test (including the mentioned fix) via the bindings, seems like the issue still persists and needs a little more research
There was a problem hiding this comment.
Ok. Maybe we're missing proper ZK rows handling in the wasm bindings
| 2 What is the maximum allowed number chunks? From my understanding, setting iteration to `2^20` and `num_chunks:16` requires that the wrap domain must be `16`, but it only accepts `N0 | N1 | N2`. Setting the wrap domain to anything `<= N2` yields an error. How do we deal with large number chunks and constraints? `override_wrap_domain` is of type `Pickles_base.Proofs_verified.t` which only accepts `N0 | N1 | N2`. | ||
| See `large_wrap_domain.ml` for this case. `This circuit was compiled for proofs using the wrap domain of size 15, but the actual wrap domain size for the circuit has size 16` |
There was a problem hiding this comment.
Interesting! I wouldn't even have thought that number of chunks influences the wrap domain size.
There was a problem hiding this comment.
From Matthew (as below):
The wrap domain size is capped -- and we don't do chunking there -- so there's a limit on the number of chunks we can use until / unless we also chunk the wrap proof
|
|
||
| While trying to prototype Chunking in o1js, I came up with the following unresolved questions. The tests can be found in [this](https://github.com/MinaProtocol/mina/tree/chunking_rfc_unresolved/src/lib/pickles/test/chunked_circuits) branch. | ||
|
|
||
| 1 Changing the loop that constructs the circuit from `2^17` (roughly `2^16` constraints) iterations to `2^18` iterations (or `2^17` constraints) and also adjusting `num_chunks: 4` and `override_wrap_domain:N1` compiles the program correctly, generates a proof _but_ doesn't verify the proof: |
There was a problem hiding this comment.
Wrap circuit also needs chunking
No description provided.