Incorporate recent updates to halo2 branch #1656
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Use the current processes bound cores to limit the possible cores that threads can be bound to. This allows core binding to work properly when the lotus-worker service is limited to certain CPUs by cgroups.
`ec-gpu-gen` needs to be updated to v0.5 as v0.4 has dependencies that depend on yanked version. It's an indirect dependency of `bellperson` and `neptune`, which are upgraded here. Moving from `memmap` (which is deprecated) to `memmap2` was also needed als dependencies also switched. `chrono` updated their API, so there was also a small change needed.
Instead of constructing the `PoRepConfig` directly, use a constructor function. This simplifies the code and makes things less error-prone. Fixes #1632. Co-authored-by: 卜翰 <[email protected]>
There were quite a few public type definitions, that were mostly replaced by the `SectorShape*` types. This commit cleans them up and moves them around if appropriate. This should make the code easier to follow and the public API surface smaller. BREAKING CHANGE: `BinaryLCMerkleTree`, `BinaryMerkleTree`, `BinarySubMerkleTree`, `LCMerkleTree`, `LCStore`, `MerkleStore`, `MerkleTree`, `OctLCMerkleTree`, `OctLCSubMerkleTree`, `OctLCTopMerkleTree`, `OctMerkleTree`, `OctSubMerkleTree`, `OctTopMerkleTree`, `QuadLCMerkleTree` and `QuadMerkleTree` are removed from the public interface.
`ec-gpu-gen` needs to be updated to v0.5 as v0.4 has dependencies that depend on yanked version. It's an indirect dependency of `bellperson` and `neptune`, which are upgraded here. Moving from `memmap` (which is deprecated) to `memmap2` was also needed als dependencies also switched. `chrono` updated their API, so there was also a small change needed.
Neptune currently is a fork of pasta_curves. That needs patching as well, in order to get the correct names for the fields out.
…into halo2_pick_updates
storojs72
requested review from
cryptonemo,
DrPeterVanNostrand and
vmx
as code owners
DrPeterVanNostrand
requested changes
Jan 16, 2023
vmx
reviewed
Jan 17, 2023
storojs72
force-pushed
the
halo2_pick_updates
branch
3 times, most recently
from
6927003
to
c132af3
Compare
storojs72
force-pushed
the
halo2_pick_updates
branch
from
c132af3
to
3b74d96
Compare
vmx
approved these changes
Jan 18, 2023
DrPeterVanNostrand
approved these changes
Jan 19, 2023
storojs72
added a commit
that referenced
this pull request
Jan 19, 2023
* fix: use current process binding to limit thread cores (#1633) Use the current processes bound cores to limit the possible cores that threads can be bound to. This allows core binding to work properly when the lotus-worker service is limited to certain CPUs by cgroups. * fix: update ec-gpu-gen (#1638) `ec-gpu-gen` needs to be updated to v0.5 as v0.4 has dependencies that depend on yanked version. It's an indirect dependency of `bellperson` and `neptune`, which are upgraded here. Moving from `memmap` (which is deprecated) to `memmap2` was also needed als dependencies also switched. `chrono` updated their API, so there was also a small change needed. * fix: broken Links in README.md #1637 (#1649) * feat: Introduce PoRepConfig::new_groth16() (#1635) Instead of constructing the `PoRepConfig` directly, use a constructor function. This simplifies the code and makes things less error-prone. Fixes #1632. Co-authored-by: 卜翰 <[email protected]> * fix: clean up tree definitions (#1655) There were quite a few public type definitions, that were mostly replaced by the `SectorShape*` types. This commit cleans them up and moves them around if appropriate. This should make the code easier to follow and the public API surface smaller. BREAKING CHANGE: `BinaryLCMerkleTree`, `BinaryMerkleTree`, `BinarySubMerkleTree`, `LCMerkleTree`, `LCStore`, `MerkleStore`, `MerkleTree`, `OctLCMerkleTree`, `OctLCSubMerkleTree`, `OctLCTopMerkleTree`, `OctMerkleTree`, `OctSubMerkleTree`, `OctTopMerkleTree`, `QuadLCMerkleTree` and `QuadMerkleTree` are removed from the public interface. * fix: update ec-gpu-gen (#1638) `ec-gpu-gen` needs to be updated to v0.5 as v0.4 has dependencies that depend on yanked version. It's an indirect dependency of `bellperson` and `neptune`, which are upgraded here. Moving from `memmap` (which is deprecated) to `memmap2` was also needed als dependencies also switched. `chrono` updated their API, so there was also a small change needed. * chore: update Cargo.lock * fix: there was a memmap -> memmap2 missing * fix: make poseidon tests pass Neptune currently is a fork of pasta_curves. That needs patching as well, in order to get the correct names for the fields out. * ci: Apply rustfmt and fix clippy * Pick relevant branch of neptune * fix: Use SHA256 hasher for binary trees Co-authored-by: Clint Armstrong <[email protected]> Co-authored-by: Volker Mische <[email protected]> Co-authored-by: hanbu97 <[email protected]> Co-authored-by: 卜翰 <[email protected]>
storojs72
added a commit
that referenced
this pull request
Jan 19, 2023
* fix: use current process binding to limit thread cores (#1633) Use the current processes bound cores to limit the possible cores that threads can be bound to. This allows core binding to work properly when the lotus-worker service is limited to certain CPUs by cgroups. * fix: update ec-gpu-gen (#1638) `ec-gpu-gen` needs to be updated to v0.5 as v0.4 has dependencies that depend on yanked version. It's an indirect dependency of `bellperson` and `neptune`, which are upgraded here. Moving from `memmap` (which is deprecated) to `memmap2` was also needed als dependencies also switched. `chrono` updated their API, so there was also a small change needed. * fix: broken Links in README.md #1637 (#1649) * feat: Introduce PoRepConfig::new_groth16() (#1635) Instead of constructing the `PoRepConfig` directly, use a constructor function. This simplifies the code and makes things less error-prone. Fixes #1632. Co-authored-by: 卜翰 <[email protected]> * fix: clean up tree definitions (#1655) There were quite a few public type definitions, that were mostly replaced by the `SectorShape*` types. This commit cleans them up and moves them around if appropriate. This should make the code easier to follow and the public API surface smaller. BREAKING CHANGE: `BinaryLCMerkleTree`, `BinaryMerkleTree`, `BinarySubMerkleTree`, `LCMerkleTree`, `LCStore`, `MerkleStore`, `MerkleTree`, `OctLCMerkleTree`, `OctLCSubMerkleTree`, `OctLCTopMerkleTree`, `OctMerkleTree`, `OctSubMerkleTree`, `OctTopMerkleTree`, `QuadLCMerkleTree` and `QuadMerkleTree` are removed from the public interface. * fix: update ec-gpu-gen (#1638) `ec-gpu-gen` needs to be updated to v0.5 as v0.4 has dependencies that depend on yanked version. It's an indirect dependency of `bellperson` and `neptune`, which are upgraded here. Moving from `memmap` (which is deprecated) to `memmap2` was also needed als dependencies also switched. `chrono` updated their API, so there was also a small change needed. * chore: update Cargo.lock * fix: there was a memmap -> memmap2 missing * fix: make poseidon tests pass Neptune currently is a fork of pasta_curves. That needs patching as well, in order to get the correct names for the fields out. * ci: Apply rustfmt and fix clippy * Pick relevant branch of neptune * fix: Use SHA256 hasher for binary trees Co-authored-by: Clint Armstrong <[email protected]> Co-authored-by: Volker Mische <[email protected]> Co-authored-by: hanbu97 <[email protected]> Co-authored-by: 卜翰 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR (
in conjunction with argumentcomputer/neptune#168see UPD below) attempts to updatehalo2branch to the latest master which I need for further adding docs. I tried couple of times to do it directly, but got weird compilation errors. Looking for debug workaround and experimenting quite a lot I've come up with somehalo2branch "hybrid" that at first incorporates changes from halo2-vmx branch (by cherry-picking commits one by one from 187cb6f to e020458) and, secondly merges commits frommasteron top of that.P.S. As CircleCI is currently broken, I couldn't finally check the build - but locally it compiles and runs tests OK. Once we are able to re-create deploy key, I can check build remotely.Nemo added new deploy key and now CI is working
UPD: After some discussions with @vmx, we decided to rebase one of existing neptune's branches (as it already contained most of changes from neptune's master) and make it single source of Halo2 work - @vmx thanks! - so this PR doesn't depend on argumentcomputer/neptune#168 anymore and uses rebased
halo2branch of neptune.