Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add solidity verifier workflow #2749

Merged
merged 114 commits into from
Sep 26, 2023
Merged
Show file tree
Hide file tree
Changes from 106 commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
9de0268
Add solidity verifier workflow
jonybur Sep 19, 2023
bd694b3
Fix install nargo step
jonybur Sep 19, 2023
02a6bab
Fix noirup
jonybur Sep 19, 2023
f0d93d0
Run nargo -v
jonybur Sep 19, 2023
9de8e88
Fix nargo binary path
jonybur Sep 19, 2023
5ba830b
codegen-verifier for contracts
jonybur Sep 19, 2023
489fe64
Fix path
jonybur Sep 19, 2023
2b8b84b
Fix path
jonybur Sep 19, 2023
865a8db
Try fix
jonybur Sep 19, 2023
239efc7
Fix nargo
jonybur Sep 19, 2023
32aaed7
Fix codegen-verifier
jonybur Sep 19, 2023
c60af0c
Fix forge init
jonybur Sep 19, 2023
13f76c9
Fix init forge
jonybur Sep 19, 2023
b6593c3
Forge build
jonybur Sep 19, 2023
3ae69ae
Replace macos for ubuntu
jonybur Sep 19, 2023
bcce077
Fix for ubuntu
jonybur Sep 19, 2023
fe069ac
Move .bashrc
jonybur Sep 19, 2023
a40625d
Try fixes
jonybur Sep 19, 2023
7e76880
Add git config
jonybur Sep 19, 2023
6317c38
Fix nargo
jonybur Sep 19, 2023
2f95c2a
Fixes
jonybur Sep 19, 2023
93cdb97
Fix copy
jonybur Sep 19, 2023
ac0c6c2
Add deploy
jonybur Sep 19, 2023
1e192fc
Enable double_verify_proof
jonybur Sep 19, 2023
0f680a2
Add mnemonic
jonybur Sep 19, 2023
ba247ed
Remove output
jonybur Sep 19, 2023
62b0725
Add test for solidity verifier
jonybur Sep 20, 2023
e21327a
Comment unused code
jonybur Sep 20, 2023
2e6940b
Log out address
jonybur Sep 20, 2023
ead7b4c
Run full test
jonybur Sep 20, 2023
5f186c5
Fix test run
jonybur Sep 20, 2023
25b8c1e
Fix integration test run
jonybur Sep 20, 2023
f3dc070
Merge branch 'master' of github.com:noir-lang/noir into jb-test-solid…
jonybur Sep 20, 2023
00d2461
Fix bad import
jonybur Sep 20, 2023
b3492c0
Fix lint
jonybur Sep 20, 2023
5a27616
Import contract correctly
jonybur Sep 20, 2023
b8888a1
Inject contract address
jonybur Sep 20, 2023
74d8008
Fix injection
jonybur Sep 20, 2023
ff9c292
Fix var name
jonybur Sep 20, 2023
d4244e9
Fix var name test side
jonybur Sep 20, 2023
040c991
Uncomment setup, comment double_verify
jonybur Sep 20, 2023
6ebf880
Remove walletPath
jonybur Sep 20, 2023
3b8fd45
Fix import
jonybur Sep 20, 2023
71d94c3
Rename function
jonybur Sep 20, 2023
825cb8d
ABI debug
jonybur Sep 20, 2023
530fa8f
Debug
jonybur Sep 20, 2023
fb40df1
Fix verify
jonybur Sep 20, 2023
99c0668
Slice for verify
jonybur Sep 20, 2023
3147b72
Pass no public input
jonybur Sep 20, 2023
25eaca8
Fix nargo
jonybur Sep 21, 2023
0c26bb8
Add nargo to PATH
jonybur Sep 21, 2023
ff9b7fe
Separate build-nargo step
jonybur Sep 21, 2023
837d93e
Test nargo_binary
jonybur Sep 21, 2023
96c5542
Use nargo by name
jonybur Sep 21, 2023
d0797c6
Enable double_verify
jonybur Sep 21, 2023
d368969
Separate build stages
jonybur Sep 21, 2023
1471d01
Fix dv verify
jonybur Sep 21, 2023
af5bd01
Fix lint
jonybur Sep 21, 2023
f9f7e32
Fix download path for noir_wasm
jonybur Sep 21, 2023
0ee3cc6
Output error
jonybur Sep 21, 2023
a977886
Revert split build stages
jonybur Sep 21, 2023
13dfbdc
Fix nargo
jonybur Sep 21, 2023
e2e18e6
Revert
jonybur Sep 21, 2023
097f601
Merge branch 'master' of github.com:noir-lang/noir into jb-test-solid…
jonybur Sep 21, 2023
b1f2286
Update yarn lock
jonybur Sep 21, 2023
524d89b
Name action
jonybur Sep 21, 2023
387091c
Fix
jonybur Sep 21, 2023
89abeb8
Prettier
jonybur Sep 21, 2023
1f039a9
Comment test case
jonybur Sep 21, 2023
70e1c60
Uncomment test
jonybur Sep 21, 2023
e14b15c
Bump bb.js
jonybur Sep 21, 2023
9ab73cb
fix: make proof non-recursive
TomAFrench Sep 21, 2023
00ffd6d
chore: remove wrapping layer around public inputs
TomAFrench Sep 21, 2023
bdcdb82
chore: update runners
TomAFrench Sep 21, 2023
46e242b
chore: add better error message for failing proof
TomAFrench Sep 21, 2023
8488121
chore: go back down to small runners
TomAFrench Sep 21, 2023
6e992ca
chore: turn off `double_verify_proof` testcase
TomAFrench Sep 21, 2023
2e2607c
chore: linter
TomAFrench Sep 21, 2023
813af9e
chore: linter fix
TomAFrench Sep 21, 2023
536e0f1
Update .github/workflows/test-integration.yml
TomAFrench Sep 22, 2023
1449f9f
Update .github/workflows/test-integration.yml
TomAFrench Sep 22, 2023
6a8c31e
Update .github/workflows/test-integration.yml
TomAFrench Sep 22, 2023
7b037a4
Update .github/workflows/test-integration.yml
TomAFrench Sep 22, 2023
386b138
Update .github/workflows/test-integration.yml
TomAFrench Sep 22, 2023
44d5c68
Update .github/workflows/test-integration.yml
TomAFrench Sep 22, 2023
24bdc76
Clean up workflow
jonybur Sep 25, 2023
1f83cff
Merge branch 'master' of github.com:noir-lang/noir into jb-test-solid…
jonybur Sep 25, 2023
dd59f71
Set up addresses in external file
jonybur Sep 25, 2023
f923b24
Merge branch 'master' of github.com:noir-lang/noir into jb-test-solid…
jonybur Sep 25, 2023
f689677
Log out address
jonybur Sep 25, 2023
4a6a82a
Lint test
jonybur Sep 25, 2023
bbcf391
Remove log
jonybur Sep 25, 2023
1c8e3db
Update .github/workflows/test-integration.yml
jonybur Sep 25, 2023
f9fa839
Replace dv with main circuit
jonybur Sep 25, 2023
bb5da59
Merge branch 'jb-test-solidity-verifier' of github.com:noir-lang/noir…
jonybur Sep 25, 2023
d01f8b0
Lint test
jonybur Sep 25, 2023
35e3084
Use json output for addresses
jonybur Sep 25, 2023
77b4fce
Lint fix
jonybur Sep 25, 2023
ff92891
Fix public inputs length
jonybur Sep 25, 2023
c4827fc
Output debug values
jonybur Sep 25, 2023
5fc1dc1
Fix path
jonybur Sep 25, 2023
72ace48
Disable second test
jonybur Sep 25, 2023
fe3eb79
Set up workflow running condition
jonybur Sep 25, 2023
33857ca
Readd second test
jonybur Sep 25, 2023
6199f43
Console log result
jonybur Sep 25, 2023
5c8f97b
Changes for merging
jonybur Sep 25, 2023
b7ac6f7
Change [Uint8Array] to Uint8Array[]
jonybur Sep 26, 2023
f1d3602
Modify publicsInput length
jonybur Sep 26, 2023
61f3f14
Update compile_prove_verify.test.ts
TomAFrench Sep 26, 2023
5400eb7
Update compile_prove_verify.test.ts
TomAFrench Sep 26, 2023
ec01bfc
Merge branch 'master' into jb-test-solidity-verifier
TomAFrench Sep 26, 2023
5ac123b
chore: fix linter
TomAFrench Sep 26, 2023
7ff4b94
Set up schedule and dispatch
jonybur Sep 26, 2023
10b6ed1
chore: install JS dependencies on github runner
TomAFrench Sep 26, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
194 changes: 182 additions & 12 deletions .github/workflows/test-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,50 @@ on:
- ./compiler/integration-tests/**
schedule:
- cron: "0 2 * * *" # Run nightly at 2 AM UTC

jobs:
wasm-packages-build-test:
build-nargo:
runs-on: ubuntu-latest
strategy:
matrix:
target: [x86_64-unknown-linux-gnu]

steps:
- name: Checkout Noir repo
uses: actions/checkout@v4

- name: Setup toolchain
uses: dtolnay/[email protected]

- uses: Swatinem/rust-cache@v2
with:
key: ${{ matrix.target }}
cache-on-failure: true
save-if: ${{ github.event_name != 'merge_group' }}

- name: Build Nargo
run: cargo build --package nargo_cli --release

- name: Package artifacts
run: |
mkdir dist
cp ./target/release/nargo ./dist/nargo
7z a -ttar -so -an ./dist/* | 7z a -si ./nargo-x86_64-unknown-linux-gnu.tar.gz

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: nargo
path: ./dist/*
retention-days: 3

build-wasm:
runs-on: ubuntu-latest
env:
CACHED_PATH: /tmp/nix-cache

steps:
- name: Checkout noir sources
- name: Checkout sources
uses: actions/checkout@v4

- name: Setup Nix
Expand All @@ -28,38 +64,172 @@ jobs:
name: barretenberg
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"


- name: Build noir_wasm package
- name: Build wasm package
run: |
nix build -L .#noir_wasm
echo "UPLOAD_PATH=$(readlink -f ./result/noir_wasm)" >> $GITHUB_ENV
cp -r ./result/noir_wasm/nodejs ./compiler/wasm
cp -r ./result/noir_wasm/web ./compiler/wasm

- name: Upload `noir_wasm` artifact
- name: Export cache from nix store
if: ${{ steps.cache.outputs.cache-hit != 'true' && github.event_name != 'merge_group' }}
run: |
nix copy --to "file://${{ env.CACHED_PATH }}?compression=zstd&parallel-compression=true" .#noir-wasm-cargo-artifacts

- uses: actions/cache/save@v3
# Don't create cache entries for the merge queue.
if: ${{ steps.cache.outputs.cache-hit != 'true' && github.event_name != 'merge_group' }}
with:
path: ${{ env.CACHED_PATH }}
key: ${{ steps.cache.outputs.cache-primary-key }}

- name: Dereference symlink
run: echo "UPLOAD_PATH=$(readlink -f ./result/noir_wasm)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: noir_wasm
path: ${{ env.UPLOAD_PATH }}
retention-days: 3

- name: Build noirc_abi_wasm package
build-noirc:
runs-on: ubuntu-latest
env:
CACHED_PATH: /tmp/nix-cache

steps:
- name: Checkout sources
uses: actions/checkout@v3

- name: Setup Nix
uses: cachix/install-nix-action@v22
with:
nix_path: nixpkgs=channel:nixos-23.05
github_access_token: ${{ secrets.GITHUB_TOKEN }}

- uses: cachix/cachix-action@v12
with:
name: barretenberg
authToken: "${{ secrets.CACHIX_AUTH_TOKEN }}"

- name: Restore nix store cache
uses: actions/cache/restore@v3
id: cache
with:
path: ${{ env.CACHED_PATH }}
key: ${{ runner.os }}-flake-abi-wasm-${{ hashFiles('*.lock') }}

# Based on https://github.com/marigold-dev/deku/blob/b5016f0cf4bf6ac48db9111b70dd7fb49b969dfd/.github/workflows/build.yml#L26
- name: Copy cache into nix store
if: steps.cache.outputs.cache-hit == 'true'
# We don't check the signature because we're the one that created the cache
run: |
for narinfo in ${{ env.CACHED_PATH }}/*.narinfo; do
path=$(head -n 1 "$narinfo" | awk '{print $2}')
nix copy --no-check-sigs --from "file://${{ env.CACHED_PATH }}" "$path"
done

- name: Build noirc_abi_wasm
run: |
nix build -L .#noirc_abi_wasm
echo "UPLOAD_PATH=$(readlink -f ./result/noirc_abi_wasm)" >> $GITHUB_ENV
cp -r ./result/noirc_abi_wasm/nodejs ./tooling/noirc_abi_wasm
cp -r ./result/noirc_abi_wasm/web ./tooling/noirc_abi_wasm

- name: Upload `noirc_abi_wasm` artifact
- name: Export cache from nix store
if: ${{ steps.cache.outputs.cache-hit != 'true' && github.event_name != 'merge_group' }}
run: |
nix copy --to "file://${{ env.CACHED_PATH }}?compression=zstd&parallel-compression=true" .#noirc-abi-wasm-cargo-artifacts

- uses: actions/cache/save@v3
# Don't create cache entries for the merge queue.
if: ${{ steps.cache.outputs.cache-hit != 'true' && github.event_name != 'merge_group' }}
with:
path: ${{ env.CACHED_PATH }}
key: ${{ steps.cache.outputs.cache-primary-key }}

- name: Dereference symlink
run: echo "UPLOAD_PATH=$(readlink -f ./result/noirc_abi_wasm)" >> $GITHUB_ENV

- name: Upload artifact
uses: actions/upload-artifact@v3
with:
name: noirc_abi_wasm
path: ${{ env.UPLOAD_PATH }}
retention-days: 3
retention-days: 10

test-solidity-verifier:
runs-on: ubuntu-latest
needs: [build-wasm, build-nargo, build-noirc]
env:
CACHED_PATH: /tmp/nix-cache

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Download nargo binary
uses: actions/download-artifact@v3
with:
name: nargo
path: ./nargo

- name: Download wasm package artifact
uses: actions/download-artifact@v3
with:
name: noir_wasm
path: ./compiler/wasm

- name: Download noirc package artifact
uses: actions/download-artifact@v3
with:
name: noirc_abi_wasm
path: ./tooling/noirc_abi_wasm

- name: Set nargo on PATH
run: |
nargo_binary="${{ github.workspace }}/nargo/nargo"
chmod +x $nargo_binary
echo "$(dirname $nargo_binary)" >> $GITHUB_PATH
export PATH="$PATH:$(dirname $nargo_binary)"
nargo -V

- name: Install Yarn dependencies
uses: ./.github/actions/setup

- name: Install Foundry
uses: onbjerg/foundry-toolchain@v1
with:
version: nightly

- name: Run Anvil
run: |
anvil > /dev/null 2>&1 &
sleep 10

- name: Create new Foundry project
run: forge init --no-git --no-commit --force foundry-project

- name: Run codegen-verifier for 1_mul
working-directory: tooling/nargo_cli/tests/execution_success/1_mul
run: nargo codegen-verifier

- name: Run codegen-verifier for main
working-directory: compiler/integration-tests/test/circuits/main
run: nargo codegen-verifier

- name: Copy compiled contracts
run: |
cp tooling/nargo_cli/tests/execution_success/1_mul/contract/1_mul/plonk_vk.sol foundry-project/src/1_mul.sol
cp compiler/integration-tests/test/circuits/main/contract/main/plonk_vk.sol foundry-project/src/main.sol

- name: Forge build
working-directory: foundry-project
run: forge build

- name: Forge deploy
working-directory: foundry-project
run: |
forge create --rpc-url http://127.0.0.1:8545 --mnemonic "test test test test test test test test test test test junk" src/1_mul.sol:UltraVerifier --json > mul_output.json
forge create --rpc-url http://127.0.0.1:8545 --mnemonic "test test test test test test test test test test test junk" src/main.sol:UltraVerifier --json > main_output.json

- name: Setup `integration-tests`
run: |
yarn workspace @noir-lang/source-resolver build
Expand Down
3 changes: 2 additions & 1 deletion compiler/integration-tests/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,14 @@
"lint": "NODE_NO_WARNINGS=1 eslint . --ext .ts --ignore-path ./.eslintignore --max-warnings 0"
},
"dependencies": {
"@aztec/bb.js": "0.7.2",
"@aztec/bb.js": "^0.7.2",
"@noir-lang/noir_js": "workspace:*",
"@noir-lang/noir_wasm": "workspace:*",
"@noir-lang/source-resolver": "workspace:*",
"@web/dev-server-esbuild": "^0.3.6",
"@web/test-runner": "^0.15.3",
"@web/test-runner-webdriver": "^0.7.0",
"ethers": "^6.7.1",
"fflate": "^0.8.0",
"smol-toml": "^1.1.2",
"tslog": "^4.9.2"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,16 @@ import newCompiler, {
compile,
init_log_level as compilerLogLevel,
} from "@noir-lang/noir_wasm";
// eslint-disable-next-line @typescript-eslint/ban-ts-comment
//@ts-ignore
import { Barretenberg, RawBuffer, Crs } from "@aztec/bb.js";
import { acvm, abi } from "@noir-lang/noir_js";
import { Barretenberg, RawBuffer, Crs } from "@aztec/bb.js";
import { decompressSync as gunzip } from "fflate";

import { ethers } from "ethers";
import * as TOML from "smol-toml";

const mnemonic = "test test test test test test test test test test test junk";
const provider = new ethers.JsonRpcProvider("http://localhost:8545");
const walletMnemonic = ethers.Wallet.fromPhrase(mnemonic);
const wallet = walletMnemonic.connect(provider);
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
const logger = new Logger({ name: "test", minLevel: TEST_LOG_LEVEL });

const { default: initACVM, executeCircuit, compressWitness } = acvm;
Expand All @@ -40,9 +42,15 @@ const CIRCUIT_SIZE = 2 ** 19;
const test_cases = [
{
case: "tooling/nargo_cli/tests/execution_success/1_mul",
compiled: "foundry-project/out/1_mul.sol/UltraVerifier.json",
deployInformation: "foundry-project/mul_output.json",
publicInputsLength: 0,
},
{
case: "tooling/nargo_cli/tests/execution_success/double_verify_proof",
case: "compiler/integration-tests/test/circuits/main",
compiled: "foundry-project/out/main.sol/UltraVerifier.json",
deployInformation: "foundry-project/main_output.json",
publicInputsLength: 32,
},
];

Expand Down Expand Up @@ -71,9 +79,24 @@ test_cases.forEach((testInfo) => {
`${base_relative_path}/${test_case}/Prover.toml`,
import.meta.url,
);
const compiled_contract_url = new URL(
`${base_relative_path}/${testInfo.compiled}`,
import.meta.url,
);
const deploy_information_url = new URL(
`${base_relative_path}/${testInfo.deployInformation}`,
import.meta.url,
);

const noir_source = await getFile(noir_source_url);
const prover_toml = await getFile(prover_toml_url);
const compiled_contract = await getFile(compiled_contract_url);
const deploy_information = await getFile(deploy_information_url);

const { abi } = JSON.parse(compiled_contract);
const { deployedTo } = JSON.parse(deploy_information);

const contract = new ethers.Contract(deployedTo, abi, wallet);

expect(noir_source).to.be.a.string;

Expand Down Expand Up @@ -130,7 +153,7 @@ test_cases.forEach((testInfo) => {
const acirUint8Array = gunzip(compressedByteCode);
const witnessUint8Array = gunzip(compressedWitness);

const isRecursive = true;
const isRecursive = false;
const api = await Barretenberg.new(numberOfThreads);
await api.commonInitSlabAllocator(CIRCUIT_SIZE);

Expand Down Expand Up @@ -159,7 +182,23 @@ test_cases.forEach((testInfo) => {
isRecursive,
);

expect(verified).to.be.true;
expect(verified, "Proof fails verification in JS").to.be.true;

try {
let result;
if (testInfo.publicInputsLength === 0) {
result = await contract.verify(proof, []);
} else {
const publicInputs = proof.slice(0, testInfo.publicInputsLength);
const slicedProof = proof.slice(testInfo.publicInputsLength);
result = await contract.verify(slicedProof, [publicInputs]);
TomAFrench marked this conversation as resolved.
Show resolved Hide resolved
}

expect(result).to.be.true;
} catch (error) {
console.error("Error while submitting the proof:", error);
throw error;
}
} catch (e) {
expect(e, "Proving and Verifying").to.not.be.an("error");
throw e;
Expand Down
Loading
Loading