Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Dockerfile.espresso #156

Merged

Conversation

Sneh1999
Copy link

@Sneh1999 Sneh1999 commented Jul 19, 2024

Closes #145

#145

This PR:

Remove Dockerfile.espresso and make the necessary changes to Dockerfile.

Local build output:

~/Espresso/nitro-espresso-integration remove-dockerfile-espresso !1 ❯ docker build .                                                               14s
[+] Building 392.7s (117/117) FINISHED                                                                                            docker:desktop-linux
 => [internal] load build definition from Dockerfile                                                                                              0.0s
 => => transferring dockerfile: 16.82kB                                                                                                           0.0s
 => [internal] load metadata for docker.io/offchainlabs/nitro-node:v2.3.4-rc.5-b4cc111                                                            0.7s
 => [internal] load metadata for docker.io/library/node:18-bookworm-slim                                                                          0.7s
 => [internal] load metadata for docker.io/library/debian:bookworm-slim                                                                           0.7s
 => [internal] load metadata for docker.io/library/rust:1.75-slim-bookworm                                                                        0.7s
 => [internal] load metadata for docker.io/library/golang:1.21.10-bookworm                                                                        0.7s
 => [internal] load .dockerignore                                                                                                                 0.0s
 => => transferring context: 993B                                                                                                                 0.0s
 => [nitro-legacy 1/1] FROM docker.io/offchainlabs/nitro-node:v2.3.4-rc.5-b4cc111@sha256:02fd584267485c9a62652330e578acd5057030e2ebf7691b4c8289  20.5s
 => => resolve docker.io/offchainlabs/nitro-node:v2.3.4-rc.5-b4cc111@sha256:02fd584267485c9a62652330e578acd5057030e2ebf7691b4c82895dc3f1c7f8      0.0s
 => => sha256:22d97f6a5d13532e867231d23d92620a81874d51a456196be50154eeb32edc08 29.18MB / 29.18MB                                                  2.0s
 => => sha256:02fd584267485c9a62652330e578acd5057030e2ebf7691b4c82895dc3f1c7f8 772B / 772B                                                        0.0s
 => => sha256:6a42313ba639d81798168ace7ce07844ee87d30511d5d623f94dd0e3c3f5b9ab 5.01kB / 5.01kB                                                    0.0s
 => => sha256:e361cfd45aff1f87484550909bd17b882459733b42b8d8240902438392f17f3e 148B / 148B                                                        0.2s
 => => sha256:300dc3f7693428e5180bfd7c666bce6ecfcc1712338f0d48b3e5817a1a896245 3.06kB / 3.06kB                                                    0.0s
 => => sha256:43018c91e00491a0e6233814415264718f49b10a90eedb26e4ace3f7dff1d992 77.21MB / 77.21MB                                                  4.1s
 => => sha256:671037e74b5c93e50cb1106760eb342d7556fa38b01dc2d4ee341437f344b082 15.49MB / 15.49MB                                                  2.5s
 => => sha256:da286f42c50cda0b93dae2c315f81e4b146410b8169fda9b42f219871590a00d 38.14MB / 38.14MB                                                  4.6s
 => => extracting sha256:22d97f6a5d13532e867231d23d92620a81874d51a456196be50154eeb32edc08                                                         1.0s
 => => sha256:79835e5d388e260427245df22b52a56a7b7e91636d240b6ca4d7b510975d12de 6.67MB / 6.67MB                                                    3.5s
 => => extracting sha256:e361cfd45aff1f87484550909bd17b882459733b42b8d8240902438392f17f3e                                                         0.0s
 => => sha256:bf48a605f2583835236495f674215ac46344c82bb809077edcaef1521da8b4b1 70.16MB / 70.16MB                                                 13.3s
 => => extracting sha256:43018c91e00491a0e6233814415264718f49b10a90eedb26e4ace3f7dff1d992                                                         0.6s
 => => sha256:fd84bcfab0e7f41b29ec7fcabd7f34f941d179e9fc5e78147c4ae8b7250f9cb1 77.39MB / 77.39MB                                                 14.4s
 => => extracting sha256:671037e74b5c93e50cb1106760eb342d7556fa38b01dc2d4ee341437f344b082                                                         0.1s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 32B / 32B                                                          4.9s
 => => extracting sha256:da286f42c50cda0b93dae2c315f81e4b146410b8169fda9b42f219871590a00d                                                         0.3s
 => => sha256:8485ebb3883ce93b59074087f1214857afe3678aac9db7cd95f12a9945430f0c 48.24MB / 48.24MB                                                 11.6s
 => => extracting sha256:79835e5d388e260427245df22b52a56a7b7e91636d240b6ca4d7b510975d12de                                                         0.0s
 => => sha256:25b48507d0f834723dfdc49134517b76b341e2a96f291a8ba61385149f84cd89 48.78MB / 48.78MB                                                 14.2s
 => => extracting sha256:bf48a605f2583835236495f674215ac46344c82bb809077edcaef1521da8b4b1                                                         0.6s
 => => sha256:bf1d92a0ce1903cc14b266c3da55b01bccb0cdb0f4f205b5ee18da9d223129ca 40.57MB / 40.57MB                                                 17.3s
 => => sha256:d00841b4662cf72165b06598cf24dcbbce50c4d452fce1328c6979869af53eef 51.08MB / 51.08MB                                                 17.4s
 => => extracting sha256:fd84bcfab0e7f41b29ec7fcabd7f34f941d179e9fc5e78147c4ae8b7250f9cb1                                                         0.7s
 => => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                                                         0.0s
 => => extracting sha256:8485ebb3883ce93b59074087f1214857afe3678aac9db7cd95f12a9945430f0c                                                         0.7s
 => => extracting sha256:25b48507d0f834723dfdc49134517b76b341e2a96f291a8ba61385149f84cd89                                                         0.4s
 => => extracting sha256:bf1d92a0ce1903cc14b266c3da55b01bccb0cdb0f4f205b5ee18da9d223129ca                                                         0.7s
 => => extracting sha256:d00841b4662cf72165b06598cf24dcbbce50c4d452fce1328c6979869af53eef                                                         2.1s
 => [internal] load build context                                                                                                                 1.4s
 => => transferring context: 782.16kB                                                                                                             1.4s
 => [machine-versions 1/8] FROM docker.io/library/debian:bookworm-slim@sha256:f528891ab1aa484bf7233dbcc84f3c806c3e427571d75510a9d74bb5ec535b33    0.0s
 => [contracts-builder 1/9] FROM docker.io/library/node:18-bookworm-slim@sha256:8bc58ec22d5114a8cef0d2de719d0b88c11fa17b7a49ab6f18291ab98c5d4b7  17.1s
 => => resolve docker.io/library/node:18-bookworm-slim@sha256:8bc58ec22d5114a8cef0d2de719d0b88c11fa17b7a49ab6f18291ab98c5d4b70                    0.0s
 => => sha256:8bc58ec22d5114a8cef0d2de719d0b88c11fa17b7a49ab6f18291ab98c5d4b70 6.49kB / 6.49kB                                                    0.0s
 => => sha256:6f23261701721e7cfebc975c0d0de8f4d58400c227645577f92bced6a4083103 1.93kB / 1.93kB                                                    0.0s
 => => sha256:7d591e1233173d353bf00ed8a3a56daf68f34fa48203b0b3cf73a4474433eacc 6.88kB / 6.88kB                                                    0.0s
 => => sha256:503fd181343924f0ef1848277bc89363a955e321327f37ee49e7d46868a5e847 3.31kB / 3.31kB                                                   14.7s
 => => extracting sha256:503fd181343924f0ef1848277bc89363a955e321327f37ee49e7d46868a5e847                                                         0.0s
 => => sha256:6840eb01c4ca3319e7bcc5acd06f8a0754968b3d5676f0528fde5abc8a792fe2 38.22MB / 38.22MB                                                 16.3s
 => => sha256:082d1e72052dfa325d7ecabaa0266d3cdf84cb3a1790cef132503e11ed8eb597 1.71MB / 1.71MB                                                   16.4s
 => => extracting sha256:6840eb01c4ca3319e7bcc5acd06f8a0754968b3d5676f0528fde5abc8a792fe2                                                         0.7s
 => => sha256:418252e60c4d1d6def41e88346dc2b9b112a512875695a86163784bb174688f6 449B / 449B                                                       16.6s
 => => extracting sha256:082d1e72052dfa325d7ecabaa0266d3cdf84cb3a1790cef132503e11ed8eb597                                                         0.0s
 => => extracting sha256:418252e60c4d1d6def41e88346dc2b9b112a512875695a86163784bb174688f6                                                         0.0s
 => CACHED [nitro-node-slim 2/9] WORKDIR /home/user                                                                                               0.0s
 => [node-builder  1/18] FROM docker.io/library/golang:1.21.10-bookworm@sha256:b8ff8c869417952d84dc521d821cc8cb55f6c28469e60133fe8aea5ee795267e  55.5s
 => => resolve docker.io/library/golang:1.21.10-bookworm@sha256:b8ff8c869417952d84dc521d821cc8cb55f6c28469e60133fe8aea5ee795267e                  0.0s
 => => sha256:e946bfb614dc9f4a658e98b7795c5a888bc5c6c68dd9d633d9c07e0920199b53 2.84kB / 2.84kB                                                    0.0s
 => => sha256:7a56982245e6e85476cd3867a9a21bda806c32370bdeacc8452b2039381c7a69 2.33kB / 2.33kB                                                    0.0s
 => => sha256:b8ff8c869417952d84dc521d821cc8cb55f6c28469e60133fe8aea5ee795267e 9.10kB / 9.10kB                                                    0.0s
 => => sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0B / 32B                                                         391.9s
 => => sha256:91e301773f03e9e0fabc5c177fe6bfe8daf14e992ab816f151692b814ddc462c 49.61MB / 49.61MB                                                 20.4s
 => => sha256:30ed4c12791345d3f20f66024e1f22275ce507868c508509b83dcf231b1c9adc 63.99MB / 63.99MB                                                 20.4s
 => => sha256:15856ca26414127b85cee6d10acbc4cee6eba9070f3f5a04b9cc72ce95abfa7f 23.59MB / 23.59MB                                                 21.0s
 => => extracting sha256:91e301773f03e9e0fabc5c177fe6bfe8daf14e992ab816f151692b814ddc462c                                                         1.5s
 => => sha256:3cf3dd101b450545902b16112a329f7ce70d182d1d6b1dfce883f830030e03c1 86.25MB / 86.25MB                                                 48.6s
 => => sha256:0c9a7eb7dd16a3cb705d643f2c1b3ccb0cce9120eef39b1b1844864195af6c32 64.11MB / 64.11MB                                                 51.7s
 => => sha256:2013edc6dc9779e195fb4e37475b4569b1226af3caac82ddaf934a996e44c7a6 126B / 126B                                                       21.2s
 => => extracting sha256:15856ca26414127b85cee6d10acbc4cee6eba9070f3f5a04b9cc72ce95abfa7f                                                         0.3s
 => => extracting sha256:30ed4c12791345d3f20f66024e1f22275ce507868c508509b83dcf231b1c9adc                                                         1.3s
 => => extracting sha256:3cf3dd101b450545902b16112a329f7ce70d182d1d6b1dfce883f830030e03c1                                                         2.2s
 => => extracting sha256:0c9a7eb7dd16a3cb705d643f2c1b3ccb0cce9120eef39b1b1844864195af6c32                                                         3.3s
 => => extracting sha256:2013edc6dc9779e195fb4e37475b4569b1226af3caac82ddaf934a996e44c7a6                                                         0.0s
 => => extracting sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1                                                         0.0s
 => CACHED [brotli-library-builder 2/5] WORKDIR /workspace                                                                                        0.0s
 => [prover-builder  1/29] FROM docker.io/library/rust:1.75-slim-bookworm@sha256:70c2a016184099262fd7cee46f3d35fec3568c45c62f87e37f7f665f766b1f  59.4s
 => => resolve docker.io/library/rust:1.75-slim-bookworm@sha256:70c2a016184099262fd7cee46f3d35fec3568c45c62f87e37f7f665f766b1f74                  0.0s
 => => sha256:70c2a016184099262fd7cee46f3d35fec3568c45c62f87e37f7f665f766b1f74 6.00kB / 6.00kB                                                    0.0s
 => => sha256:13289631332573595a1deefe5721e27c6420e60db78765442b02995853edb75b 1.31kB / 1.31kB                                                    0.0s
 => => sha256:25cba573cfbb166f22691550279dfe19ca020b77d423a4fbb074e8a3c8301cf3 2.67kB / 2.67kB                                                    0.0s
 => => sha256:25d3892798f8b99159e3c1136799bfed560027ce451b50d57d961f4f02577ff5 29.18MB / 29.18MB                                                 24.2s
 => => extracting sha256:25d3892798f8b99159e3c1136799bfed560027ce451b50d57d961f4f02577ff5                                                         0.7s
 => => sha256:d5ac7f77b4f7aeff496d1ad91be5278d50f005338271545ce57f62ea5eeedde7 313.58MB / 313.58MB                                               54.1s
 => => extracting sha256:d5ac7f77b4f7aeff496d1ad91be5278d50f005338271545ce57f62ea5eeedde7                                                         5.2s
 => [brotli-wasm-builder 3/6] RUN apt-get update &&     apt-get install -y cmake make git lbzip2 python3 xz-utils &&     git clone https://gith  47.2s
 => CACHED [machine-versions 2/8] RUN apt-get update && apt-get install -y unzip wget curl                                                        0.0s
 => CACHED [machine-versions 3/8] WORKDIR /workspace/machines                                                                                     0.0s
 => CACHED [machine-versions 4/8] COPY ./scripts/download-machine.sh .                                                                            0.0s
 => CACHED [machine-versions 5/8] RUN ./download-machine.sh consensus-v30 0xb0de9cb89e4d944ae6023a3b62276e54804c242fd8c4c2d8e6cc4450f5fa8b1b &&   0.0s
 => CACHED [machine-versions 6/8] RUN ./download-machine.sh consensus-v31 0x260f5fa5c3176a856893642e149cf128b5a8de9f828afec8d11184415dd8dc69      0.0s
 => CACHED [machine-versions 7/8] COPY ./scripts/download-machine-espresso.sh .                                                                   0.0s
 => [machine-versions 8/8] RUN ./download-machine-espresso.sh 20240711 0x5409e2c3aab6db3d6cea9603646d6cdf2ac6bc1b39325939ba81a498f88f334f         4.7s
 => CACHED [brotli-library-builder 3/5] COPY scripts/build-brotli.sh scripts/                                                                     0.0s
 => CACHED [brotli-library-builder 4/5] COPY brotli brotli                                                                                        0.0s
 => [brotli-library-builder 5/5] RUN apt-get update &&     apt-get install -y cmake make gcc git &&     ./scripts/build-brotli.sh -l -t /worksp  22.5s
 => [contracts-builder 2/9] RUN apt-get update &&     apt-get install -y git python3 make g++ curl                                               14.6s
 => [brotli-library-export 1/1] COPY --from=brotli-library-builder /workspace/install/ /                                                          0.0s
 => [contracts-builder 3/9] RUN curl -L https://foundry.paradigm.xyz | bash && . ~/.bashrc && ~/.foundry/bin/foundryup                            5.9s
 => [contracts-builder 4/9] WORKDIR /workspace                                                                                                    0.0s
 => [contracts-builder 5/9] COPY contracts/package.json contracts/yarn.lock contracts/                                                            0.0s
 => [contracts-builder 6/9] RUN cd contracts && yarn install                                                                                     34.6s
 => [brotli-wasm-builder 4/6] COPY scripts/build-brotli.sh scripts/                                                                               0.0s
 => [brotli-wasm-builder 5/6] COPY brotli brotli                                                                                                  0.1s
 => [brotli-wasm-builder 6/6] RUN cd emsdk && . ./emsdk_env.sh && cd .. && ./scripts/build-brotli.sh -w -t /workspace/install/                   17.7s
 => [node-builder  2/18] WORKDIR /workspace                                                                                                       0.1s
 => [node-builder  3/18] RUN export DEBIAN_FRONTEND=noninteractive &&     apt-get update &&     apt-get install -y wabt                           3.6s
 => [node-builder  4/18] COPY go.mod go.sum ./                                                                                                    0.0s
 => [node-builder  5/18] COPY go-ethereum/go.mod go-ethereum/go.sum go-ethereum/                                                                  0.0s
 => [node-builder  6/18] COPY fastcache/go.mod fastcache/go.sum fastcache/                                                                        0.0s
 => [node-builder  7/18] RUN go mod download                                                                                                     18.0s
 => [prover-builder  2/29] WORKDIR /workspace                                                                                                     0.0s
 => [prover-builder  3/29] RUN export DEBIAN_FRONTEND=noninteractive &&     apt-get update &&     apt-get install -y make wget gpg software-pro  17.0s
 => [prover-header-builder  3/19] RUN export DEBIAN_FRONTEND=noninteractive &&     apt-get update &&     apt-get install -y make clang wabt &&   56.6s
 => CACHED [brotli-wasm-export 1/1] COPY --from=brotli-wasm-builder /workspace/install/ /                                                         0.0s
 => [contracts-builder 7/9] COPY contracts contracts/                                                                                             0.1s
 => [contracts-builder 8/9] COPY Makefile .                                                                                                       0.0s
 => [contracts-builder 9/9] RUN . ~/.bashrc && NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-solidity                                               78.2s
 => [prover-builder  4/29] RUN wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key | apt-key add - &&     add-apt-repository 'deb http://apt.l  15.6s
 => [node-builder  8/18] COPY . ./                                                                                                                4.0s
 => [prover-builder  5/29] COPY --from=brotli-library-export / target/                                                                            0.0s
 => [prover-builder  6/29] COPY arbitrator/Cargo.* arbitrator/                                                                                    0.0s
 => [prover-builder  7/29] COPY arbitrator/arbutil arbitrator/arbutil                                                                             0.0s
 => [prover-builder  8/29] COPY arbitrator/brotli arbitrator/brotli                                                                               0.1s
 => [prover-builder  9/29] COPY arbitrator/caller-env arbitrator/caller-env                                                                       0.1s
 => [prover-builder 10/29] COPY arbitrator/prover/Cargo.toml arbitrator/prover/                                                                   0.1s
 => [prover-builder 11/29] COPY arbitrator/jit/Cargo.toml arbitrator/jit/                                                                         0.1s
 => [prover-builder 12/29] COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper                                               0.1s
 => [prover-builder 13/29] COPY arbitrator/stylus/Cargo.toml arbitrator/stylus/                                                                   0.1s
 => [prover-builder 14/29] COPY arbitrator/tools/wasmer arbitrator/tools/wasmer                                                                   0.7s
 => [prover-builder 15/29] COPY arbitrator/wasm-libraries/user-host-trait/Cargo.toml arbitrator/wasm-libraries/user-host-trait/Cargo.toml         0.0s
 => [prover-builder 16/29] RUN bash -c 'mkdir arbitrator/{prover,jit,stylus}/src arbitrator/wasm-libraries/user-host-trait/src'                   0.2s
 => [prover-builder 17/29] RUN echo "fn test() {}" > arbitrator/jit/src/lib.rs &&     echo "fn test() {}" > arbitrator/prover/src/lib.rs &&     103.3s
 => [prover-header-builder  4/19] COPY arbitrator/Cargo.* arbitrator/                                                                             0.4s
 => [prover-header-builder  5/19] COPY ./Makefile ./                                                                                              0.2s
 => [prover-header-builder  6/19] COPY arbitrator/arbutil arbitrator/arbutil                                                                      0.1s
 => [prover-header-builder  7/19] COPY arbitrator/brotli arbitrator/brotli                                                                        0.2s
 => [prover-header-builder  8/19] COPY arbitrator/caller-env arbitrator/caller-env                                                                0.1s
 => [prover-header-builder  9/19] COPY arbitrator/prover arbitrator/prover                                                                        0.3s
 => [prover-header-builder 10/19] COPY arbitrator/wasm-libraries arbitrator/wasm-libraries                                                        0.4s
 => [prover-header-builder 11/19] COPY arbitrator/jit arbitrator/jit                                                                              0.1s
 => [prover-header-builder 12/19] COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper                                        0.1s
 => [prover-header-builder 13/19] COPY arbitrator/stylus arbitrator/stylus                                                                        0.1s
 => [prover-header-builder 14/19] COPY arbitrator/tools/wasmer arbitrator/tools/wasmer                                                            1.6s
 => [prover-header-builder 15/19] COPY --from=brotli-wasm-export / target/                                                                        0.1s
 => [prover-header-builder 16/19] COPY scripts/build-brotli.sh scripts/                                                                           0.1s
 => [prover-header-builder 17/19] COPY brotli brotli                                                                                              0.4s
 => [prover-header-builder 18/19] RUN apt-get update && apt-get install -y cmake                                                                  6.7s
 => [prover-header-builder 19/19] RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-prover-header                                                   28.6s
 => [prover-header-export 1/1] COPY --from=prover-header-builder /workspace/target/ /                                                             0.1s
 => [node-builder  9/18] COPY --from=contracts-builder workspace/contracts/build/ contracts/build/                                                0.6s
 => [node-builder 10/18] COPY --from=contracts-builder workspace/contracts/out/ contracts/out/                                                    0.1s
 => [node-builder 11/18] COPY --from=contracts-builder workspace/contracts/node_modules/@offchainlabs/upgrade-executor/build/contracts/src/Upgra  0.1s
 => [node-builder 12/18] COPY --from=contracts-builder workspace/.make/ .make/                                                                    0.3s
 => [node-builder 13/18] COPY --from=prover-header-export / target/                                                                               0.1s
 => [node-builder 14/18] COPY --from=brotli-library-export / target/                                                                              0.2s
 => [prover-builder 18/29] COPY ./Makefile ./                                                                                                     0.0s
 => [prover-builder 19/29] COPY arbitrator/prover arbitrator/prover                                                                               0.0s
 => [prover-builder 20/29] COPY arbitrator/wasm-libraries arbitrator/wasm-libraries                                                               0.1s
 => [prover-builder 21/29] COPY arbitrator/jit arbitrator/jit                                                                                     0.0s
 => [prover-builder 22/29] COPY arbitrator/stylus arbitrator/stylus                                                                               0.0s
 => [prover-builder 23/29] COPY --from=brotli-wasm-export / target/                                                                               0.0s
 => [prover-builder 24/29] COPY scripts/build-brotli.sh scripts/                                                                                  0.0s
 => [prover-builder 25/29] COPY brotli brotli                                                                                                     0.2s
 => [prover-builder 26/29] RUN touch -a -m arbitrator/prover/src/lib.rs                                                                           0.1s
 => [prover-builder 27/29] RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-prover-lib                                                             59.3s
 => [prover-builder 28/29] RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-prover-bin                                                             13.9s
 => [prover-builder 29/29] RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build-jit                                                                    40.4s
 => [prover-export 1/1] COPY --from=prover-builder /workspace/target/ /                                                                           0.9s
 => [node-builder 15/18] COPY --from=prover-export / target/                                                                                      1.0s
 => [node-builder 16/18] RUN mkdir -p target/bin                                                                                                  0.1s
 => [node-builder 17/18] COPY .nitro-tag.txt /nitro-tag.txt                                                                                       0.0s
 => [node-builder 18/18] RUN NITRO_BUILD_IGNORE_TIMESTAMPS=1 make build                                                                          65.2s
 => [nitro-node-slim 3/9] COPY --from=node-builder /workspace/target/bin/nitro /usr/local/bin/                                                    0.1s
 => [nitro-node-slim 4/9] COPY --from=node-builder /workspace/target/bin/relay /usr/local/bin/                                                    0.0s
 => [nitro-node-slim 5/9] COPY --from=node-builder /workspace/target/bin/nitro-val /usr/local/bin/                                                0.1s
 => [nitro-node-slim 6/9] COPY --from=node-builder /workspace/target/bin/seq-coordinator-manager /usr/local/bin/                                  0.0s
 => [nitro-node-slim 7/9] COPY --from=machine-versions /workspace/machines /home/user/target/machines                                             0.1s
 => [nitro-node-slim 8/9] RUN export DEBIAN_FRONTEND=noninteractive &&     apt-get update &&     apt-get install -y     ca-certificates     wabt  4.3s
 => [nitro-node-slim 9/9] WORKDIR /home/user/                                                                                                     0.0s
 => [nitro-node 1/6] COPY --from=prover-export /bin/jit                        /usr/local/bin/                                                    0.4s
 => [nitro-node 2/6] COPY --from=node-builder  /workspace/target/bin/daserver  /usr/local/bin/                                                    0.0s
 => [nitro-node 3/6] COPY --from=node-builder  /workspace/target/bin/datool    /usr/local/bin/                                                    0.0s
 => [nitro-node 4/6] COPY --from=nitro-legacy /home/user/target/machines /home/user/nitro-legacy/machines                                         0.2s
 => [nitro-node 5/6] RUN rm -rf /workspace/target/legacy-machines/latest                                                                          0.1s
 => [nitro-node 6/6] RUN export DEBIAN_FRONTEND=noninteractive &&     apt-get update &&     apt-get install -y     curl procps jq rsync     node  6.1s
 => exporting to image                                                                                                                            0.4s
 => => exporting layers                                                                                                                           0.4s
 => => writing image sha256:87363635663e8022c0bcfd15c9151f631bcd4082801b929b245b285ff1d7a7e7

This PR does not:

Key places to review:

@ImJeremyHe
Copy link
Member

COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper

Our main changes on the dockerfile is adding a WASM library named espresso-crypto-library. This WASM library is for validating if txes are all from hotshot or not.
So I believe the current changes on this PR are not sufficient.

To test the Dockerfile, you can run the Espresso Docker build CI manually and paste the result/link in the PR description. (But you are working on your fork repo, it would be a bit problematic.). Or you can build the docker image locally.

Dockerfile Show resolved Hide resolved
Copy link

@nomaxg nomaxg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with adding existing binaries, besides that LGTM

@@ -139,7 +136,6 @@ COPY arbitrator/caller-env arbitrator/caller-env
COPY arbitrator/prover/Cargo.toml arbitrator/prover/
COPY arbitrator/jit/Cargo.toml arbitrator/jit/
COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
COPY config/vid_srs.json config/vid_srs.json
Copy link
Author

@Sneh1999 Sneh1999 Jul 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed this line because this file doesnt exist anymore in our repo

@@ -125,8 +123,7 @@ FROM rust:1.75-slim-bookworm AS prover-builder
WORKDIR /workspace
RUN export DEBIAN_FRONTEND=noninteractive && \
apt-get update && \
apt-get install -y make wget gpg software-properties-common zlib1g-dev libstdc++-12-dev wabt \
clang llvm-dev libclang-common-14-dev libpolly-14-dev
apt-get install -y make wget gpg software-properties-common zlib1g-dev libstdc++-12-dev wabt
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updating this to match upstream here

@Sneh1999 Sneh1999 requested a review from sveitser July 22, 2024 14:23
@sveitser
Copy link

I see this diff now compared to upstream which looks correct to me for git diff v3.1.0 -- Dockerfile

diff --git a/Dockerfile b/Dockerfile
index 69e688679..605629fe0 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -48,6 +48,7 @@ RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --de
 COPY ./Makefile ./
 COPY arbitrator/Cargo.* arbitrator/
 COPY arbitrator/arbutil arbitrator/arbutil
+COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
 COPY arbitrator/brotli arbitrator/brotli
 COPY arbitrator/caller-env arbitrator/caller-env
 COPY arbitrator/prover arbitrator/prover
@@ -69,6 +70,7 @@ COPY ./Makefile ./go.mod ./go.sum ./
 COPY ./arbcompress ./arbcompress
 COPY ./arbos ./arbos
 COPY ./arbstate ./arbstate
+COPY ./espressocrypto ./espressocrypto
 COPY ./arbutil ./arbutil
 COPY ./gethhook ./gethhook
 COPY ./blsSignatures ./blsSignatures
@@ -105,6 +107,7 @@ COPY arbitrator/caller-env arbitrator/caller-env
 COPY arbitrator/prover arbitrator/prover
 COPY arbitrator/wasm-libraries arbitrator/wasm-libraries
 COPY arbitrator/jit arbitrator/jit
+COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
 COPY arbitrator/stylus arbitrator/stylus
 COPY arbitrator/tools/wasmer arbitrator/tools/wasmer
 COPY --from=brotli-wasm-export / target/
@@ -132,6 +135,7 @@ COPY arbitrator/brotli arbitrator/brotli
 COPY arbitrator/caller-env arbitrator/caller-env
 COPY arbitrator/prover/Cargo.toml arbitrator/prover/
 COPY arbitrator/jit/Cargo.toml arbitrator/jit/
+COPY arbitrator/espresso-crypto-helper arbitrator/espresso-crypto-helper
 COPY arbitrator/stylus/Cargo.toml arbitrator/stylus/
 COPY arbitrator/tools/wasmer arbitrator/tools/wasmer
 COPY arbitrator/wasm-libraries/user-host-trait/Cargo.toml arbitrator/wasm-libraries/user-host-trait/Cargo.toml
@@ -207,6 +211,15 @@ COPY ./scripts/download-machine.sh .
 RUN ./download-machine.sh consensus-v30 0xb0de9cb89e4d944ae6023a3b62276e54804c242fd8c4c2d8e6cc4450f5fa8b1b && true
 RUN ./download-machine.sh consensus-v31 0x260f5fa5c3176a856893642e149cf128b5a8de9f828afec8d11184415dd8dc69

+#Download Espresso WASM machine
+COPY ./scripts/download-machine-espresso.sh .
+#
+# To use a new wasm machine specify release tag from the page
+# https://github.com/EspressoSystems/nitro-espresso-integration/releases and the
+# corresponding module-root.
+#
+RUN ./download-machine-espresso.sh 20240711 0x5409e2c3aab6db3d6cea9603646d6cdf2ac6bc1b39325939ba81a498f88f334f
+
 FROM golang:1.21.10-bookworm AS node-builder
 WORKDIR /workspace
 ARG version=""

@sveitser
Copy link

sveitser commented Jul 22, 2024

Not sure how to trigger the CI manually for the docker build. I think we can't do it because the branch is on a fork.

https://github.com/EspressoSystems/nitro-espresso-integration/actions/workflows/espresso-docker.yml

Maybe @Sneh1999 can trigger it directly on her fork?

https://github.com/sneh1999/nitro-espresso-integration/actions/workflows/espresso-docker.yml

We can also just merge and fix it afterwards if needed. It's broken already anyway.

Edit: Running from the fork probably won't work without approval (and I'm not sure who has to approve this and where). because we're using our private runners for this workflow. So how about we merge, and if the docker build fails we make a new PR from this repo such that we can trigger workflow runs.

@Sneh1999
Copy link
Author

Not sure how to trigger the CI manually for the docker build. I think we can't do it because the branch is on a fork.

https://github.com/EspressoSystems/nitro-espresso-integration/actions/workflows/espresso-docker.yml

Maybe @Sneh1999 can trigger it directly on her fork?

https://github.com/sneh1999/nitro-espresso-integration/actions/workflows/espresso-docker.yml

We can also just merge and fix it afterwards if needed. It's broken already anyway.

Edit: Running from the fork probably won't work without approval (and I'm not sure who has to approve this and where). because we're using our private runners for this workflow. So how about we merge, and if the docker build fails we make a new PR from this repo such that we can trigger workflow runs.

Sounds good, I will merge once all the CI checks pass

@Sneh1999 Sneh1999 merged commit 26300e1 into EspressoSystems:integration Jul 22, 2024
8 checks passed
@Sneh1999 Sneh1999 deleted the remove-dockerfile-espresso branch July 22, 2024 19:19
Sneh1999 added a commit that referenced this pull request Nov 20, 2024
* add some mock merkle logic

* use mock data

* more mock validation code

* Increase the timeout limit for the e2e validation check

* Add SwitchSequencer

* Fix the switch function

* Create SwitchSequencer

* Update the espresso-sequencer-go

* Use light client as the trigger

* Add the switch test

* Make the linter happy

* Add the switch test to the e2e ci

* Increase the limit

* Clean the log with the log helper

* more mock code

* Fix the e2e test

* serialize proof variables correctly

* update espresso-go version

* wasm compatibility

* remove stale file, lint fix

* update to recent espresso go version

* refactor block merkle verification

* work on E2E

* progress on merkle tree verification

* add test vectors, fetch proofs at the correct indicies

* cleanup, remove outfiles, fetch merkle proof at the correct justified height

* add logic to compare circuit field

* check that circuit representation equals u256 representation in the test

* send byte slice to wasm instead of byte array which lives on the stack

* cleanup, remove debug statements, add documentation

* remove error logs

* fix build

* lint fix

* progress on e2e tests, comment out vid stuff for now because of wasm io issues

* update espresso-sequencer-go tag, lint ci fix

* cleanup, add issues for unresolved problems

* remove block that spams transactions

* comment out unused txns

* add test for advz equivalence and use a smaller srs

* Allow dead code in validation

* Specify foundry version

* Add random_get stub in wasi_snapshot_preview1

* Add stubs required in wasi_snapshot_preview1

* Fix `make lint`

* Enable prover service to talk to L1 on host

* Add builder timeout

This allows the sequencer to make progress past block 1.

These env vars have been changed in espresso-sequencer.

* Add missing env var, more logging in e2e test

* Revert "Fix `make lint`"

This reverts commit 8be940c.

* Fix go linting

* Parse espresso messages

* Add timeout limit

* Fix the e2e test

* Add timeout limit

* Run the e2e test and switch test in 1 thread

* Update the e2e workflow

* Increase time limit

* Specify the foundryup version in Dockerfile

* Add sock_accept stub

* Increase timeout limit

* Build successfully

* move ark-srs to dev dependencies

* remove debug statements

* remove out.txt

* Fix the machine test

* Fix CI

* Fix switch test

* Fix CI

* Increase CI time limie

* update lockfile

* remove commented out replaces

* newline

* Use the latest foundry

* RESET ME: debug

* RESET ME: not run the CI in a container

* RESET ME: remove the parallel

* Update script to run tests locally

- Run the same commands as on CI.
- Add ability to run `cargo +nightly` to dev shell.
- make clean: remove brotli build files.

* set -parallel=2

* Fix e2e test

* Skip some tests in CI

* Skip tests in stylus

* Remove parallelism restriction

* Impl escape hatch on validator side

* Fix tests

* Generate the osp of hotshot liveness

* Fix the fraud proof (#124)

* Update contracts submodule

* Upgrade go client

* Fix the staker challenge tests
---------

Co-authored-by: sveitser <[email protected]>

* Use espresso dev node

* attempt to fix dockerfile

* newline

* WIP

* WIP

* download latest espresso module root

* incorporate vid common data

* Fix the validation pipeline

* update go modules, e2e build step

* add cbrotli wasm

* ci fix attempt

* dont build all test deps for e2e test

* lint fixes

* Use the self-hosted runner and install cbindgen in CI

* Add time limit

* Remove parallel in building

* Fix the switch test

* Decrease the waiting hotshot blocks

* Use the ubuntu

* Specify the foundry version

* Enable -j

* update foundry version

* module root update

* Fix the staker test

* increase validation timeout for switch test

* pin espresso-sequencer-go to new release v0.0.20

* Commit go.sum from running go mod tidy

* sequencer info logs

* Add escape hatch test in e2e test

* Check the reactivate hotshot

* Update validator to handle preconditions and panics before entering the espresso STF logic.

* Improve readability of conditions in `handleEspressoPreConditions` and update the functions comments.

* Further improve readability

* Fix lint issue

* Update the nitro testnode

* Fix tests

* Update contracts

* Fix CI

* Changes to `handleEspressoPreConditions` in `espresso_validation.go` to fix incorrect logic, and give more granular control to where and how the generated panic is handled in the validators STF via returned closures.

* return nil when we need a no-op panic handler

* Fix lint errors

* Update contracts

* Fix the validation

* Remove the switch test

* Fix CI

* Remove postgres in espresso e2e test

* Codeowners: add Alysia, Zach, Sneh (#154)

For #146

* Remove Dockerfile.espresso (#156)

* Remove Dockerfile.espresso

* address review comments

* add new line

* Release 20240723 with 20240723-consensus

This should include all our changes for nitro v3.1.

- Check that the module root matches when downloading releases from
  github.

* Check all "not found" errors

Currently only the leveldb not found errors are handled. If a pebble not
found error is returned by the backend it is propagated, causing the
batchposter to fail to send transactions.

With this change all DB not found errors are handled the same way.

* Submit L2 Message to espresso network (#173)

* Submit L2 Message to espresso network

* add context timeout

* check for hotshot liveness and add a polling interval to see if transactions were included in an espresso block

* cleanup

* lint

* Draft the sovereign sequencer test

* Add espresso transaction queue

* Draft the espresso submission

* Set a new l2 message type for the sovereign sequencer transactions

* Attempt to write messages again

* Fix typos and add some comments

* Fix typo

* add header to jst and submit transaction to espresso

* Update go client for development

* Confirm the transaction inclusion in hotshot

* Fix the test

* Unblock the building blocks

* Update go client

* Fix the sovereign test

* add logs

* cleanup

* lint

* Updage go client

* not to fetch merkle proof if hotshot height is 0

* Cleanup

* Fix CI

* Add CI skip tests

* remove unwanted variable

* skipping some tests to make ci pass

* skipping some tests to make ci pass

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Persistent storage for sovereign builder (#180)

* Persistence storage for sovereign builder

* add unit test

* add unit test to ci and add mutex

* Optional header in jst

* initialize pending transactions if not present

* remove unwanted test

* resolve bugs

* remove nil pointer panic

* cleanup

* revert to strings.contains

* fix isErrNotFound

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Save digest artifacts into own directory per run (#182)

* Save digest artifacts into own directory per run

Currently it seems that we have too many digests in the folder and I
think this breaks the assembly of the multiarch docker image.

With this change only digests for the same run are saved in the same
directory.

* Fail docker image merge on wrong number of digests
* Clean digests dir

* Add dependabot rule to update nitro-testnode (#189)

* Fix build (#191)

* Add missing dependency
* Use renamed variable, fix typo

* dependabot: update contracts submodule (#193)

* Pin the golint version in CI

* Fix pebble panic (#192)

* Fix pebble panic

* lint

* Update some outdated help on flags (#203)

* Improve help texts for flags

* Update e2e tests to use Sovereign sequencer (#204)

* Update e2e tests to use Sovereign sequencer


Co-authored-by: ImJeremyHe <[email protected]>

* Remove the global state for HotShot height (#210)

* Remove the global state

* Update the submodule

* CI: use buildjet instead of self-hosted runner (#212)

* CI: use buildjet instead of self-hosted runner

* CI: use public runner / buildjet for docker build

This removes usage of our self-hosted github runners.

* CI: more disk space on public runner

* Check disk space at end of runs

* Try running CI tests from different directory

This may avoid failures due to IPC paths being too long

* Make workspace dir a symlink to /tmp/gha-work

* Fix redis URL

* Move redis tests to own workflow

* CI: fix rust and go cache (#217)

* rust-cache: add more target dirs
* Don't set custom TMPDIR

This seems to be the culprit for the extremely long path names that
are too long for IPC paths.

* modify arbosstate.go to allow for upgrade to arbOS version 35 (#216)

* modify arbosstate.go to allow for upgrade to arbOS version 35

* Fix lint

* Modify the chain config during the ArbOS upgrade

---------

Co-authored-by: sveitser <[email protected]>

* CI: make more space (#218)

* Store transaction pos in batch poster (#219)

* Move submit transaction pos from tx_streamer to batch poster

* Remove espresso sequencer (#214)

* Remove espresso sequencer

* lint

* remove enable espresso

* commit go-ethereum

* fix solgen/gen.go

* fix build

* revert .gitmodules

* revert go.mod

* add enable espresso to validation check

* re-add enable espresso to tests

---------

Co-authored-by: Mathis <[email protected]>

* Update espresso-sequencer-go with new LC ABI (#230)

* Update espresso-sequencer-go with new LC ABI

Instead of using the block height from the first snapshot after the
height in the justification it uses the latest height in the LC
contract. Currently not 100% convinced about correctness but also
believe it should work because any newer tree still contains the header
of interest.

* Undo LC ABI changes

We're not comfortable with the previous change because making a proof
w.r.t to the newest finalized state may cause issues if the batch poster
is lagging behind. This is currently not easy to test so we decided to
revert back to the old flow.

* Use tag of espresso-sequencer-go

* Stop setting the chain config during ArbOS upgrade in favor of setting it manually during migration process. (#234)

* Use the tag before marketplace update (#238)

* Sign payload for sovereign builder (#181)

* Sign payload for sovereign builder

* save signed payload to db

* change parsing logic to include signature

* add some comments

* skip tests

* address comments

* add signature payload test

* test working

* fix logging and comments

* move placement of log

* fix ci

* fix ci

* only submit transaction if jst nil

* add log to debug ci

* use the latest instead of main for espresso-dev-node

* pin to legacy lc version

* pin to before marketplace version

* remove log

* Full node with espresso finality (#237)

* wip

* wip

* Add espresso finality node test

* fix ci

* fix ci

* pin to before marketplace version

* fix tests

* add comments

* address comments

* address comments

* address comments

* cleanup

* fix test

* fmt

* rerun ci

* ci skip tests

* Update to sequencer.go to remove `EnableEspressoSovereign` flag, and instead sequence based on hotshot liveness at l1 block height.

* Update node.go to remove reference to `EnableEspressoSovereign`

* update Espresso tests to reflect removed config variables, and add escape hatch test to e2e test.

* Add wait for hotshot liveness in sovereign sequencer test to ensure sovereign sequencer is enabled.

* go fmt

* Add comment to explain changes needed in replay binary when panics are no longer an issue.

* Fix lint issues

* Modify construction of lightClientReader in sequencer.go to allow for nil `l1Reader`

* Update test configs to align with changes.

* Update comments according to feedback.

* Bump nitro-testnode from `03095e0` to `884a63c` (#240)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `03095e0` to `884a63c`.
- [Commits](EspressoSystems/nitro-testnode@03095e0...884a63c)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mathis <[email protected]>

* Revert removal of `EnableEspressoSovereign`

* Add additional check to ensure that if EnableEspressoSovereign is on we can't construct a sequencer with a nil lightClientReader

* Add `EnableEspressoSovereign` config flag to test configs.

* Bump contracts from `d226e9d` to `2ae1bbd` (#244)

Bumps [contracts](https://github.com/EspressoSystems/nitro-contracts) from `d226e9d` to `2ae1bbd`.
- [Release notes](https://github.com/EspressoSystems/nitro-contracts/releases)
- [Commits](EspressoSystems/nitro-contracts@d226e9d...2ae1bbd)

---
updated-dependencies:
- dependency-name: contracts
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Pin rust to 1.81.0

See #257

* Set cargo config for espresso_crypto wasm build

This was recently added for the other wasm compilation steps.

* Pin rust in E2E test

* CI: remove deprecated rust toolchain action

* Fix if statement

* Regenerate data-poster external-signer test certs

(cherry picked from commit 8f961b0)

* Use ArbOS chain config as the cannonial chain config. (#252)

* Add checks to sequencer.go and transaction_streamer.go to utilize the chain config switch

* Update interface.go to export interface for `GetArbOSConfigAtHeight()`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionEngine`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionNode`

* Update `sequencer.go` and `transaction_streamer.go` to read from the `ArbOS` `chainConfig` as the cannonical `chainConfig` rather than the `chainConfig` held by the `Blockchain` struct.

* Add ArbOS chain config check in batch poster to stop adding espresso messages to the transaction streamers DB if EnableEspresso is false.

* Add missing error check in sequencer.go and nil check on arbos config in sequencer.go and batch_poster.go.

* Remove ineffectual assignment in sequencer.go

* Add test for new GetArbOSConfigAtHeight() method in ExecutionNode

This commit adds an integration test for the GetArbOSConfigAtHeight()
method on the Execution node that grabs the initial config, and
validates it, then updates the config and ensures that the new method
will fetch the updated config.

* restore nitro-testnode to proper commit

* Fix trie node node found error during recording

I'm not exactly clear why we need to do this or if it's indeed the right
approach to solving the issue however it does seem to make it possible
to validate again.

Without it we get errors like

    INFO [10-30|12:24:29.127] DelayedSequencer: Sequenced              msgnum=1 startpos=1
    INFO [10-30|12:24:30.143] Reading message result remotely.         count=3
    WARN [10-30|12:24:32.143] prepareblocks failed to get state for block hdrNum=0 err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found"
    ERROR[10-30|12:24:32.143] Error while recording                    err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found" status=2

* Fix the OSP test

- Enable 4844 blobs to avoid contract deployment error
- Fix address of OSP entry

* Disable batch poster in Espresso finality node

We don't need this type of node to have a batch poster. Enabling it now
causes the test to fail because there's a check to ensure that redis is
used for the batch poster on the second node.

* Re-allow setting L1 stack config for tests

After the sync the code would always use the default values.

Don't think this is very clean but I couldn't think of something better
yet.

* CI: increase parallelism

* Fix linting

- Run `golang-lint run --fix`
- Check error after `waitForConfigUpdate`

* Move finality node test to espresso tests

The race test detected a data race in

https://github.com/EspressoSystems/nitro-espresso-integration/actions/runs/11705919237/job/32601803846?pr=254

* Update contracts submodule

* Fix typo

* Add wabt and cmake in wasm shell

* Skip tests

* Add tags in ci

* Add ci skip tests

* Add ci skip tests

* Remove rerun-fails

This flag in gotestsum causes some bugs which multiple the test time.

gotestyourself/gotestsum#417
gotestyourself/gotestsum#423

* Revert "Remove rerun-fails"

This reverts commit 6bef1c3.

* CI: simplify

- Add scripts to debug more easily locally.
- Skip tests by substring match instead of exact match.
- Don't run coverage on CI, we never look at it.
- Move all Espresso tests to Espresso CI job.

* Fix build with nix on macos (#296)

* CI: install latest gotestsum

This enables the --hide-summary argument

* Update submodule forks with merge commits

* Bump nitro-testnode from `07a25f1` to `77d1447` (#297)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `07a25f1` to `77d1447`.
- [Commits](EspressoSystems/nitro-testnode@07a25f1...77d1447)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add FFI bindings for Espresso validation (#294)

* Example call to espresso-crypto function stub in batch poster.

This commit includes a log line and example call to the espresso-crypto
function stubs in batch_poster.go

* Result of `make fmt`

Run `make fmt` to clean up formatting not apparent in editor

* Make current lib.rs c abi compatible

* result of make fmt

* Add C ABI ffi between rust crypto lib and espressocrypto go module.

This commit adds FFI support between exsisting Rust crypto helper code
and the exsisting espressocrypto go module in native.go These functions
will now serve as utility functions to use in the integration that will
handle ffi to the Rust code.

This commit also moves the espresso-crypto-helper code to the
espressocrypto directory as we no longer need it for the arbitrator.

* Build native.go successfully

* Update the espresso-crypto and add test for merkle proof verification

* update flake.nix to new go version and fix errors

* Current state of work

The current state of my work revolves around editing the Makefile to
include the new espressocrypto ffi functions in the build process.
Additionally, this commit restores some espressocrypto code to the
arbitrator in an attempt to get a docker image built via the test node.
I felt as though this would be easier for this PR as opposed to ripping
out all of the code from the arbitrator.

* Update the flake.lock

* Add the namespace proof test

* Current state of build system changes

This commit represents the current state of the build system after
testing today.
Currently things compile locally, but not when building an image from
the docker file. There are errors related to duplicate function
definitions from what seem to be the FFI lib that stylus exposes.

* Build nitro docker image sucessfully

- Modify makefile to have a file target. Since the build artifact of our
  static library is a file we should use this as the make target
  (instead of only having PHONY targets).

- Use rust 1.81 to build all shared rust libraries in docker.

Our crate requires a minimum version of 1.81. However the other
libraries like libstylus.a are currently compiled with 1.75 and this
seems to cause issues with duplicate symbol definitions during linking.

For example:

    8.718 go build -ldflags="-extldflags '-ldl' -X github.com/offchainlabs/nitro/cmd/util/confighelpers.version= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.datetime= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.modified=" -o target/bin/nitro "/workspace/cmd/nitro"
    42.74 # github.com/offchainlabs/nitro/cmd/nitro
    42.74 /usr/local/go/pkg/tool/linux_amd64/link: running clang failed: exit status 1
    42.74 /usr/bin/ld: /workspace/espressocrypto/lib/espresso-crypto-helper/target/release/libespresso_crypto_helper.a(std-1c4b19562077c20d.std.1f2242ed6435445e-cgu.0.rcgu.o): in function `__rdl_alloc':
    42.74 /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/alloc.rs:397: multiple definition of `__rdl_alloc'; /workspace/execution/gethexec/../../target/lib/libstylus.a(std-90f6ddbf82de36ec.std.409886f6357001f0-cgu.0.rcgu.o):/rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:389: first defined here

Bump the rust version of the prover-header-builder to 1.81 and compile
the espresso crypto library in the same container.

* Fix LDFLAGS, remove unnecessary CFLAGS

* Fix some merge errors

- We need rust 1.81 to compile the espresso crypto lib
- Re-add makefile rules

* Fix merge errors

* Remove FFI call in batch poster to expidite merge.

* Update bf submodule in arbitrator to match upstream version (This was set to a different commit during my erroneous merge)

* Remove extra change in espressocrypto_wasm.go

* Fix error with temporary espressocryptowasm directory preventing CI from running

* fix error of missing function definitions in espressocryptowasm directory to allow CI to run

* Re-add nosec comment about gosec error that will be removed soon

* result of make fmt

---------

Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: ImJeremyHe <[email protected]>

* Update the go client (#256)

Still using the legacy dev node

* ci: fix `git ls-remote` error in docker build (#304)

* Use path context instead of git context

With the default git context the repository is re-cloned at the start of
the docker build and the .dockerignore file is ignored.

> Be careful because any file mutation in the steps that precede the
> build step will be ignored, including processing of the .dockerignore
> file since the context is based on the Git reference.

https://github.com/docker/build-push-action?tab=readme-ov-file#git-context

This leads to some `.git` worktree files showing up inside the
dockerbuild with triggers a bug in yarn.

See yarnpkg/yarn#7537 (comment)
for more details about the yarn bug.

* Use the latest espresso dev node (#303)

* Use the latest espresso dev node

- Fix an issue that may cause the validation reorg
- Waiting for L1 to be ready can avoid some potential bugs in dev node

* Add proof validation to batch poster (#301)

* Add espresso proof validation to batch poster

Adds calls to the espresso validation code to the batch poster to
validate the namespace proof and merkle proof in a TEE

* result of make fmt

* Fix lint errors

* Add better logging

* Fix batch poster after merge of upstream/integration

* Remove espresso replay binary modifications (#307)

* Remove espresso validation in the replay binary

* Remove espresso-crypto related dependencies in arbitrator

* remove espressocryptowasm

* Update Makefile and Dockerfile to remove references to unused espresso crypto wasm lib

* remove batch_poster.go.orig file

* Fixes from make fmt

* Remove espresso_ops_test

* Remove additional code related to espresso validation in the staker

* Remove changes on stateless block validator

* Undo the stopwaiter

Previous modifications on this was because we had to stop and start the seuquencers
to function the escape hatch. Right now we don't need that anymore. These changes are
from the upstream

* Rename test flag

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Add attestation quote in batch poster (#306)

* Add attestation quote in batch poster

* write file

* update batch poster

* use read file

* update to userDataHash

* reduce file permissions

* nit

* Submit multiple messages in one transaction (#308)

* Submit multiple messages in one transaction

* Remove the espresso_transaction_payload_signature_test.go

* Update go client

* Remove signature and data signer

* Remove arbitrator/espresso-crypto-helper

* Add size limit to hotshot transaction

---------

Co-authored-by: Sneh Koul <[email protected]>

* Use upstream contracts submodule (#310)

We no longer need the customizations in nitro-contracts (except
potentially some deployment tools) so this commit resets it to the
before the celestia contracts were added

celestiaorg/nitro-contracts@4f8c65c

and makes the nitro code work with that version by removing the hotshot
/ light client address passed to the OSP.

* Remove the legacy code and fix the batch poster (#318)

* Remove the legacy code and fix the test

* fix ci

* format

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: nomaxg <[email protected]>
Co-authored-by: ImJeremyHe <[email protected]>
Co-authored-by: sveitser <[email protected]>
Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sneh1999 added a commit that referenced this pull request Nov 20, 2024
* Add SwitchSequencer

* Fix the switch function

* Create SwitchSequencer

* Update the espresso-sequencer-go

* Use light client as the trigger

* Add the switch test

* Make the linter happy

* Add the switch test to the e2e ci

* Increase the limit

* Clean the log with the log helper

* more mock code

* Fix the e2e test

* serialize proof variables correctly

* update espresso-go version

* wasm compatibility

* remove stale file, lint fix

* update to recent espresso go version

* refactor block merkle verification

* work on E2E

* progress on merkle tree verification

* add test vectors, fetch proofs at the correct indicies

* cleanup, remove outfiles, fetch merkle proof at the correct justified height

* add logic to compare circuit field

* check that circuit representation equals u256 representation in the test

* send byte slice to wasm instead of byte array which lives on the stack

* cleanup, remove debug statements, add documentation

* remove error logs

* fix build

* lint fix

* progress on e2e tests, comment out vid stuff for now because of wasm io issues

* update espresso-sequencer-go tag, lint ci fix

* cleanup, add issues for unresolved problems

* remove block that spams transactions

* comment out unused txns

* add test for advz equivalence and use a smaller srs

* Allow dead code in validation

* Specify foundry version

* Add random_get stub in wasi_snapshot_preview1

* Add stubs required in wasi_snapshot_preview1

* Fix `make lint`

* Enable prover service to talk to L1 on host

* Add builder timeout

This allows the sequencer to make progress past block 1.

These env vars have been changed in espresso-sequencer.

* Add missing env var, more logging in e2e test

* Revert "Fix `make lint`"

This reverts commit 8be940c.

* Fix go linting

* Parse espresso messages

* Add timeout limit

* Fix the e2e test

* Add timeout limit

* Run the e2e test and switch test in 1 thread

* Update the e2e workflow

* Increase time limit

* Specify the foundryup version in Dockerfile

* Add sock_accept stub

* Increase timeout limit

* Build successfully

* move ark-srs to dev dependencies

* remove debug statements

* remove out.txt

* Fix the machine test

* Fix CI

* Fix switch test

* Fix CI

* Increase CI time limie

* update lockfile

* remove commented out replaces

* newline

* Use the latest foundry

* RESET ME: debug

* RESET ME: not run the CI in a container

* RESET ME: remove the parallel

* Update script to run tests locally

- Run the same commands as on CI.
- Add ability to run `cargo +nightly` to dev shell.
- make clean: remove brotli build files.

* set -parallel=2

* Fix e2e test

* Skip some tests in CI

* Skip tests in stylus

* Remove parallelism restriction

* Impl escape hatch on validator side

* Fix tests

* Generate the osp of hotshot liveness

* Fix the fraud proof (#124)

* Update contracts submodule

* Upgrade go client

* Fix the staker challenge tests
---------

Co-authored-by: sveitser <[email protected]>

* Use espresso dev node

* attempt to fix dockerfile

* newline

* WIP

* WIP

* download latest espresso module root

* incorporate vid common data

* Fix the validation pipeline

* update go modules, e2e build step

* add cbrotli wasm

* ci fix attempt

* dont build all test deps for e2e test

* lint fixes

* Use the self-hosted runner and install cbindgen in CI

* Add time limit

* Remove parallel in building

* Fix the switch test

* Decrease the waiting hotshot blocks

* Use the ubuntu

* Specify the foundry version

* Enable -j

* update foundry version

* module root update

* Fix the staker test

* increase validation timeout for switch test

* pin espresso-sequencer-go to new release v0.0.20

* Commit go.sum from running go mod tidy

* sequencer info logs

* Add escape hatch test in e2e test

* Check the reactivate hotshot

* Update validator to handle preconditions and panics before entering the espresso STF logic.

* Improve readability of conditions in `handleEspressoPreConditions` and update the functions comments.

* Further improve readability

* Fix lint issue

* Update the nitro testnode

* Fix tests

* Update contracts

* Fix CI

* Changes to `handleEspressoPreConditions` in `espresso_validation.go` to fix incorrect logic, and give more granular control to where and how the generated panic is handled in the validators STF via returned closures.

* return nil when we need a no-op panic handler

* Fix lint errors

* Update contracts

* Fix the validation

* Remove the switch test

* Fix CI

* Remove postgres in espresso e2e test

* Codeowners: add Alysia, Zach, Sneh (#154)

For #146

* Remove Dockerfile.espresso (#156)

* Remove Dockerfile.espresso

* address review comments

* add new line

* Release 20240723 with 20240723-consensus

This should include all our changes for nitro v3.1.

- Check that the module root matches when downloading releases from
  github.

* Check all "not found" errors

Currently only the leveldb not found errors are handled. If a pebble not
found error is returned by the backend it is propagated, causing the
batchposter to fail to send transactions.

With this change all DB not found errors are handled the same way.

* Submit L2 Message to espresso network (#173)

* Submit L2 Message to espresso network

* add context timeout

* check for hotshot liveness and add a polling interval to see if transactions were included in an espresso block

* cleanup

* lint

* Draft the sovereign sequencer test

* Add espresso transaction queue

* Draft the espresso submission

* Set a new l2 message type for the sovereign sequencer transactions

* Attempt to write messages again

* Fix typos and add some comments

* Fix typo

* add header to jst and submit transaction to espresso

* Update go client for development

* Confirm the transaction inclusion in hotshot

* Fix the test

* Unblock the building blocks

* Update go client

* Fix the sovereign test

* add logs

* cleanup

* lint

* Updage go client

* not to fetch merkle proof if hotshot height is 0

* Cleanup

* Fix CI

* Add CI skip tests

* remove unwanted variable

* skipping some tests to make ci pass

* skipping some tests to make ci pass

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Persistent storage for sovereign builder (#180)

* Persistence storage for sovereign builder

* add unit test

* add unit test to ci and add mutex

* Optional header in jst

* initialize pending transactions if not present

* remove unwanted test

* resolve bugs

* remove nil pointer panic

* cleanup

* revert to strings.contains

* fix isErrNotFound

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Save digest artifacts into own directory per run (#182)

* Save digest artifacts into own directory per run

Currently it seems that we have too many digests in the folder and I
think this breaks the assembly of the multiarch docker image.

With this change only digests for the same run are saved in the same
directory.

* Fail docker image merge on wrong number of digests
* Clean digests dir

* Add dependabot rule to update nitro-testnode (#189)

* Fix build (#191)

* Add missing dependency
* Use renamed variable, fix typo

* dependabot: update contracts submodule (#193)

* Pin the golint version in CI

* Fix pebble panic (#192)

* Fix pebble panic

* lint

* Update some outdated help on flags (#203)

* Improve help texts for flags

* Update e2e tests to use Sovereign sequencer (#204)

* Update e2e tests to use Sovereign sequencer


Co-authored-by: ImJeremyHe <[email protected]>

* Remove the global state for HotShot height (#210)

* Remove the global state

* Update the submodule

* CI: use buildjet instead of self-hosted runner (#212)

* CI: use buildjet instead of self-hosted runner

* CI: use public runner / buildjet for docker build

This removes usage of our self-hosted github runners.

* CI: more disk space on public runner

* Check disk space at end of runs

* Try running CI tests from different directory

This may avoid failures due to IPC paths being too long

* Make workspace dir a symlink to /tmp/gha-work

* Fix redis URL

* Move redis tests to own workflow

* CI: fix rust and go cache (#217)

* rust-cache: add more target dirs
* Don't set custom TMPDIR

This seems to be the culprit for the extremely long path names that
are too long for IPC paths.

* modify arbosstate.go to allow for upgrade to arbOS version 35 (#216)

* modify arbosstate.go to allow for upgrade to arbOS version 35

* Fix lint

* Modify the chain config during the ArbOS upgrade

---------

Co-authored-by: sveitser <[email protected]>

* CI: make more space (#218)

* Store transaction pos in batch poster (#219)

* Move submit transaction pos from tx_streamer to batch poster

* Remove espresso sequencer (#214)

* Remove espresso sequencer

* lint

* remove enable espresso

* commit go-ethereum

* fix solgen/gen.go

* fix build

* revert .gitmodules

* revert go.mod

* add enable espresso to validation check

* re-add enable espresso to tests

---------

Co-authored-by: Mathis <[email protected]>

* Update espresso-sequencer-go with new LC ABI (#230)

* Update espresso-sequencer-go with new LC ABI

Instead of using the block height from the first snapshot after the
height in the justification it uses the latest height in the LC
contract. Currently not 100% convinced about correctness but also
believe it should work because any newer tree still contains the header
of interest.

* Undo LC ABI changes

We're not comfortable with the previous change because making a proof
w.r.t to the newest finalized state may cause issues if the batch poster
is lagging behind. This is currently not easy to test so we decided to
revert back to the old flow.

* Use tag of espresso-sequencer-go

* Stop setting the chain config during ArbOS upgrade in favor of setting it manually during migration process. (#234)

* Use the tag before marketplace update (#238)

* Sign payload for sovereign builder (#181)

* Sign payload for sovereign builder

* save signed payload to db

* change parsing logic to include signature

* add some comments

* skip tests

* address comments

* add signature payload test

* test working

* fix logging and comments

* move placement of log

* fix ci

* fix ci

* only submit transaction if jst nil

* add log to debug ci

* use the latest instead of main for espresso-dev-node

* pin to legacy lc version

* pin to before marketplace version

* remove log

* Full node with espresso finality (#237)

* wip

* wip

* Add espresso finality node test

* fix ci

* fix ci

* pin to before marketplace version

* fix tests

* add comments

* address comments

* address comments

* address comments

* cleanup

* fix test

* fmt

* rerun ci

* ci skip tests

* Update to sequencer.go to remove `EnableEspressoSovereign` flag, and instead sequence based on hotshot liveness at l1 block height.

* Update node.go to remove reference to `EnableEspressoSovereign`

* update Espresso tests to reflect removed config variables, and add escape hatch test to e2e test.

* Add wait for hotshot liveness in sovereign sequencer test to ensure sovereign sequencer is enabled.

* go fmt

* Add comment to explain changes needed in replay binary when panics are no longer an issue.

* Fix lint issues

* Modify construction of lightClientReader in sequencer.go to allow for nil `l1Reader`

* Update test configs to align with changes.

* Update comments according to feedback.

* Bump nitro-testnode from `03095e0` to `884a63c` (#240)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `03095e0` to `884a63c`.
- [Commits](EspressoSystems/nitro-testnode@03095e0...884a63c)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mathis <[email protected]>

* Revert removal of `EnableEspressoSovereign`

* Add additional check to ensure that if EnableEspressoSovereign is on we can't construct a sequencer with a nil lightClientReader

* Add `EnableEspressoSovereign` config flag to test configs.

* Bump contracts from `d226e9d` to `2ae1bbd` (#244)

Bumps [contracts](https://github.com/EspressoSystems/nitro-contracts) from `d226e9d` to `2ae1bbd`.
- [Release notes](https://github.com/EspressoSystems/nitro-contracts/releases)
- [Commits](EspressoSystems/nitro-contracts@d226e9d...2ae1bbd)

---
updated-dependencies:
- dependency-name: contracts
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Pin rust to 1.81.0

See #257

* Set cargo config for espresso_crypto wasm build

This was recently added for the other wasm compilation steps.

* Pin rust in E2E test

* CI: remove deprecated rust toolchain action

* Fix if statement

* Regenerate data-poster external-signer test certs

(cherry picked from commit 8f961b0)

* Use ArbOS chain config as the cannonial chain config. (#252)

* Add checks to sequencer.go and transaction_streamer.go to utilize the chain config switch

* Update interface.go to export interface for `GetArbOSConfigAtHeight()`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionEngine`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionNode`

* Update `sequencer.go` and `transaction_streamer.go` to read from the `ArbOS` `chainConfig` as the cannonical `chainConfig` rather than the `chainConfig` held by the `Blockchain` struct.

* Add ArbOS chain config check in batch poster to stop adding espresso messages to the transaction streamers DB if EnableEspresso is false.

* Add missing error check in sequencer.go and nil check on arbos config in sequencer.go and batch_poster.go.

* Remove ineffectual assignment in sequencer.go

* Add test for new GetArbOSConfigAtHeight() method in ExecutionNode

This commit adds an integration test for the GetArbOSConfigAtHeight()
method on the Execution node that grabs the initial config, and
validates it, then updates the config and ensures that the new method
will fetch the updated config.

* restore nitro-testnode to proper commit

* Fix trie node node found error during recording

I'm not exactly clear why we need to do this or if it's indeed the right
approach to solving the issue however it does seem to make it possible
to validate again.

Without it we get errors like

    INFO [10-30|12:24:29.127] DelayedSequencer: Sequenced              msgnum=1 startpos=1
    INFO [10-30|12:24:30.143] Reading message result remotely.         count=3
    WARN [10-30|12:24:32.143] prepareblocks failed to get state for block hdrNum=0 err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found"
    ERROR[10-30|12:24:32.143] Error while recording                    err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found" status=2

* Fix the OSP test

- Enable 4844 blobs to avoid contract deployment error
- Fix address of OSP entry

* Disable batch poster in Espresso finality node

We don't need this type of node to have a batch poster. Enabling it now
causes the test to fail because there's a check to ensure that redis is
used for the batch poster on the second node.

* Re-allow setting L1 stack config for tests

After the sync the code would always use the default values.

Don't think this is very clean but I couldn't think of something better
yet.

* CI: increase parallelism

* Fix linting

- Run `golang-lint run --fix`
- Check error after `waitForConfigUpdate`

* Move finality node test to espresso tests

The race test detected a data race in

https://github.com/EspressoSystems/nitro-espresso-integration/actions/runs/11705919237/job/32601803846?pr=254

* Update contracts submodule

* Fix typo

* Add wabt and cmake in wasm shell

* Skip tests

* Add tags in ci

* Add ci skip tests

* Add ci skip tests

* Remove rerun-fails

This flag in gotestsum causes some bugs which multiple the test time.

gotestyourself/gotestsum#417
gotestyourself/gotestsum#423

* Revert "Remove rerun-fails"

This reverts commit 6bef1c3.

* CI: simplify

- Add scripts to debug more easily locally.
- Skip tests by substring match instead of exact match.
- Don't run coverage on CI, we never look at it.
- Move all Espresso tests to Espresso CI job.

* Fix build with nix on macos (#296)

* CI: install latest gotestsum

This enables the --hide-summary argument

* Update submodule forks with merge commits

* Bump nitro-testnode from `07a25f1` to `77d1447` (#297)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `07a25f1` to `77d1447`.
- [Commits](EspressoSystems/nitro-testnode@07a25f1...77d1447)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add FFI bindings for Espresso validation (#294)

* Example call to espresso-crypto function stub in batch poster.

This commit includes a log line and example call to the espresso-crypto
function stubs in batch_poster.go

* Result of `make fmt`

Run `make fmt` to clean up formatting not apparent in editor

* Make current lib.rs c abi compatible

* result of make fmt

* Add C ABI ffi between rust crypto lib and espressocrypto go module.

This commit adds FFI support between exsisting Rust crypto helper code
and the exsisting espressocrypto go module in native.go These functions
will now serve as utility functions to use in the integration that will
handle ffi to the Rust code.

This commit also moves the espresso-crypto-helper code to the
espressocrypto directory as we no longer need it for the arbitrator.

* Build native.go successfully

* Update the espresso-crypto and add test for merkle proof verification

* update flake.nix to new go version and fix errors

* Current state of work

The current state of my work revolves around editing the Makefile to
include the new espressocrypto ffi functions in the build process.
Additionally, this commit restores some espressocrypto code to the
arbitrator in an attempt to get a docker image built via the test node.
I felt as though this would be easier for this PR as opposed to ripping
out all of the code from the arbitrator.

* Update the flake.lock

* Add the namespace proof test

* Current state of build system changes

This commit represents the current state of the build system after
testing today.
Currently things compile locally, but not when building an image from
the docker file. There are errors related to duplicate function
definitions from what seem to be the FFI lib that stylus exposes.

* Build nitro docker image sucessfully

- Modify makefile to have a file target. Since the build artifact of our
  static library is a file we should use this as the make target
  (instead of only having PHONY targets).

- Use rust 1.81 to build all shared rust libraries in docker.

Our crate requires a minimum version of 1.81. However the other
libraries like libstylus.a are currently compiled with 1.75 and this
seems to cause issues with duplicate symbol definitions during linking.

For example:

    8.718 go build -ldflags="-extldflags '-ldl' -X github.com/offchainlabs/nitro/cmd/util/confighelpers.version= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.datetime= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.modified=" -o target/bin/nitro "/workspace/cmd/nitro"
    42.74 # github.com/offchainlabs/nitro/cmd/nitro
    42.74 /usr/local/go/pkg/tool/linux_amd64/link: running clang failed: exit status 1
    42.74 /usr/bin/ld: /workspace/espressocrypto/lib/espresso-crypto-helper/target/release/libespresso_crypto_helper.a(std-1c4b19562077c20d.std.1f2242ed6435445e-cgu.0.rcgu.o): in function `__rdl_alloc':
    42.74 /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/alloc.rs:397: multiple definition of `__rdl_alloc'; /workspace/execution/gethexec/../../target/lib/libstylus.a(std-90f6ddbf82de36ec.std.409886f6357001f0-cgu.0.rcgu.o):/rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:389: first defined here

Bump the rust version of the prover-header-builder to 1.81 and compile
the espresso crypto library in the same container.

* Fix LDFLAGS, remove unnecessary CFLAGS

* Fix some merge errors

- We need rust 1.81 to compile the espresso crypto lib
- Re-add makefile rules

* Fix merge errors

* Remove FFI call in batch poster to expidite merge.

* Update bf submodule in arbitrator to match upstream version (This was set to a different commit during my erroneous merge)

* Remove extra change in espressocrypto_wasm.go

* Fix error with temporary espressocryptowasm directory preventing CI from running

* fix error of missing function definitions in espressocryptowasm directory to allow CI to run

* Re-add nosec comment about gosec error that will be removed soon

* result of make fmt

---------

Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: ImJeremyHe <[email protected]>

* Update the go client (#256)

Still using the legacy dev node

* ci: fix `git ls-remote` error in docker build (#304)

* Use path context instead of git context

With the default git context the repository is re-cloned at the start of
the docker build and the .dockerignore file is ignored.

> Be careful because any file mutation in the steps that precede the
> build step will be ignored, including processing of the .dockerignore
> file since the context is based on the Git reference.

https://github.com/docker/build-push-action?tab=readme-ov-file#git-context

This leads to some `.git` worktree files showing up inside the
dockerbuild with triggers a bug in yarn.

See yarnpkg/yarn#7537 (comment)
for more details about the yarn bug.

* Use the latest espresso dev node (#303)

* Use the latest espresso dev node

- Fix an issue that may cause the validation reorg
- Waiting for L1 to be ready can avoid some potential bugs in dev node

* Add proof validation to batch poster (#301)

* Add espresso proof validation to batch poster

Adds calls to the espresso validation code to the batch poster to
validate the namespace proof and merkle proof in a TEE

* result of make fmt

* Fix lint errors

* Add better logging

* Fix batch poster after merge of upstream/integration

* Remove espresso replay binary modifications (#307)

* Remove espresso validation in the replay binary

* Remove espresso-crypto related dependencies in arbitrator

* remove espressocryptowasm

* Update Makefile and Dockerfile to remove references to unused espresso crypto wasm lib

* remove batch_poster.go.orig file

* Fixes from make fmt

* Remove espresso_ops_test

* Remove additional code related to espresso validation in the staker

* Remove changes on stateless block validator

* Undo the stopwaiter

Previous modifications on this was because we had to stop and start the seuquencers
to function the escape hatch. Right now we don't need that anymore. These changes are
from the upstream

* Rename test flag

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Add attestation quote in batch poster (#306)

* Add attestation quote in batch poster

* write file

* update batch poster

* use read file

* update to userDataHash

* reduce file permissions

* nit

* Submit multiple messages in one transaction (#308)

* Submit multiple messages in one transaction

* Remove the espresso_transaction_payload_signature_test.go

* Update go client

* Remove signature and data signer

* Remove arbitrator/espresso-crypto-helper

* Add size limit to hotshot transaction

---------

Co-authored-by: Sneh Koul <[email protected]>

* Use espresso-types instead of hard coded files

* WIP: locally failed

* Use upstream contracts submodule (#310)

We no longer need the customizations in nitro-contracts (except
potentially some deployment tools) so this commit resets it to the
before the celestia contracts were added

celestiaorg/nitro-contracts@4f8c65c

and makes the nitro code work with that version by removing the hotshot
/ light client address passed to the OSP.

* Remove the legacy code and fix the batch poster (#318)

* Remove the legacy code and fix the test

* fix nix build on linux

- flake: add openssl, pkg-config
- rust: update espresso-types to latest to not require testing features

* nix: fix build on darwin

- Update nixpkgs
- Set NIX_LDFLAGS to find missing symbols
- Leave note about updating flake since the apple SDKs in nixpkgs have
  been changed substantially.

* Transmute nstable

* add celestia-integration branch to the ci

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: nomaxg <[email protected]>
Co-authored-by: sveitser <[email protected]>
Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sneh Koul <[email protected]>
ImJeremyHe added a commit that referenced this pull request Dec 4, 2024
* Add SwitchSequencer

* Fix the switch function

* Create SwitchSequencer

* Update the espresso-sequencer-go

* Use light client as the trigger

* Add the switch test

* Make the linter happy

* Add the switch test to the e2e ci

* Increase the limit

* Clean the log with the log helper

* more mock code

* Fix the e2e test

* serialize proof variables correctly

* update espresso-go version

* wasm compatibility

* remove stale file, lint fix

* update to recent espresso go version

* refactor block merkle verification

* work on E2E

* progress on merkle tree verification

* add test vectors, fetch proofs at the correct indicies

* cleanup, remove outfiles, fetch merkle proof at the correct justified height

* add logic to compare circuit field

* check that circuit representation equals u256 representation in the test

* send byte slice to wasm instead of byte array which lives on the stack

* cleanup, remove debug statements, add documentation

* remove error logs

* fix build

* lint fix

* progress on e2e tests, comment out vid stuff for now because of wasm io issues

* update espresso-sequencer-go tag, lint ci fix

* cleanup, add issues for unresolved problems

* remove block that spams transactions

* comment out unused txns

* add test for advz equivalence and use a smaller srs

* Allow dead code in validation

* Specify foundry version

* Add random_get stub in wasi_snapshot_preview1

* Add stubs required in wasi_snapshot_preview1

* Fix `make lint`

* Enable prover service to talk to L1 on host

* Add builder timeout

This allows the sequencer to make progress past block 1.

These env vars have been changed in espresso-sequencer.

* Add missing env var, more logging in e2e test

* Revert "Fix `make lint`"

This reverts commit 8be940c.

* Fix go linting

* Parse espresso messages

* Add timeout limit

* Fix the e2e test

* Add timeout limit

* Run the e2e test and switch test in 1 thread

* Update the e2e workflow

* Increase time limit

* Specify the foundryup version in Dockerfile

* Add sock_accept stub

* Increase timeout limit

* Build successfully

* move ark-srs to dev dependencies

* remove debug statements

* remove out.txt

* Fix the machine test

* Fix CI

* Fix switch test

* Fix CI

* Increase CI time limie

* update lockfile

* remove commented out replaces

* newline

* Use the latest foundry

* RESET ME: debug

* RESET ME: not run the CI in a container

* RESET ME: remove the parallel

* Update script to run tests locally

- Run the same commands as on CI.
- Add ability to run `cargo +nightly` to dev shell.
- make clean: remove brotli build files.

* set -parallel=2

* Fix e2e test

* Skip some tests in CI

* Skip tests in stylus

* Remove parallelism restriction

* Impl escape hatch on validator side

* Fix tests

* Generate the osp of hotshot liveness

* Fix the fraud proof (#124)

* Update contracts submodule

* Upgrade go client

* Fix the staker challenge tests
---------

Co-authored-by: sveitser <[email protected]>

* Use espresso dev node

* attempt to fix dockerfile

* newline

* WIP

* WIP

* download latest espresso module root

* incorporate vid common data

* Fix the validation pipeline

* update go modules, e2e build step

* add cbrotli wasm

* ci fix attempt

* dont build all test deps for e2e test

* lint fixes

* Use the self-hosted runner and install cbindgen in CI

* Add time limit

* Remove parallel in building

* Fix the switch test

* Decrease the waiting hotshot blocks

* Use the ubuntu

* Specify the foundry version

* Enable -j

* update foundry version

* module root update

* Fix the staker test

* increase validation timeout for switch test

* pin espresso-sequencer-go to new release v0.0.20

* Commit go.sum from running go mod tidy

* sequencer info logs

* Add escape hatch test in e2e test

* Check the reactivate hotshot

* Update validator to handle preconditions and panics before entering the espresso STF logic.

* Improve readability of conditions in `handleEspressoPreConditions` and update the functions comments.

* Further improve readability

* Fix lint issue

* Update the nitro testnode

* Fix tests

* Update contracts

* Fix CI

* Changes to `handleEspressoPreConditions` in `espresso_validation.go` to fix incorrect logic, and give more granular control to where and how the generated panic is handled in the validators STF via returned closures.

* return nil when we need a no-op panic handler

* Fix lint errors

* Update contracts

* Fix the validation

* Remove the switch test

* Fix CI

* Remove postgres in espresso e2e test

* Codeowners: add Alysia, Zach, Sneh (#154)

For #146

* Remove Dockerfile.espresso (#156)

* Remove Dockerfile.espresso

* address review comments

* add new line

* Release 20240723 with 20240723-consensus

This should include all our changes for nitro v3.1.

- Check that the module root matches when downloading releases from
  github.

* Check all "not found" errors

Currently only the leveldb not found errors are handled. If a pebble not
found error is returned by the backend it is propagated, causing the
batchposter to fail to send transactions.

With this change all DB not found errors are handled the same way.

* Submit L2 Message to espresso network (#173)

* Submit L2 Message to espresso network

* add context timeout

* check for hotshot liveness and add a polling interval to see if transactions were included in an espresso block

* cleanup

* lint

* Draft the sovereign sequencer test

* Add espresso transaction queue

* Draft the espresso submission

* Set a new l2 message type for the sovereign sequencer transactions

* Attempt to write messages again

* Fix typos and add some comments

* Fix typo

* add header to jst and submit transaction to espresso

* Update go client for development

* Confirm the transaction inclusion in hotshot

* Fix the test

* Unblock the building blocks

* Update go client

* Fix the sovereign test

* add logs

* cleanup

* lint

* Updage go client

* not to fetch merkle proof if hotshot height is 0

* Cleanup

* Fix CI

* Add CI skip tests

* remove unwanted variable

* skipping some tests to make ci pass

* skipping some tests to make ci pass

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Persistent storage for sovereign builder (#180)

* Persistence storage for sovereign builder

* add unit test

* add unit test to ci and add mutex

* Optional header in jst

* initialize pending transactions if not present

* remove unwanted test

* resolve bugs

* remove nil pointer panic

* cleanup

* revert to strings.contains

* fix isErrNotFound

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Save digest artifacts into own directory per run (#182)

* Save digest artifacts into own directory per run

Currently it seems that we have too many digests in the folder and I
think this breaks the assembly of the multiarch docker image.

With this change only digests for the same run are saved in the same
directory.

* Fail docker image merge on wrong number of digests
* Clean digests dir

* Add dependabot rule to update nitro-testnode (#189)

* Fix build (#191)

* Add missing dependency
* Use renamed variable, fix typo

* dependabot: update contracts submodule (#193)

* Pin the golint version in CI

* Fix pebble panic (#192)

* Fix pebble panic

* lint

* Update some outdated help on flags (#203)

* Improve help texts for flags

* Update e2e tests to use Sovereign sequencer (#204)

* Update e2e tests to use Sovereign sequencer


Co-authored-by: ImJeremyHe <[email protected]>

* Remove the global state for HotShot height (#210)

* Remove the global state

* Update the submodule

* CI: use buildjet instead of self-hosted runner (#212)

* CI: use buildjet instead of self-hosted runner

* CI: use public runner / buildjet for docker build

This removes usage of our self-hosted github runners.

* CI: more disk space on public runner

* Check disk space at end of runs

* Try running CI tests from different directory

This may avoid failures due to IPC paths being too long

* Make workspace dir a symlink to /tmp/gha-work

* Fix redis URL

* Move redis tests to own workflow

* CI: fix rust and go cache (#217)

* rust-cache: add more target dirs
* Don't set custom TMPDIR

This seems to be the culprit for the extremely long path names that
are too long for IPC paths.

* modify arbosstate.go to allow for upgrade to arbOS version 35 (#216)

* modify arbosstate.go to allow for upgrade to arbOS version 35

* Fix lint

* Modify the chain config during the ArbOS upgrade

---------

Co-authored-by: sveitser <[email protected]>

* CI: make more space (#218)

* Store transaction pos in batch poster (#219)

* Move submit transaction pos from tx_streamer to batch poster

* Remove espresso sequencer (#214)

* Remove espresso sequencer

* lint

* remove enable espresso

* commit go-ethereum

* fix solgen/gen.go

* fix build

* revert .gitmodules

* revert go.mod

* add enable espresso to validation check

* re-add enable espresso to tests

---------

Co-authored-by: Mathis <[email protected]>

* Update espresso-sequencer-go with new LC ABI (#230)

* Update espresso-sequencer-go with new LC ABI

Instead of using the block height from the first snapshot after the
height in the justification it uses the latest height in the LC
contract. Currently not 100% convinced about correctness but also
believe it should work because any newer tree still contains the header
of interest.

* Undo LC ABI changes

We're not comfortable with the previous change because making a proof
w.r.t to the newest finalized state may cause issues if the batch poster
is lagging behind. This is currently not easy to test so we decided to
revert back to the old flow.

* Use tag of espresso-sequencer-go

* Stop setting the chain config during ArbOS upgrade in favor of setting it manually during migration process. (#234)

* Use the tag before marketplace update (#238)

* Sign payload for sovereign builder (#181)

* Sign payload for sovereign builder

* save signed payload to db

* change parsing logic to include signature

* add some comments

* skip tests

* address comments

* add signature payload test

* test working

* fix logging and comments

* move placement of log

* fix ci

* fix ci

* only submit transaction if jst nil

* add log to debug ci

* use the latest instead of main for espresso-dev-node

* pin to legacy lc version

* pin to before marketplace version

* remove log

* Full node with espresso finality (#237)

* wip

* wip

* Add espresso finality node test

* fix ci

* fix ci

* pin to before marketplace version

* fix tests

* add comments

* address comments

* address comments

* address comments

* cleanup

* fix test

* fmt

* rerun ci

* ci skip tests

* Update to sequencer.go to remove `EnableEspressoSovereign` flag, and instead sequence based on hotshot liveness at l1 block height.

* Update node.go to remove reference to `EnableEspressoSovereign`

* update Espresso tests to reflect removed config variables, and add escape hatch test to e2e test.

* Add wait for hotshot liveness in sovereign sequencer test to ensure sovereign sequencer is enabled.

* go fmt

* Add comment to explain changes needed in replay binary when panics are no longer an issue.

* Fix lint issues

* Modify construction of lightClientReader in sequencer.go to allow for nil `l1Reader`

* Update test configs to align with changes.

* Update comments according to feedback.

* Bump nitro-testnode from `03095e0` to `884a63c` (#240)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `03095e0` to `884a63c`.
- [Commits](EspressoSystems/nitro-testnode@03095e0...884a63c)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mathis <[email protected]>

* Revert removal of `EnableEspressoSovereign`

* Add additional check to ensure that if EnableEspressoSovereign is on we can't construct a sequencer with a nil lightClientReader

* Add `EnableEspressoSovereign` config flag to test configs.

* Bump contracts from `d226e9d` to `2ae1bbd` (#244)

Bumps [contracts](https://github.com/EspressoSystems/nitro-contracts) from `d226e9d` to `2ae1bbd`.
- [Release notes](https://github.com/EspressoSystems/nitro-contracts/releases)
- [Commits](EspressoSystems/nitro-contracts@d226e9d...2ae1bbd)

---
updated-dependencies:
- dependency-name: contracts
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Pin rust to 1.81.0

See #257

* Set cargo config for espresso_crypto wasm build

This was recently added for the other wasm compilation steps.

* Pin rust in E2E test

* CI: remove deprecated rust toolchain action

* Fix if statement

* Regenerate data-poster external-signer test certs

(cherry picked from commit 8f961b0)

* Use ArbOS chain config as the cannonial chain config. (#252)

* Add checks to sequencer.go and transaction_streamer.go to utilize the chain config switch

* Update interface.go to export interface for `GetArbOSConfigAtHeight()`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionEngine`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionNode`

* Update `sequencer.go` and `transaction_streamer.go` to read from the `ArbOS` `chainConfig` as the cannonical `chainConfig` rather than the `chainConfig` held by the `Blockchain` struct.

* Add ArbOS chain config check in batch poster to stop adding espresso messages to the transaction streamers DB if EnableEspresso is false.

* Add missing error check in sequencer.go and nil check on arbos config in sequencer.go and batch_poster.go.

* Remove ineffectual assignment in sequencer.go

* Add test for new GetArbOSConfigAtHeight() method in ExecutionNode

This commit adds an integration test for the GetArbOSConfigAtHeight()
method on the Execution node that grabs the initial config, and
validates it, then updates the config and ensures that the new method
will fetch the updated config.

* restore nitro-testnode to proper commit

* Fix trie node node found error during recording

I'm not exactly clear why we need to do this or if it's indeed the right
approach to solving the issue however it does seem to make it possible
to validate again.

Without it we get errors like

    INFO [10-30|12:24:29.127] DelayedSequencer: Sequenced              msgnum=1 startpos=1
    INFO [10-30|12:24:30.143] Reading message result remotely.         count=3
    WARN [10-30|12:24:32.143] prepareblocks failed to get state for block hdrNum=0 err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found"
    ERROR[10-30|12:24:32.143] Error while recording                    err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found" status=2

* Fix the OSP test

- Enable 4844 blobs to avoid contract deployment error
- Fix address of OSP entry

* Disable batch poster in Espresso finality node

We don't need this type of node to have a batch poster. Enabling it now
causes the test to fail because there's a check to ensure that redis is
used for the batch poster on the second node.

* Re-allow setting L1 stack config for tests

After the sync the code would always use the default values.

Don't think this is very clean but I couldn't think of something better
yet.

* CI: increase parallelism

* Fix linting

- Run `golang-lint run --fix`
- Check error after `waitForConfigUpdate`

* Move finality node test to espresso tests

The race test detected a data race in

https://github.com/EspressoSystems/nitro-espresso-integration/actions/runs/11705919237/job/32601803846?pr=254

* Update contracts submodule

* Fix typo

* Add wabt and cmake in wasm shell

* Skip tests

* Add tags in ci

* Add ci skip tests

* Add ci skip tests

* Remove rerun-fails

This flag in gotestsum causes some bugs which multiple the test time.

gotestyourself/gotestsum#417
gotestyourself/gotestsum#423

* Revert "Remove rerun-fails"

This reverts commit 6bef1c3.

* CI: simplify

- Add scripts to debug more easily locally.
- Skip tests by substring match instead of exact match.
- Don't run coverage on CI, we never look at it.
- Move all Espresso tests to Espresso CI job.

* Fix build with nix on macos (#296)

* CI: install latest gotestsum

This enables the --hide-summary argument

* Update submodule forks with merge commits

* Bump nitro-testnode from `07a25f1` to `77d1447` (#297)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `07a25f1` to `77d1447`.
- [Commits](EspressoSystems/nitro-testnode@07a25f1...77d1447)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add FFI bindings for Espresso validation (#294)

* Example call to espresso-crypto function stub in batch poster.

This commit includes a log line and example call to the espresso-crypto
function stubs in batch_poster.go

* Result of `make fmt`

Run `make fmt` to clean up formatting not apparent in editor

* Make current lib.rs c abi compatible

* result of make fmt

* Add C ABI ffi between rust crypto lib and espressocrypto go module.

This commit adds FFI support between exsisting Rust crypto helper code
and the exsisting espressocrypto go module in native.go These functions
will now serve as utility functions to use in the integration that will
handle ffi to the Rust code.

This commit also moves the espresso-crypto-helper code to the
espressocrypto directory as we no longer need it for the arbitrator.

* Build native.go successfully

* Update the espresso-crypto and add test for merkle proof verification

* update flake.nix to new go version and fix errors

* Current state of work

The current state of my work revolves around editing the Makefile to
include the new espressocrypto ffi functions in the build process.
Additionally, this commit restores some espressocrypto code to the
arbitrator in an attempt to get a docker image built via the test node.
I felt as though this would be easier for this PR as opposed to ripping
out all of the code from the arbitrator.

* Update the flake.lock

* Add the namespace proof test

* Current state of build system changes

This commit represents the current state of the build system after
testing today.
Currently things compile locally, but not when building an image from
the docker file. There are errors related to duplicate function
definitions from what seem to be the FFI lib that stylus exposes.

* Build nitro docker image sucessfully

- Modify makefile to have a file target. Since the build artifact of our
  static library is a file we should use this as the make target
  (instead of only having PHONY targets).

- Use rust 1.81 to build all shared rust libraries in docker.

Our crate requires a minimum version of 1.81. However the other
libraries like libstylus.a are currently compiled with 1.75 and this
seems to cause issues with duplicate symbol definitions during linking.

For example:

    8.718 go build -ldflags="-extldflags '-ldl' -X github.com/offchainlabs/nitro/cmd/util/confighelpers.version= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.datetime= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.modified=" -o target/bin/nitro "/workspace/cmd/nitro"
    42.74 # github.com/offchainlabs/nitro/cmd/nitro
    42.74 /usr/local/go/pkg/tool/linux_amd64/link: running clang failed: exit status 1
    42.74 /usr/bin/ld: /workspace/espressocrypto/lib/espresso-crypto-helper/target/release/libespresso_crypto_helper.a(std-1c4b19562077c20d.std.1f2242ed6435445e-cgu.0.rcgu.o): in function `__rdl_alloc':
    42.74 /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/alloc.rs:397: multiple definition of `__rdl_alloc'; /workspace/execution/gethexec/../../target/lib/libstylus.a(std-90f6ddbf82de36ec.std.409886f6357001f0-cgu.0.rcgu.o):/rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:389: first defined here

Bump the rust version of the prover-header-builder to 1.81 and compile
the espresso crypto library in the same container.

* Fix LDFLAGS, remove unnecessary CFLAGS

* Fix some merge errors

- We need rust 1.81 to compile the espresso crypto lib
- Re-add makefile rules

* Fix merge errors

* Remove FFI call in batch poster to expidite merge.

* Update bf submodule in arbitrator to match upstream version (This was set to a different commit during my erroneous merge)

* Remove extra change in espressocrypto_wasm.go

* Fix error with temporary espressocryptowasm directory preventing CI from running

* fix error of missing function definitions in espressocryptowasm directory to allow CI to run

* Re-add nosec comment about gosec error that will be removed soon

* result of make fmt

---------

Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: ImJeremyHe <[email protected]>

* Update the go client (#256)

Still using the legacy dev node

* ci: fix `git ls-remote` error in docker build (#304)

* Use path context instead of git context

With the default git context the repository is re-cloned at the start of
the docker build and the .dockerignore file is ignored.

> Be careful because any file mutation in the steps that precede the
> build step will be ignored, including processing of the .dockerignore
> file since the context is based on the Git reference.

https://github.com/docker/build-push-action?tab=readme-ov-file#git-context

This leads to some `.git` worktree files showing up inside the
dockerbuild with triggers a bug in yarn.

See yarnpkg/yarn#7537 (comment)
for more details about the yarn bug.

* Use the latest espresso dev node (#303)

* Use the latest espresso dev node

- Fix an issue that may cause the validation reorg
- Waiting for L1 to be ready can avoid some potential bugs in dev node

* Add proof validation to batch poster (#301)

* Add espresso proof validation to batch poster

Adds calls to the espresso validation code to the batch poster to
validate the namespace proof and merkle proof in a TEE

* result of make fmt

* Fix lint errors

* Add better logging

* Fix batch poster after merge of upstream/integration

* Remove espresso replay binary modifications (#307)

* Remove espresso validation in the replay binary

* Remove espresso-crypto related dependencies in arbitrator

* remove espressocryptowasm

* Update Makefile and Dockerfile to remove references to unused espresso crypto wasm lib

* remove batch_poster.go.orig file

* Fixes from make fmt

* Remove espresso_ops_test

* Remove additional code related to espresso validation in the staker

* Remove changes on stateless block validator

* Undo the stopwaiter

Previous modifications on this was because we had to stop and start the seuquencers
to function the escape hatch. Right now we don't need that anymore. These changes are
from the upstream

* Rename test flag

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Add attestation quote in batch poster (#306)

* Add attestation quote in batch poster

* write file

* update batch poster

* use read file

* update to userDataHash

* reduce file permissions

* nit

* Submit multiple messages in one transaction (#308)

* Submit multiple messages in one transaction

* Remove the espresso_transaction_payload_signature_test.go

* Update go client

* Remove signature and data signer

* Remove arbitrator/espresso-crypto-helper

* Add size limit to hotshot transaction

---------

Co-authored-by: Sneh Koul <[email protected]>

* Use espresso-types instead of hard coded files

* WIP: locally failed

* Use upstream contracts submodule (#310)

We no longer need the customizations in nitro-contracts (except
potentially some deployment tools) so this commit resets it to the
before the celestia contracts were added

celestiaorg/nitro-contracts@4f8c65c

and makes the nitro code work with that version by removing the hotshot
/ light client address passed to the OSP.

* Remove the legacy code and fix the batch poster (#318)

* Remove the legacy code and fix the test

* fix nix build on linux

- flake: add openssl, pkg-config
- rust: update espresso-types to latest to not require testing features

* nix: fix build on darwin

- Update nixpkgs
- Set NIX_LDFLAGS to find missing symbols
- Leave note about updating flake since the apple SDKs in nixpkgs have
  been changed substantially.

* Transmute nstable

* add celestia-integration branch to the ci

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: nomaxg <[email protected]>
Co-authored-by: sveitser <[email protected]>
Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sneh Koul <[email protected]>
Sneh1999 added a commit that referenced this pull request Dec 9, 2024
* Clean the espresso crypto lib (#317)

* Add SwitchSequencer

* Fix the switch function

* Create SwitchSequencer

* Update the espresso-sequencer-go

* Use light client as the trigger

* Add the switch test

* Make the linter happy

* Add the switch test to the e2e ci

* Increase the limit

* Clean the log with the log helper

* more mock code

* Fix the e2e test

* serialize proof variables correctly

* update espresso-go version

* wasm compatibility

* remove stale file, lint fix

* update to recent espresso go version

* refactor block merkle verification

* work on E2E

* progress on merkle tree verification

* add test vectors, fetch proofs at the correct indicies

* cleanup, remove outfiles, fetch merkle proof at the correct justified height

* add logic to compare circuit field

* check that circuit representation equals u256 representation in the test

* send byte slice to wasm instead of byte array which lives on the stack

* cleanup, remove debug statements, add documentation

* remove error logs

* fix build

* lint fix

* progress on e2e tests, comment out vid stuff for now because of wasm io issues

* update espresso-sequencer-go tag, lint ci fix

* cleanup, add issues for unresolved problems

* remove block that spams transactions

* comment out unused txns

* add test for advz equivalence and use a smaller srs

* Allow dead code in validation

* Specify foundry version

* Add random_get stub in wasi_snapshot_preview1

* Add stubs required in wasi_snapshot_preview1

* Fix `make lint`

* Enable prover service to talk to L1 on host

* Add builder timeout

This allows the sequencer to make progress past block 1.

These env vars have been changed in espresso-sequencer.

* Add missing env var, more logging in e2e test

* Revert "Fix `make lint`"

This reverts commit 8be940c.

* Fix go linting

* Parse espresso messages

* Add timeout limit

* Fix the e2e test

* Add timeout limit

* Run the e2e test and switch test in 1 thread

* Update the e2e workflow

* Increase time limit

* Specify the foundryup version in Dockerfile

* Add sock_accept stub

* Increase timeout limit

* Build successfully

* move ark-srs to dev dependencies

* remove debug statements

* remove out.txt

* Fix the machine test

* Fix CI

* Fix switch test

* Fix CI

* Increase CI time limie

* update lockfile

* remove commented out replaces

* newline

* Use the latest foundry

* RESET ME: debug

* RESET ME: not run the CI in a container

* RESET ME: remove the parallel

* Update script to run tests locally

- Run the same commands as on CI.
- Add ability to run `cargo +nightly` to dev shell.
- make clean: remove brotli build files.

* set -parallel=2

* Fix e2e test

* Skip some tests in CI

* Skip tests in stylus

* Remove parallelism restriction

* Impl escape hatch on validator side

* Fix tests

* Generate the osp of hotshot liveness

* Fix the fraud proof (#124)

* Update contracts submodule

* Upgrade go client

* Fix the staker challenge tests
---------

Co-authored-by: sveitser <[email protected]>

* Use espresso dev node

* attempt to fix dockerfile

* newline

* WIP

* WIP

* download latest espresso module root

* incorporate vid common data

* Fix the validation pipeline

* update go modules, e2e build step

* add cbrotli wasm

* ci fix attempt

* dont build all test deps for e2e test

* lint fixes

* Use the self-hosted runner and install cbindgen in CI

* Add time limit

* Remove parallel in building

* Fix the switch test

* Decrease the waiting hotshot blocks

* Use the ubuntu

* Specify the foundry version

* Enable -j

* update foundry version

* module root update

* Fix the staker test

* increase validation timeout for switch test

* pin espresso-sequencer-go to new release v0.0.20

* Commit go.sum from running go mod tidy

* sequencer info logs

* Add escape hatch test in e2e test

* Check the reactivate hotshot

* Update validator to handle preconditions and panics before entering the espresso STF logic.

* Improve readability of conditions in `handleEspressoPreConditions` and update the functions comments.

* Further improve readability

* Fix lint issue

* Update the nitro testnode

* Fix tests

* Update contracts

* Fix CI

* Changes to `handleEspressoPreConditions` in `espresso_validation.go` to fix incorrect logic, and give more granular control to where and how the generated panic is handled in the validators STF via returned closures.

* return nil when we need a no-op panic handler

* Fix lint errors

* Update contracts

* Fix the validation

* Remove the switch test

* Fix CI

* Remove postgres in espresso e2e test

* Codeowners: add Alysia, Zach, Sneh (#154)

For #146

* Remove Dockerfile.espresso (#156)

* Remove Dockerfile.espresso

* address review comments

* add new line

* Release 20240723 with 20240723-consensus

This should include all our changes for nitro v3.1.

- Check that the module root matches when downloading releases from
  github.

* Check all "not found" errors

Currently only the leveldb not found errors are handled. If a pebble not
found error is returned by the backend it is propagated, causing the
batchposter to fail to send transactions.

With this change all DB not found errors are handled the same way.

* Submit L2 Message to espresso network (#173)

* Submit L2 Message to espresso network

* add context timeout

* check for hotshot liveness and add a polling interval to see if transactions were included in an espresso block

* cleanup

* lint

* Draft the sovereign sequencer test

* Add espresso transaction queue

* Draft the espresso submission

* Set a new l2 message type for the sovereign sequencer transactions

* Attempt to write messages again

* Fix typos and add some comments

* Fix typo

* add header to jst and submit transaction to espresso

* Update go client for development

* Confirm the transaction inclusion in hotshot

* Fix the test

* Unblock the building blocks

* Update go client

* Fix the sovereign test

* add logs

* cleanup

* lint

* Updage go client

* not to fetch merkle proof if hotshot height is 0

* Cleanup

* Fix CI

* Add CI skip tests

* remove unwanted variable

* skipping some tests to make ci pass

* skipping some tests to make ci pass

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Persistent storage for sovereign builder (#180)

* Persistence storage for sovereign builder

* add unit test

* add unit test to ci and add mutex

* Optional header in jst

* initialize pending transactions if not present

* remove unwanted test

* resolve bugs

* remove nil pointer panic

* cleanup

* revert to strings.contains

* fix isErrNotFound

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Save digest artifacts into own directory per run (#182)

* Save digest artifacts into own directory per run

Currently it seems that we have too many digests in the folder and I
think this breaks the assembly of the multiarch docker image.

With this change only digests for the same run are saved in the same
directory.

* Fail docker image merge on wrong number of digests
* Clean digests dir

* Add dependabot rule to update nitro-testnode (#189)

* Fix build (#191)

* Add missing dependency
* Use renamed variable, fix typo

* dependabot: update contracts submodule (#193)

* Pin the golint version in CI

* Fix pebble panic (#192)

* Fix pebble panic

* lint

* Update some outdated help on flags (#203)

* Improve help texts for flags

* Update e2e tests to use Sovereign sequencer (#204)

* Update e2e tests to use Sovereign sequencer


Co-authored-by: ImJeremyHe <[email protected]>

* Remove the global state for HotShot height (#210)

* Remove the global state

* Update the submodule

* CI: use buildjet instead of self-hosted runner (#212)

* CI: use buildjet instead of self-hosted runner

* CI: use public runner / buildjet for docker build

This removes usage of our self-hosted github runners.

* CI: more disk space on public runner

* Check disk space at end of runs

* Try running CI tests from different directory

This may avoid failures due to IPC paths being too long

* Make workspace dir a symlink to /tmp/gha-work

* Fix redis URL

* Move redis tests to own workflow

* CI: fix rust and go cache (#217)

* rust-cache: add more target dirs
* Don't set custom TMPDIR

This seems to be the culprit for the extremely long path names that
are too long for IPC paths.

* modify arbosstate.go to allow for upgrade to arbOS version 35 (#216)

* modify arbosstate.go to allow for upgrade to arbOS version 35

* Fix lint

* Modify the chain config during the ArbOS upgrade

---------

Co-authored-by: sveitser <[email protected]>

* CI: make more space (#218)

* Store transaction pos in batch poster (#219)

* Move submit transaction pos from tx_streamer to batch poster

* Remove espresso sequencer (#214)

* Remove espresso sequencer

* lint

* remove enable espresso

* commit go-ethereum

* fix solgen/gen.go

* fix build

* revert .gitmodules

* revert go.mod

* add enable espresso to validation check

* re-add enable espresso to tests

---------

Co-authored-by: Mathis <[email protected]>

* Update espresso-sequencer-go with new LC ABI (#230)

* Update espresso-sequencer-go with new LC ABI

Instead of using the block height from the first snapshot after the
height in the justification it uses the latest height in the LC
contract. Currently not 100% convinced about correctness but also
believe it should work because any newer tree still contains the header
of interest.

* Undo LC ABI changes

We're not comfortable with the previous change because making a proof
w.r.t to the newest finalized state may cause issues if the batch poster
is lagging behind. This is currently not easy to test so we decided to
revert back to the old flow.

* Use tag of espresso-sequencer-go

* Stop setting the chain config during ArbOS upgrade in favor of setting it manually during migration process. (#234)

* Use the tag before marketplace update (#238)

* Sign payload for sovereign builder (#181)

* Sign payload for sovereign builder

* save signed payload to db

* change parsing logic to include signature

* add some comments

* skip tests

* address comments

* add signature payload test

* test working

* fix logging and comments

* move placement of log

* fix ci

* fix ci

* only submit transaction if jst nil

* add log to debug ci

* use the latest instead of main for espresso-dev-node

* pin to legacy lc version

* pin to before marketplace version

* remove log

* Full node with espresso finality (#237)

* wip

* wip

* Add espresso finality node test

* fix ci

* fix ci

* pin to before marketplace version

* fix tests

* add comments

* address comments

* address comments

* address comments

* cleanup

* fix test

* fmt

* rerun ci

* ci skip tests

* Update to sequencer.go to remove `EnableEspressoSovereign` flag, and instead sequence based on hotshot liveness at l1 block height.

* Update node.go to remove reference to `EnableEspressoSovereign`

* update Espresso tests to reflect removed config variables, and add escape hatch test to e2e test.

* Add wait for hotshot liveness in sovereign sequencer test to ensure sovereign sequencer is enabled.

* go fmt

* Add comment to explain changes needed in replay binary when panics are no longer an issue.

* Fix lint issues

* Modify construction of lightClientReader in sequencer.go to allow for nil `l1Reader`

* Update test configs to align with changes.

* Update comments according to feedback.

* Bump nitro-testnode from `03095e0` to `884a63c` (#240)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `03095e0` to `884a63c`.
- [Commits](EspressoSystems/nitro-testnode@03095e0...884a63c)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mathis <[email protected]>

* Revert removal of `EnableEspressoSovereign`

* Add additional check to ensure that if EnableEspressoSovereign is on we can't construct a sequencer with a nil lightClientReader

* Add `EnableEspressoSovereign` config flag to test configs.

* Bump contracts from `d226e9d` to `2ae1bbd` (#244)

Bumps [contracts](https://github.com/EspressoSystems/nitro-contracts) from `d226e9d` to `2ae1bbd`.
- [Release notes](https://github.com/EspressoSystems/nitro-contracts/releases)
- [Commits](EspressoSystems/nitro-contracts@d226e9d...2ae1bbd)

---
updated-dependencies:
- dependency-name: contracts
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Pin rust to 1.81.0

See #257

* Set cargo config for espresso_crypto wasm build

This was recently added for the other wasm compilation steps.

* Pin rust in E2E test

* CI: remove deprecated rust toolchain action

* Fix if statement

* Regenerate data-poster external-signer test certs

(cherry picked from commit 8f961b0)

* Use ArbOS chain config as the cannonial chain config. (#252)

* Add checks to sequencer.go and transaction_streamer.go to utilize the chain config switch

* Update interface.go to export interface for `GetArbOSConfigAtHeight()`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionEngine`

* Add implementation of `GetArbOSConfigAtHeight()` for `ExecutionNode`

* Update `sequencer.go` and `transaction_streamer.go` to read from the `ArbOS` `chainConfig` as the cannonical `chainConfig` rather than the `chainConfig` held by the `Blockchain` struct.

* Add ArbOS chain config check in batch poster to stop adding espresso messages to the transaction streamers DB if EnableEspresso is false.

* Add missing error check in sequencer.go and nil check on arbos config in sequencer.go and batch_poster.go.

* Remove ineffectual assignment in sequencer.go

* Add test for new GetArbOSConfigAtHeight() method in ExecutionNode

This commit adds an integration test for the GetArbOSConfigAtHeight()
method on the Execution node that grabs the initial config, and
validates it, then updates the config and ensures that the new method
will fetch the updated config.

* restore nitro-testnode to proper commit

* Fix trie node node found error during recording

I'm not exactly clear why we need to do this or if it's indeed the right
approach to solving the issue however it does seem to make it possible
to validate again.

Without it we get errors like

    INFO [10-30|12:24:29.127] DelayedSequencer: Sequenced              msgnum=1 startpos=1
    INFO [10-30|12:24:30.143] Reading message result remotely.         count=3
    WARN [10-30|12:24:32.143] prepareblocks failed to get state for block hdrNum=0 err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found"
    ERROR[10-30|12:24:32.143] Error while recording                    err="moved beyond genesis looking for state 0, genesis 0: missing trie node 147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 (path ) state 0x147a5a93c0202b07e7d7e3c936a36a8d97913056045be3af23b95b0aa0b356c7 is not available, not found" status=2

* Fix the OSP test

- Enable 4844 blobs to avoid contract deployment error
- Fix address of OSP entry

* Disable batch poster in Espresso finality node

We don't need this type of node to have a batch poster. Enabling it now
causes the test to fail because there's a check to ensure that redis is
used for the batch poster on the second node.

* Re-allow setting L1 stack config for tests

After the sync the code would always use the default values.

Don't think this is very clean but I couldn't think of something better
yet.

* CI: increase parallelism

* Fix linting

- Run `golang-lint run --fix`
- Check error after `waitForConfigUpdate`

* Move finality node test to espresso tests

The race test detected a data race in

https://github.com/EspressoSystems/nitro-espresso-integration/actions/runs/11705919237/job/32601803846?pr=254

* Update contracts submodule

* Fix typo

* Add wabt and cmake in wasm shell

* Skip tests

* Add tags in ci

* Add ci skip tests

* Add ci skip tests

* Remove rerun-fails

This flag in gotestsum causes some bugs which multiple the test time.

gotestyourself/gotestsum#417
gotestyourself/gotestsum#423

* Revert "Remove rerun-fails"

This reverts commit 6bef1c3.

* CI: simplify

- Add scripts to debug more easily locally.
- Skip tests by substring match instead of exact match.
- Don't run coverage on CI, we never look at it.
- Move all Espresso tests to Espresso CI job.

* Fix build with nix on macos (#296)

* CI: install latest gotestsum

This enables the --hide-summary argument

* Update submodule forks with merge commits

* Bump nitro-testnode from `07a25f1` to `77d1447` (#297)

Bumps [nitro-testnode](https://github.com/EspressoSystems/nitro-testnode) from `07a25f1` to `77d1447`.
- [Commits](EspressoSystems/nitro-testnode@07a25f1...77d1447)

---
updated-dependencies:
- dependency-name: nitro-testnode
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add FFI bindings for Espresso validation (#294)

* Example call to espresso-crypto function stub in batch poster.

This commit includes a log line and example call to the espresso-crypto
function stubs in batch_poster.go

* Result of `make fmt`

Run `make fmt` to clean up formatting not apparent in editor

* Make current lib.rs c abi compatible

* result of make fmt

* Add C ABI ffi between rust crypto lib and espressocrypto go module.

This commit adds FFI support between exsisting Rust crypto helper code
and the exsisting espressocrypto go module in native.go These functions
will now serve as utility functions to use in the integration that will
handle ffi to the Rust code.

This commit also moves the espresso-crypto-helper code to the
espressocrypto directory as we no longer need it for the arbitrator.

* Build native.go successfully

* Update the espresso-crypto and add test for merkle proof verification

* update flake.nix to new go version and fix errors

* Current state of work

The current state of my work revolves around editing the Makefile to
include the new espressocrypto ffi functions in the build process.
Additionally, this commit restores some espressocrypto code to the
arbitrator in an attempt to get a docker image built via the test node.
I felt as though this would be easier for this PR as opposed to ripping
out all of the code from the arbitrator.

* Update the flake.lock

* Add the namespace proof test

* Current state of build system changes

This commit represents the current state of the build system after
testing today.
Currently things compile locally, but not when building an image from
the docker file. There are errors related to duplicate function
definitions from what seem to be the FFI lib that stylus exposes.

* Build nitro docker image sucessfully

- Modify makefile to have a file target. Since the build artifact of our
  static library is a file we should use this as the make target
  (instead of only having PHONY targets).

- Use rust 1.81 to build all shared rust libraries in docker.

Our crate requires a minimum version of 1.81. However the other
libraries like libstylus.a are currently compiled with 1.75 and this
seems to cause issues with duplicate symbol definitions during linking.

For example:

    8.718 go build -ldflags="-extldflags '-ldl' -X github.com/offchainlabs/nitro/cmd/util/confighelpers.version= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.datetime= -X github.com/offchainlabs/nitro/cmd/util/confighelpers.modified=" -o target/bin/nitro "/workspace/cmd/nitro"
    42.74 # github.com/offchainlabs/nitro/cmd/nitro
    42.74 /usr/local/go/pkg/tool/linux_amd64/link: running clang failed: exit status 1
    42.74 /usr/bin/ld: /workspace/espressocrypto/lib/espresso-crypto-helper/target/release/libespresso_crypto_helper.a(std-1c4b19562077c20d.std.1f2242ed6435445e-cgu.0.rcgu.o): in function `__rdl_alloc':
    42.74 /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/alloc.rs:397: multiple definition of `__rdl_alloc'; /workspace/execution/gethexec/../../target/lib/libstylus.a(std-90f6ddbf82de36ec.std.409886f6357001f0-cgu.0.rcgu.o):/rustc/82e1608dfa6e0b5569232559e3d385fea5a93112/library/std/src/alloc.rs:389: first defined here

Bump the rust version of the prover-header-builder to 1.81 and compile
the espresso crypto library in the same container.

* Fix LDFLAGS, remove unnecessary CFLAGS

* Fix some merge errors

- We need rust 1.81 to compile the espresso crypto lib
- Re-add makefile rules

* Fix merge errors

* Remove FFI call in batch poster to expidite merge.

* Update bf submodule in arbitrator to match upstream version (This was set to a different commit during my erroneous merge)

* Remove extra change in espressocrypto_wasm.go

* Fix error with temporary espressocryptowasm directory preventing CI from running

* fix error of missing function definitions in espressocryptowasm directory to allow CI to run

* Re-add nosec comment about gosec error that will be removed soon

* result of make fmt

---------

Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: ImJeremyHe <[email protected]>

* Update the go client (#256)

Still using the legacy dev node

* ci: fix `git ls-remote` error in docker build (#304)

* Use path context instead of git context

With the default git context the repository is re-cloned at the start of
the docker build and the .dockerignore file is ignored.

> Be careful because any file mutation in the steps that precede the
> build step will be ignored, including processing of the .dockerignore
> file since the context is based on the Git reference.

https://github.com/docker/build-push-action?tab=readme-ov-file#git-context

This leads to some `.git` worktree files showing up inside the
dockerbuild with triggers a bug in yarn.

See yarnpkg/yarn#7537 (comment)
for more details about the yarn bug.

* Use the latest espresso dev node (#303)

* Use the latest espresso dev node

- Fix an issue that may cause the validation reorg
- Waiting for L1 to be ready can avoid some potential bugs in dev node

* Add proof validation to batch poster (#301)

* Add espresso proof validation to batch poster

Adds calls to the espresso validation code to the batch poster to
validate the namespace proof and merkle proof in a TEE

* result of make fmt

* Fix lint errors

* Add better logging

* Fix batch poster after merge of upstream/integration

* Remove espresso replay binary modifications (#307)

* Remove espresso validation in the replay binary

* Remove espresso-crypto related dependencies in arbitrator

* remove espressocryptowasm

* Update Makefile and Dockerfile to remove references to unused espresso crypto wasm lib

* remove batch_poster.go.orig file

* Fixes from make fmt

* Remove espresso_ops_test

* Remove additional code related to espresso validation in the staker

* Remove changes on stateless block validator

* Undo the stopwaiter

Previous modifications on this was because we had to stop and start the seuquencers
to function the escape hatch. Right now we don't need that anymore. These changes are
from the upstream

* Rename test flag

---------

Co-authored-by: ImJeremyHe <[email protected]>

* Add attestation quote in batch poster (#306)

* Add attestation quote in batch poster

* write file

* update batch poster

* use read file

* update to userDataHash

* reduce file permissions

* nit

* Submit multiple messages in one transaction (#308)

* Submit multiple messages in one transaction

* Remove the espresso_transaction_payload_signature_test.go

* Update go client

* Remove signature and data signer

* Remove arbitrator/espresso-crypto-helper

* Add size limit to hotshot transaction

---------

Co-authored-by: Sneh Koul <[email protected]>

* Use espresso-types instead of hard coded files

* WIP: locally failed

* Use upstream contracts submodule (#310)

We no longer need the customizations in nitro-contracts (except
potentially some deployment tools) so this commit resets it to the
before the celestia contracts were added

celestiaorg/nitro-contracts@4f8c65c

and makes the nitro code work with that version by removing the hotshot
/ light client address passed to the OSP.

* Remove the legacy code and fix the batch poster (#318)

* Remove the legacy code and fix the test

* fix nix build on linux

- flake: add openssl, pkg-config
- rust: update espresso-types to latest to not require testing features

* nix: fix build on darwin

- Update nixpkgs
- Set NIX_LDFLAGS to find missing symbols
- Leave note about updating flake since the apple SDKs in nixpkgs have
  been changed substantially.

* Transmute nstable

* add celestia-integration branch to the ci

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: nomaxg <[email protected]>
Co-authored-by: sveitser <[email protected]>
Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sneh Koul <[email protected]>

* Remove the oudated WASM binary (#329)

* Add escape hatch (#330)

* Clean flags (#335)

* Checkout submodules recursively

* Remove unnecessary flags in tx-streamer and sequencer

* Remove redundant code (#340)

* fix docker build (#338)

* docker: install openssl, pkg-config

Needed to compile openssl-sys crate.

* ci: run amd64 docker build on PRs

* Revert "ci: run amd64 docker build on PRs"

This reverts commit 0974e29.

* Avoid ARM build on PRs

GitHub actions makes this really complicated.

* ci: exclude: needs a list

* symbol -> string

* Fix conditional

* Fix PR trigger

* Update contracts submodule

* Run yarn and forge install

* Run yarn and forge install for espresso e2e as well

* Run yarn and forge install for espresso e2e as well

* fix e2e

* fix tests

* cleanup

* comment out multiplatform build

* fix tests

* ci: skip multiplatform build on PRs

* ci: fix indentation

* ci: skip wavm extraction on PRs

We don't push the image so we can't extract the wavm root.

* ci: remove wavm machine release

We are not planning to have an STF that differs from upstream. So this
shouldn't be needed anymore.

---------

Co-authored-by: Sneh Koul <[email protected]>

* Use verifier address (#341)

* WIP: use geth submodule

* replace EnableEspresso flag into verifier address

* Update geth submodule

* Nit

* Increase the poll interval

* Fix arbos test

* DelayThreshold

* Stop processing if no verifier address in chain config

* chores

* Lint

* Fix tests

* Fix

* Clean up (#351)

* Clean up

* Remove changes to parse_l2.go

* Clean up

* remove attestation log

---------

Co-authored-by: Sneh Koul <[email protected]>

* Fix logs for when espresso should not be enabled (#357)

* Fix logs for when espresso should not be enabled

* fix transacion_streamer

* Add dockerfiles and other files needed to decode attestation quote (#358)

* Update contract submodule

* Update espresso dev node docker file

* Remove unnecessary directory

* Remove unnecessary changes

* skip failing test

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: nomaxg <[email protected]>
Co-authored-by: sveitser <[email protected]>
Co-authored-by: Zach Showalter <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sneh Koul <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Recreate Dockerfile based on upstream Dockerfile (and make sure it works)
4 participants