Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build blocks using Reth 1.1.0 #115

Open
wants to merge 91 commits into
base: main
Choose a base branch
from
Open

Build blocks using Reth 1.1.0 #115

wants to merge 91 commits into from

Conversation

hashcashier
Copy link
Contributor

@hashcashier hashcashier commented Oct 29, 2024

This PR migrates zeth from directly using revm to using reth for building blocks. This allows zeth to support all forks supported by reth. The changes are an overhaul of many of zeth's internal components and project structure. Notably, because of Cargo's feature unification, and reth's dependence on feature flags for enabling/disabling OP functionality, two separate zeth binaries are now necessary.

The project has been restructured into the following crates:

  • zeth-core Library for defining and using the sparse MPT to statelessly build a block using reth.
  • zeth-core-ethereum Specialization of the core library for Ethereum blocks.
  • zeth-core-optimism Specialization of the core library for Optimism blocks.
  • zeth-preflight Library for populating the data required to statelessly build a block using zeth-core.
  • zeth-preflight-ethereum Specialization of the preflight library for Ethereum RPCs.
  • zeth-preflight-optimism Specialization of the preflight library for Optimism RPCs.
  • zeth-guests-reth-ethereum A binary for running the stateless block building process for Ethereum blocks.
  • zeth-guests-reth-optimism A binary for running the stateless block building process for Optimism blocks.
  • zeth-guests Library for exporting the compiled rv32im bytecode of zeth-guests-reth.
  • zeth A library for running a CLI to use the RISC Zero zkVM to generate/verify proofs for the stateless block building process.
  • zeth-ethereum A binary using the zeth crate for Ethereum blocks.
  • zeth-optimism A binary using the zeth crate for Optimism blocks.
  • zeth-benchmark A binary for benchmarking proving performance for randomly sampled blocks.

Functionality changes:

  • All ethereum types have been migrated to alloy types.
  • All Bonsai and profiling logic has been delegated to r0vm's environment variables.
  • Optimism consensus verification (block derivation) logic has been removed.
  • Ethereum foundation testing is unfinished.

Note: v1.81 of the toolchain is required:

rzup update

Note: Separate binaries must be built individually

cargo build -p zeth-ethereum --bin zeth-ethereum
cargo build -p zeth-optimism --bin zeth-optimism

Issues affected:

@pdg744
Copy link

pdg744 commented Nov 7, 2024

@hashcashier @SchmErik iiuc v1.81 toolchain should now be ready to use!

@@ -95,6 +95,7 @@ impl<DB: DatabaseRef + Recoverable> DatabaseRef for Wrapper<DB> {

impl<DB: DatabaseCommit + Recoverable> DatabaseCommit for Wrapper<DB> {
fn commit(&mut self, changes: HashMap<Address, Account>) {
dbg!("COMMIT!");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that should be removed.

// resolve storage orphans
if let Some((trie, _)) = storage_tries.get_mut(&account_proof.address) {
for storage_proof in account_proof.storage_proof {
// let proof_nodes = parse_proof(&storage_proof.proof)?;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is still quite a bit of commented code, this should be removed

@@ -151,6 +153,48 @@ impl<N: Network, R: CoreDriver, P: PreflightDriver<R, N>> PreflightDB<N, R, P> {
apply_changeset(&mut self.inner, state_changeset)
}

pub fn sanity_check(&mut self, state_changeset: StateChangeset) -> anyhow::Result<()> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused function

mpt_from_proof(&proof_nodes).context("invalid storage_proof")?;
let proof_nodes = parse_proof(&storage_proof.proof)
.context("extend_proof_tries/parse storage proof")?;
mpt_from_proof(&proof_nodes).context(format!(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be a with_context

Ok(vec.into_iter().collect())
}
}

pub mod opt_ordered_map {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is never used, ordered_map + default is used instead


fn chain_spec(chain: &NamedChain) -> Option<Arc<Self::ChainSpec>> {
match chain {
NamedChain::Mainnet => Some(MAINNET.clone()),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do not support all possible forks. I know for a fact that the pre Byzantine validation is not correct in Reth: Back then, the TxReceipt still contained intermediate state hashes which was left out in Reth for performance reasons https://github.com/paradigmxyz/reth/blob/e3702cfc87449294da061f02a571a23061b8ab50/crates/ethereum/consensus/src/validation.rs#L31
So we must exclude at least everything before Byzantine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about possible future forks that are (or will be with newer reth versions) only partially supported? Do they also need to be explicitly excluded?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the submodule was removed to prevent Cargo from always checking out the huge tests.
However, then the ef-tests should be adapted to include a script or documentation that checks out the EF tests in the correct version

@@ -1,6 +1,10 @@
[workspace]
resolver = "2"
members = ["guests", "host", "lib", "primitives", "testing/ef-tests"]
members = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ef-tests have been removed here. Thus, they have not been adapted to the new reth version and will not compile.

- uses: risc0/cargo-install@v1
with:
crate: cargo-binstall
- run: cargo binstall -y --force cargo-risczero@$RISC0_VERSION
- run: cargo risczero install --version $RISC0_TOOLCHAIN_VERSION
- run: cargo test --workspace --all-targets -F ef-tests,debug-guest-build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should absolutely not disable the ef-tests. Even when we are now using reth they contain crucial testcases that we must also pass.

}

#[derive(Default)]
pub struct Wrapper<T: Recoverable> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An alternative to "abusing" the drop to get the Database back is to move the DB in and out similar to https://github.com/risc0/risc0-ethereum/blob/73eebd4efb283c7f99748d45fc730a53ee2edf5c/steel/src/contract.rs#L211
However, this should probably not also be in this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants