Shanghai Planning

[timbeiko]

Sept 19, 2022 Update

We discussed this issue on #616 and agreed to move forward with the process detailed here while ACD/CL calls are paused.

---

Original Issue Text

Using this issue to keep track about conversations related to Shanghai planning.

First discussed on ACD129, prompted by this comment:

Can we please discuss on what we want to concentrate for Shanghai? There are quite a few potential items, and it's unlikely that we'll manage to do all of them, so we'll have to pick which to work on first. I see the following candidates:

• EIP-1153: Transient storage opcodes
• EIP-2537: Precompile for BLS12-381 curve operations
• EIP-2935: Save historical block hashes in state
• EIP-3651: Warm COINBASE
• EIP-3540: EVM Object Format (EOF) v1 + EIP-3670, EIP-3690, EIP-4200
• EIP-3855: PUSH0 instruction
• EIP-3860: Limit and meter initcode
• EIP-4396: Time-Aware Base Fee Calculation
• EIP-4444: Bound Historical Data in Execution Clients
• EIP-4488: Transaction calldata gas cost reduction with total calldata limit
• Statelessness gas cost changes (Verkle Tree HF1) and the groundwork necessary for it (unhashed state + sync)
• Post-Merge Beacon ETH withdrawals

To clarify, the point is not to go through each and every of these proposals in detail during this call, but rather to get a feel of what we want to prioritize for Shanghai.

[timbeiko]

Yes! For future reference, the most up to date place for potential EIPs is https://github.com/ethereum/pm/issues. The comment quoted above will be out of date fairly quickly as we start planning Shanghai.

[holiman]

EIP-3540: EVM Object Format (EOF) v1 + EIP-3670, EIP-3690, EIP-4200

Is the proposal to do all four of them in one fork?

[gcolvin]

It's not necessary to do them all in one fork, @holiman , depending on how crowded the forks get and how long between forks.

There was talk last year of having one fork devoted to the various EVM improvements which have been six years in the making, and all-in-all form a complete, safe and efficient control-flow facility. They make for a big meal, but no bigger than EIP-615 was. A reasonable order of inclusion would be as follows.

These are essential for most of the remainder.

• EIP-3540: EVM Object Format (EOF)
• EIP-3670: EOF - Code Validation
• EIP-3860: Limit and meter initcode

These provide static jumps and subroutines. EIP-4200 does not depend on EIP-2315.

• EIP-4200: Static relative jumps
• EIP-2315: Simple Subroutines for the EVM

This specifies additional validation to constrain JUMP, JUMPI , static jumps and subroutines -- thus providing a higher level of safety than promised by EIP-615. It depends on the object format, validation, static jumps, and subroutines.

• EIP-3779: Safer Control Flow for the EVM

This is in early design stage. It provides multiple code sections, procedures with defined interfaces, and a frame stack for disciplined use of memory.

• EIP-4573: Entry Points and Procedures for EVM Code Sections

And these are independent of the others, and can be included independently.

• EIP-3855: PUSH0 instruction
• EIP-1153: Transient storage opcodes

[timbeiko]

In ACD131, we discussed a large number of proposals for Shanghai and asked client teams for their general priorities.

EIPs 3540, 3670 and 3860 were considered high priority and moved to CFI for Shanghai (PR). Beyond these, any change that could benefit Ethereum's scalability was considered high priority. The main options now are either the introduction of blob-carrying transactions or, if too complex, EIP-4488-style CALLDATA cost reduction. Additionally, an EIP to specify Beacon Chain withdrawals is expected soon.

[timbeiko]

On ACD133, we agreed to move EIP-3651 to CFI for Shanghai. Commit: ethereum/execution-specs@ded8ebe

[moodysalem]

I'd like to discuss specifically moving EIP-1153 to CFI in ACD 135 for the reasons:

• No interactions with the other EIPs
• It cannot be properly supported via changes to SSTORE/SLOAD and refunds, because the work of loading an empty slot cannot be avoided
• It enables more efficient smart contract patterns that are currently infeasible due to gas
  • The earlier it's available, the more developers can prepare for it, the faster transient uses of regular storage can be deprecated, simplifying the EVM
• Many protocol developers have expressed they would like to see it implemented: z0age from Opensea, fredlacs from Arbitrum, brockjelmore from Nascent, frangio from OpenZeppelin, and @tynes from Optimism built the proof of concept at ETHDenver
• It factors into the design considerations of long term EVM changes, e.g. should Verkle trie storage have refunds for transient storage? How should reentrancy locks be handled for parallelization?
• It is relatively small and easy to implement, there is already a WIP reference implementation in the EthereumJS VM

[holiman]

To save some time, here are my thoughts around EIP-1153

• It's a nice feature, of course we'd want it
• However, it's been a backburner-nice-to-have since 2018, the reason (I think) being that it doesn't revolutionize anything, just makes certain usecases easier/cheaper/(safer?).
• And it is quite nontrivial to implement,
• And also needs a lot of efforts to fully test
• The implementation must also be tested from a DoS perspective (a quick napkin calculation tells me that with 30M gas, up to 9MB of data can be stored, which is probably fine, but there are journalling to take into consideration too -- this would entail a journal with 300K items to roll back)
• Unfortunately, we have a lot of things competing for inclusion in Shanghai. In "competition" with those, I would not pick 1153 to be selected.
• Also, I'd be curious to hear what @AlexeyAkhunov thinks about it. Still want it?

[timbeiko]

After ACD135, client teams agreed to hold off considering new CFI EIPs until we are farther along in the merge work, see here

[protolambda]

Also EIP-4844 (Shard Blob Transactions)

[phpmac]

When will you upgrade in Shanghai? When can eth be extracted?

[solomontan88]

When will you upgrade in Shanghai? When can eth be extracted?

What's do you mean extracted ?, withdraw ?

[timbeiko]

See my thoughts about async planning here https://ethereum-magicians.org/t/shanghai-core-eip-consideration/10777

[lightclient]

Also EIP-3074

[solomontan88]

Is Shanghai upgrades includes "Surge" / "Sharding" upgrades ? or between Merge and Surge ? @timbeiko

[MostBolone]

When will you upgrade in Shanghai? When can eth be extracted?

What's do you mean extracted ?, withdraw ?

@solomontan88 - I believe he means staked ETH.
Staked ETH looks not to be unstake-able until the Shanghai Upgrade. What is the ETA for the Shanghai update?

Thanks

[holgerd77]

Staked ETH looks not to be unstake-able until the Shanghai Upgrade. What is the ETA for the Shanghai update?

Shanghai is in the very early stages of planning with the scope of the changes getting into the HF still to be discussed, see the Magicians thread linked by Tim above, so there is no timeline on this yet.

I guess it's safe to say that the Shanghai hardfork won't happen this year any more, also that it is somewhat likely that it will happen sometime next year.

[protolambda]

I'd like to discuss moving EIP-4844 to CFI in ACD 150 (Nov 24) for the reasons:

• No interactions with Shanghai EVM EIPs: the EIP adds one opcode to push a datahash onto the stack, and a precompile to verify data against the commitment.
• Compatibility with Withdrawals: the EL and CL specs are based on top of the withdrawal work (EL, CL, Engine API)
• EIP-4844 Spec completion:
  • We have iterated through EIP changes for 9 months, and the scope of changes reduced significantly over time
  • The EIP received review from most client implementer teams
  • Complementary CL specs and Engine API specs
• EIP-4844 Testing:
  • There is an active interop repository
  • Devnets of increasing size and client participation (see devnet section)
  • Ongoing work to analyze network impact, but conservatively reduced back the blobs parameter for initial iteration.
• EIP-4844 KZG-ceremony readiness:
  • Specs ready
  • Front-ends ready:
    • zkparty, and hosted here
    • chotto
    • worldcoin (WIP)
  • Sequencer ready, and audited
• EIP-4844 KZG library readiness:
  • The CL specs define a simple minimal interface, as well as the EIP
  • C-KZG-4844 implements this interface, with NodeJS, CSharp and Python bindings. And ongoing Rust bindings.
  • Go-KZG implements this interface in Go
• EIP-4844 Devnet readiness:
  • Wide client devnet support:
    • EL: Erigon, Geth, Nethermind
    • CL: Prysm, Lighthouse, Lodestar
  • Targetting devnet v3 soon after ACD, on Nov. 30

[timbeiko]

On ACD150, we moved the following EIPs to CFI:

• EIP-2537: Precompile for BLS12-381 curve operations
• EIP-4200: EOF - Static relative jumps
• EIP-4750: EOF - Functions
• EIP-4844: Shard Blob Transactions
• EIP-5450: EOF - Stack Validation

PR: ethereum/execution-specs#656

On ACD149, we moved 1153 to CFI.

[timbeiko]

On ACD151, the scope for Shanghai was finalized, see: ethereum/execution-specs#663

[gaozhenyusky]

just expect!

[timbeiko]

Update: Shanghai will go live on Sepolia on Tuesday, February 28, 2023 4:04:48 AM UTC

[timbeiko]

Given we are now moving to testnets, I'll close this planning-related issue.