ECIP-1045: Support for ETH Byzantium & Constantinople EVM and Protocol Upgrades

[whilei]

    ECIP: 1045/undecided
    Title: Support for ETH Byzantium & Constantinople EVM and Protocol Upgrades
    Status: Draft
    Type: Standard Track
    Author: Isaac Ardis <[email protected]>
    Created: 2018-06-18

Rendered

Abstract

Add support for a subset of protocol-impacting changes introduced in the Ethereum Foundation (ETH) network via the _Byzantium_ hardfork. The proposed changes include:

• Byzantium EVM opcodes and precompiled contracts, namely opcodes REVERT (EIP 206/140), RETURNDATASIZE (EIP 211), RETURNDATACOPY (EIP 211), and STATICCALL (EIP 214/116); and precompiled contracts for modular exponentiation, elliptic curve addition, scalar multiplication, and pairing (EIPs 198, 212/197, 213/196)
• Expected Constantinople EVM opcodes, namely bitwise shifting operators SHL, SHR, and SAR (EIP 215); CREATE2 (EIP 1014); and EXTCODEHASH (EIP 1052).
  • TODO: Research and discuss gas metering changes for SSTORE, proposed in EIP 1283.
• Replacing the intermediate state root field in transaction receipts with the contract return status (EIP 658).
  This document proposes block 7,100,000 as the upcoming block height at which to implement these changes in the network, placing the expected date of protocol hardfork in January 2019.

For more information on the opcodes and their respective EIPs and implementations, please see the History section of this document.

Motivation

To enable and maintain interoperability between Foundation and Classic Ethereum Virtual Machines ("EVM"s) and protocol.

[sorpaas]

@whilei I still don't understand why we don't include EIP-658. By still keeping PostState in transaction receipt, it damages parallel transaction execution (while parallel execution itself is still possible, some certain types of optimizations wouldn't be possible any more). What's more, clients that support both ETH and ETC (like parity-ethereum and multi-geth) might have some difficulties to keep RPC definitions compatible with geth-etc (and vice versa).

I would suggest we give ECIP-1040 (#86) a look first, because one of ETC's strength is that it treats the immutability philosophy seriously. By arbitrarily adding new (while useful, but not critical) opcodes, we may lose this important trait. Consider the following scenario:

• Someone uses a customized programming language that compiles to EVM opcodes. In the past, 0x3d is undefined, so it evaluates to invalid and cause the contract to exit. So the person uses that as failure return. No problem. It works well up to now, until RETURNDATASIZE is introduced. Now, the previously-working contract is broken and is most likely completely not working.

This is one of the problems solved by ECIP-1040. It also has certain benefits if we introduce more VMs into the network, like WebAssembly.

If, however, we're not treating immutability philosophy so "absolute", then I don't see why not also include the newly added precompiled contracts with this. They add certain useful functions to the blockchain, and right now as it has been running well on ETH, the security risk is low.

Edit: I also want to emphasize, that EIP-658 itself is a hard fork (i.e. it changes behavior on blockchain execution level). I have tried to clarify this multiple times on geth-etc#431. For geth-eth there isn't a specific config flag for that, but right now you can test this on parity-ethereum: assign "eip658Transition" in chain spec to a block number. The chain won't sync pass that block.

[splix]

@sorpaas I think there is a misunderstanding of what the ECIP is proposing. It's not about hardfork itself, but about opcodes (which will cause hardfork). To my understanding EIP-658 doesn't change behaviour on blockchain execution level, but only on peer-to-peer communication and API level. EIP-658 can be included into same hardfork, but it's not necessary to have it in same ECIP.

[realcodywburns]

Solidity compilers >0.4.22 compile to byzantium by default. Any contract compiled on these is unable to make cross contract calls with out doing some really wonky comand line stuff. Users using remix or truffle for deployment are currently un able to make contracts properly. This should be a pretty high priority.

[sorpaas]

@realcodywburns I don't think that is a good strategy. Constantinople hard fork will land in a few months. After that EIP-145 will likely break Solidity compiling again. There're also changes like EIP-210 that can be controversial to be implemented on ETC (while what users perceive are vastly different, technically it's actually similar to EIP-779 and EIP-999, involving irregular state changes).

Like I said before, abruptly adding new opcodes technically also breaks existing contracts' assumptions. For ETH community this works well, because things are not so absolute there, but for ETC, I don't see how this can work without ECIP-1040 (#86). A previous account with code 0x3d is nobody-can-send, but after this ECIP, it becomes everybody-can-send. Doesn't it smell like theDAO?

I also want to emphasize again, that EIP-658 is a hard fork change because it changes behavior on blockchain execution level. The current implementation in geth-etc for getTransactionStatus is a non-hard-fork change (but it does have certain disadvantage given we are going to hard fork anyway), and we cannot call it EIP-658.

[realcodywburns]

Having developers unable to deploy contracts seems like a worse strategy. We either need to adopt these opcodes, or have a sane way for the nodes to handle the opcodes w/o breaking.

[sorpaas]

@realcodywburns Yeah sure. I understand that concern. What I'm saying is that even if you fix "developers unable to deploy contracts" right now through enabling Byzantium, it will be broken again just in a few months when Constantinople lands. I don't have any better solutions other than:

1. Follow any future EVM/WASM (eWASM and pWASM) changes with ETH community.
2. ETC community maintaining its own fork of Solidity compiler.

What I'm arguing before is that directly using (1) would seriously hurt the "immutability" philosophy of ETC (unlike theDAO, by introducing new opcodes, it's possible that we're "breaking" many existing smart contracts, where in contrast, EIP-779 only "broke" one). And I would urge the ETC community to review ECIP-1040 before adding new opcodes.

In the mean time, (2) might also be a viable strategy.

[realcodywburns]

Losing solidity support would be a HUGE problem. Truffle and remix have high user adoption and would be incompatible with solidity classic. I agree we need a better long term solution, but "developers unable to deploy contracts" is nontrivial for a blockchain that's value proposition is the ability to deploy contracts.

example of a contract that will not work with solidity >0.4.22. Existing is unable to execute any of its functions.

contract Deployed {
    uint public a = 123; 
    function setA(uint _a) public returns (uint256) {   a = _a ;  return a;}
    function a() public view returns (uint256) {return a;}
}
contract Existing  {
    Deployed dc;
    function Existing(address _t) public {dc = Deployed(_t);}
    function getA() public view returns (uint result) {return dc.a();}
    function setA(uint _val) public returns (uint result) { dc.setA(_val);  return _val;   }   
}

[sorpaas]

@realcodywburns So if I understand you correctly, what you're proposing is basically (1) "Follow any future EVM/WASM (eWASM and pWASM) changes with ETH community", right?

[sorpaas]

@realcodywburns The problem you're facing is known for some time already (#79 (comment)). Any contract that involves function calls is likely to be broken now if compiled using Solidity 0.22 or newer.

[sorpaas]

@realcodywburns I don't have more comments regarding (1). If we go with that route, then the best thing we can do is to conduct a combined fork with both Byzantium and Constantinople instead of only adding a few like in this ECIP.

The problem is that by the time you made the Byzantium hard fork coming through, Solidity might already have been upgraded to support new Constantinople opcodes. And if we only get Byzantium, there will be a new set of contracts that are broken for ETC again.

[sorpaas]

@splix I think this is more like a Meta type ECIP -- all specs are already available. The only issue is to decide what to be included. If we make it into too many multiple ECIPs, then it would be even harder for people to agree on anything.

[splix]

@sorpaas agree, it's meta type ECIP, but about opcodes/bytecode. Maybe it looks too meta and about everything, that wasn't original plan

We are working on some improvements to transaction receipt already, at https://github.com/ethereumproject/go-ethereum/pull/634/files, but of course it doesn't replace https://github.com/ethereum/EIPs/blob/master/EIPS/eip-658.md and we'll need to address this issue

[whilei]

So, just in summary, here's what's being considered (AFAIK) for possible inclusions or dependencies for this proposal to reach a revised draft or replacement:

• An associated ECIP, ECIP-1040: Generalized Account Versioning Scheme for Hard Fork, which proposes a versioning scheme for opcode changes such as the one proposed, and speaks to an important philosophical and technical context for this change in specific and changes like it in general. If a decision can be reached with that proposal, it would significantly and directly impact the implementation of this proposal.
• Constantinople opcodes, eg EXTCODEHASH, CREATE2, overflow checking, SSTORE gas adjustments, and others. As spoken to in the discussion above, if the intention of this proposal is compatibility with ETH, then it might make sense to include those codes and features as well.
• Precompiled contract EVM changes from Byzantium hardfork EIP 198, EIP 212 (197) and EIP 213 (196): Precompiled contracts for modular exponentiation; elliptic curve addition, scalar multiplication and pairing (#14959,https://github.com/ethereum/go-ethereum/releases/tag/v1.7.0 #14993). Whether or not to specify inclusion for these as well in the hardfork this ECIP would necessitate should be decided. Along with this, it's worth noting that there are similar changes related to precompiled contracts associated with the anticipated Constantinople fork as well.

[whilei]

@sorpaas In response to your thought about EIP-658 inclusion, I agree. I'm working now on a next revision to this ECIP to include that with this proposal as well.

[whilei]

Above commit has included EIP-658 in this document. While IMO @splix is right in thinking that it's not "granted" that it should be included in this proposal since it's scope could be limited to changes in EVM, it can also be framed (as currently) that the proposal deals with the introduction of a subset of desired changes introduced in the ETH Byzantium hardfork. This is, of course, up for discussion, but seems to me as though introducing these ideas a set, focused on Byzantium interoperability, gives a simpler and unified place for discussion.

[darcyreno]

After some discussion with the ETCDEV team we'd like to modify this proposal to occur at block 6.9M which should be Sat Nov 10 12:28:30 CST 2018 - 75 days from today.

Looking for feedback - The main question is if the Mantis and Parity teams can update their clients in this timeframe.

The reason we're nominating this earlier date is because waiting until block 7.1M just seems like too much of a delay.

[sorpaas]

@whilei Please add Constantinople hard fork EIPs as well, as discussed above with @realcodywburns. If you follow the AllCoreDevs, currently being considered are EIP-145, EIP-1014, EIP-1052 and EIP-1283. One that might be delayed is EIP-1218.

[sorpaas]

Along with this, it's worth noting that there are similar changes related to precompiled contracts associated with the anticipated Constantinople fork as well.

No, there're currently no changes to precompiled contracts AFAIK. We did discuss EIP-1109 but it looks like it hasn't reached agreements.

[pyskell]

@darcyreno Happy to help with miner/exchange outreach for the updates. Should have an easier time since we've got more contacts now.

[whilei]

@sorpaas A couple of questions:

1. Do you know if there's a firm block number set (or even in mind) for the Constantinople fork? It seems noteworthy that ETC might actually implement those EIPs before ETH. Your thoughts there?

2. Would @darcyreno 's proposed timelined (block 6.9million) be achievable for the Parity client to have these changes implemented and released sufficiently before that?

[sorpaas]

@whilei

1. It may be before or after Devcon3. (I can be wrong on this.) I can't recall it right now, but it's probably hidden somewhere in the AllCoreDevs meeting notes. I don't think it matters whether ETC implements those before or after ETH, other than stability concerns.
2. If we only talking about technical details, parity-ethereum should have no issues getting Constantinople or Byzantium changes on November, as those changes are mostly ready, and we just need more testings and reviews. I don't know about geth-etc and sputnikvm, though. That said, I would strongly encourage the next hard fork to be a combined one of Constantinople and Byzantium, because forking ETC takes time, and if we do it wrongly, it may miss the stated goal (ETH compatibility) really soon again.

[r8d8]

@sorpaas probably you meant Devcon4 (will take place in Prague from 30 October - 02 November)

[whilei]

I've just pushed a change including specification for EIPs 145, 1014, and 1052.

@sorpaas I've left out 1253 for now because

1. doing so doesn't sacrifice the original "pure" intention of ETH interoperability (since, while the gas prices charged may be impacted, operability of contracts is not impacted)
2. EIP-1253 proposes a significant decrease in gas price, which exposes the network to - some, AFAIK, unknown degree of - risk; namely, if not duly accurate, too-low gas prices would make the network susceptible to spam. Personally, I'll need to review the documentation around the proposal in more depth to better understand it's reasoning and conclusions. As far as I know, gas prices have historically been calculated IMO rather roughly, and that has lead to required network "protocol upgrades" revising gas prices to counter these miscalculations, eg. EIP 150.

It may be noted that introduction of the "untested" Constantinople opcodes can be subject to the same train of argument in (2.). AFAIK only the bitwise shifts EIP specifies gas prices. One might venture to guess that CREATE2 would cost the same as CREATE, and that EXTCODEHASH would cost less than EXTCODECOPY, but details are not clear yet.

Futher input, knowledge, and discussion on this very welcome.

[whilei]

Regarding a possible modification to the block number of the required fork -- I've left the proposed (and tentative) block number unchanged as yet, since, given new light of possibly-incoming Constantinople changes, I'd like to get input from the Mantis client and SputnikVM developers as well around the proposed features and timeframe.

[sorpaas]

@whilei

• If we worry about gas prices and would rather apply only a partial set of hard fork changes, then we should instead evaluate all EIPs individually. Practically, Byzantium precompiled contracts have suffered much more gas price criticisms compared with sstore net gas metering. And if this is the case, then we would need more time to negotiate across different teams, and we certainly cannot do it with the current proposed fork block number. Note that no matter what we leave out, it means a few usages that we have on ETH won't be possible on ETC, so it doesn't really archive the compatibility goal.
• If you have valid critics of sstore net gas metering (EIP-1283), please raise it right now (like today or this week!). Many teams (geth, harmony, parity, trinity) have mostly agreed on EIP-1283 to replace EIP-1087, and it's about to move forward with that. So if there's any more input, it needs to get out soon!
• All EIPs have mostly been finalized. If you can't find gas costs in the spec, look for it in the issue comments!

[sorpaas]

doing so doesn't sacrifice the original "pure" intention of ETH interoperability (since, while the gas prices charged may be impacted, operability of contracts is not impacted)

That's not true. Reentry locks or sub-contract error message passing are currently too expensive to be carried out on mainnet.

[darcyreno]

@sorpaas isn't "too expensive" subjective to the value of the job being done - It's separate from functional interoperability. We're hesitant to roll forward with compatibility changes at the same time as making a 10x gas cost decrease until we understand the gas implications clearly. At which point, we could roll out gas change that is appropriate for ETC.

[sorpaas]

@darcyreno The same logic applies to many other EIPs in this bundle, though:

• REVERT opcode: this is basically a 10x gas reduction for the original error handling method.
• Precompiled contracts: 100x (or more?) gas reduction for really specific crypto algorithms. RSA, for example, was do-able with pure EVM.

Right now it really seems that we have trouble reaching consensus. If this is the case, I think we may want to delay the hard fork number, and I would encourage everyone to consider the route of using ECIP-1040 again. In that way, at least we won't accidentally break existing contracts.

[sorpaas]

isn't "too expensive" subjective to the value of the job being done

If a contract requires too many of the net gas metering features, then it may exceed the block gas limit without it. In that case, the transaction just cannot be included in any blocks.

[pyskell]

Regarding ECIP-1040/new opcodes there needs to be some sort of versioning or we'll break guarantees about the EVM's operation. Even if not a single contract uses the opcode today it:

1. Signals to the broader development community that the execution of their contracts is not guaranteed.
2. Invalidates research on the EVM (ie. its formalization in k)

ETH already has what is being discussed implemented so it's an excellent place to look for real world impact of decisions.

[whilei]

About ECIP-1040 - above @sorpaas you said

I would suggest we give ECIP-1040 (#86) a look first

and I had thought that would mean "a look at implementing first," but it looks to me like we would be able to introduce these Byzantium+Constantinople changes via (eg. along with) ECIP-1040. Am I correct in that thinking? (And I apologize if I had missed your point earlier).

[darcyreno]

I'm just bumping this thread to keep this conversation alive and because it's become larger than the original Scope:

We're talking about deploying the following in an upcoming fork:

-- ETH Byzantium ( 4 new opcodes + the precompiled contract changes )
-- ETH Constantinople ( 3 Opcodes + Gas changes )
-- EICP-1040 ( Generalized Account Versioning )

After discussing internally with ETCDEV, this stuff all makes sense - But, the overall scope has grown.
We need to nail down a blockheight or date that we can deliver upon that includes ample time for client updates, testing and network adoption - But, that also doesn't delay this release for too long. I still think this needs to get done in Nov/Dec time-frame so that we can stop chasing compatibility for a little while.

Furthermore, we need to get some kind of feedback on what it would take to update the Parity and Mantis clients and who can contribute time to doing so.

If we can't get to consensus here quickly, Is there anyone who has strong opinions on this that will be attending the ETC Summit. If so, I propose we arrange an in person conversation with beers.

@whilei @pyskell @splix @realcodywburns @sorpaas Please mention anyone else that may have opinions or concerns or weigh in with any thoughts or concerns please.

[pyskell]

Definitely want to include IOHK personnel, @AlanVerbner @dionyziz (+ anyone else, Charles, Tom, Kevin; sorry don't have everyone's github names).

[dionyziz]

Definitely in favour of going forward with these upgrades.

[tzdybal]

I think this ECIP should also propose block number at which fork will be applied on morden testnet.

[pyskell]

Also wanted to mention that the best long term approach to opcode changes is to have solidity's compiler allow for targeting different EVMs with a command line option (and getting tools that depend on solidity to use said option).

This will avoid having to "keep up" with ETH mainline in the future.

[darcyreno]

Hi Everyone, We're still working through the actual development. But, in light of the ETH core developers decision to delay their deployment of Constantinople, we're delaying our release as well. Once they propose a new date/block-height, we'll propose some for ETC.

As a reminder we're doing this stuff :

-- ETH Byzantium ( 4 new opcodes + the precompiled contract changes )
-- ETH Constantinople ( 3 Opcodes + Gas changes )

We will propose block heights for Mainnet & Morden as well.

[oberondelafay]

Not having these opcodes is a blocking issue for pretty much every dapp I'd like to bring over. So consider this my +100 for this proposal.

[sorpaas]

Just want to give everyone this information that we've found a edge case scenario related to the new EXTCODEHASH opcode that can cause geth, aleth and parity to disagree on consensus execution. This edge case can only be triggered when an empty account is present.

• Because EIP-161 is applied on ETH, no empty account can be present during execution, so it's not an issue for the upcoming Constantinople hard fork.
• However, because EIP-161 is not enabled on ETC, it means that a contract execution can meet an empty account. Therefore, it means that applying EXTCODEHASH alone on ETC is not safe.

So a heads up to everyone -- please do not apply EIP-1052 without EIP-161!