The part of the protocol that actually handles internal state and computation is referred to as the Ethereum Virtual Machine (EVM). From a practical standpoint, the EVM can be thought of as a large decentralized computer containing millions of objects.
-
Virtual Machine (Virtualbox, QEMU, cloud computing)
-
Java VM
Virtual Machine technologies such as Virtualbox and QEMU/KVM differ from the EVM in that their purpose is to provide hypervisor functionality, or a software abstraction that handles system calls, task scheduling, and resource management between a guest OS and the underlying host OS and hardware.
Certain aspects of the Java VM (JVM) specification however, do contain similarities to the EVM. From a high-level overview, the JVM is designed to provide a runtime environment that is irrespective of the underlying host OS or hardware, enabling compatibility across a wide variety of systems. High level program languages such as Java or Scala that run on the JVM are compiled into the respective instruction set bytecode. This is comparable to compiling a Solidity source file to run on the EVM.
The EVM Machine Language is divided into specific instruction set groups, such as arithmetic operations, logical and comparison operations, control flow, system calls, stack operations, and memory operations. In addition to the typical bytecode operations, the EVM must also manage account information (i.e. address and balance), current gas price, and block information.
- Common Stack Operations
-
Opcode instructions for stack and memory management:
POP //Pop item off the stack PUSH //Push item on the stack MLOAD //Load item into memory MSTORE //Store item in memory JUMP //Alter the location of program counter (PC) PC //Program counter MSIZE //Active memory size GAS //Amount of available gas for transaction DUP //Stack item duplication SWAP //Stack item exchange operation
- Common System Operations
-
Opcode instructions for the system executing the program:
CREATE //Create a new account CALL //Instruction for message passing between accounts RETURN //Execution halt REVERT //Execution halt, reverting state changes SELFDESTRUCT //Execution halt, and flag account for deletion
- Arithmetic Operations
-
Common arithmetic opcode instructions:
ADD //Add MUL //Multiplication SUB //Subtraction DIV //Integer division SDIV //Signed integer division MOD //Modulo (Remainder) operation SMOD //Signed modulo operation ADDMOD //Modulo addition MULMOD //Modulo multiplication EXP //Exponent operation STOP //Halt operation
- Environmental Opcodes
-
Common opcodes dealing with execution environment information:
ADDRESS //Address of current execution account BALANCE //Account balance CALLVALUE //Transaction value for execution environment ORIGIN //Origin address of execution environment CALLER //Address of execution caller CODESIZE //Execution environment code size GASPRICE //Gas price state EXTCODESIZE //An account's code size RETURNDATACOPY //Copy of data output from previous memory call
As with any computing system, the concept of state is an important one. Just like a CPU keeping track of a process in execution, the EVM must keep track of the status of various components in order to support a transaction. The status or state of these components ultimately drives the level of change in the overarching blockchain. This aspect leads to the description of Ethereum as a transaction-based state machine containing the following components:
- World State
-
A mapping between 160-bit address identifiers and account state, maintained in an immutable Merkle Patricia Tree data structure.
- Account State
-
Contains the following four components:
-
nonce: A value representing the number of transactions sent from this respective account.
-
balance: The number of wei owned by the account address.
-
storageRoot: A 256-bit hash of the Merkle Patricia Tree’s root node.
-
codeHash: An immutable hash value of the EVM code for the respective account.
-
- Storage State
-
Account specific state information maintained at runtime on the EVM.
- Block Information
-
The state values needed for a transaction include the following:
-
blockhash: The hash of the most recently completed blocks.
-
coinbase: The address of the recipient.
-
timestamp: The current block’s timestamp.
-
number: The current block’s number.
-
difficulty: The current block’s difficulty.
-
gaslimit: The current block’s gas-limit.
-
- Runtime Environment Information
-
Information used to facilitate transactions.
-
gasprice: Current gas price, which is specified by the transaction initiator.
-
codesize: Size of the transaction codebase.
-
caller: Address of the account executing the current transaction.
-
origin: Address of the current transactions original sender.
-
State transitions are calculated with the following functions:
- Ethereum State Transition Function
-
Used to calculate a valid state transition.
- Block Finalization State Transition Function
-
Used to determine the state of a finalized block as part of the mining process, including block reward.
- Block Level State Transition Function
-
The resulting state of the Block Finalization State Transition Function when applied to a transaction state.
Compiling a Solidity source file to EVM bytecode can be accomplished via the command line. For a list of additional compile options, simply run the following command:
$ solc --help
Generating the raw opcode stream of a Solidity source file is easily achieved with the --opcodes command line option. This opcode stream leaves out some information (the --asm option produces the full information), but is sufficient for this first introduction. For example, compiling an example Solidity file Example.sol and populating the opcode output into a directory named BytecodeDir is accomplished with the following command:
$ solc -o BytecodeOutputDir --opcodes Example.sol
or
$ solc -o BytecodeOutputDir --asm Example.sol
The following command will produce the bytecode binary for our example program:
$ solc -o BytecodeOutputDir --bin Example.sol
The output opcode files generated will depend on the specific contracts contained within the Solidity source file. Our simple Solidity file Example.sol [simple_solidity_example] has only one contract named "example".
pragma solidity ^0.4.19;
contract example {
address contractOwner;
function example() {
contractOwner = msg.sender;
}
}
If you look in the BytecodeDir directory, you will see the opcode file example.opcode (see [simple_solidity_example]) which contains the EVM machine language opcode instructions of the "example" contract. Opening up the example.opcode file in a text editor will show the following:
PUSH1 0x60 PUSH1 0x40 MSTORE CALLVALUE ISZERO PUSH1 0xE JUMPI PUSH1 0x0 DUP1 REVERT JUMPDEST CALLER PUSH1 0x0 DUP1 PUSH2 0x100 EXP DUP2 SLOAD DUP2 PUSH20 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF MUL NOT AND SWAP1 DUP4 PUSH20 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF AND MUL OR SWAP1 SSTORE POP PUSH1 0x35 DUP1 PUSH1 0x5B PUSH1 0x0 CODECOPY PUSH1 0x0 RETURN STOP PUSH1 0x60 PUSH1 0x40 MSTORE PUSH1 0x0 DUP1 REVERT STOP LOG1 PUSH6 0x627A7A723058 KECCAK256 JUMP 0xb9 SWAP14 0xcb 0x1e 0xdd RETURNDATACOPY 0xec 0xe0 0x1f 0x27 0xc9 PUSH5 0x9C5ABCC14A NUMBER 0x5e INVALID EXTCODESIZE 0xdb 0xcf EXTCODESIZE 0x27 EXTCODESIZE 0xe2 0xb8 SWAP10 0xed 0x
Compiling the example with the --asm option produces a file labed example.evm in our BytecodeDir directory. This contains the detailed EVM machine language instructions:
/* "Example.sol":26:132 contract example {... */
mstore(0x40, 0x60)
/* "Example.sol":74:130 function example() {... */
jumpi(tag_1, iszero(callvalue))
0x0
dup1
revert
tag_1:
/* "Example.sol":115:125 msg.sender */
caller
/* "Example.sol":99:112 contractOwner */
0x0
dup1
/* "Example.sol":99:125 contractOwner = msg.sender */
0x100
exp
dup2
sload
dup2
0xffffffffffffffffffffffffffffffffffffffff
mul
not
and
swap1
dup4
0xffffffffffffffffffffffffffffffffffffffff
and
mul
or
swap1
sstore
pop
/* "Example.sol":26:132 contract example {... */
dataSize(sub_0)
dup1
dataOffset(sub_0)
0x0
codecopy
0x0
return
stop
sub_0: assembly {
/* "Example.sol":26:132 contract example {... */
mstore(0x40, 0x60)
0x0
dup1
revert
auxdata: 0xa165627a7a7230582056b99dcb1edd3eece01f27c9649c5abcc14a435efe3bdbcf3b273be2b899eda90029
}
The --bin option produces the following:
60606040523415600e57600080fd5b336000806101000a81548173 ffffffffffffffffffffffffffffffffffffffff 021916908373 ffffffffffffffffffffffffffffffffffffffff 160217905550603580605b6000396000f3006060604052600080fd00a165627a7a7230582056b99dcb1e
Let’s examine the first two instructions (reference [common_stack_opcodes]):
PUSH1 0x60 PUSH1 0x40
Here we have the mnemonic "PUSH1" followed with a raw byte of value "0x60". This corresponds to the EVM instruction of interpreting the single byte following the opcode as a literal value and pushing it onto the stack. It is possible to push values of size up to 32 bytes onto the stack. For example, the following bytecode pushes a 4 byte value onto the stack:
PUSH4 0x7f1baa12
The second push opcode stores "0x40" onto the stack (on top of "0x60" already present there).
Moving on to the next two instructions:
MSTORE CALLVALUE
MSTORE is a stack/memory operation (see [common_stack_opcodes]) that saves a value to memory, while CALLVALUE is an environmental opcode (see [common_environment_opcodes]) that returns the deposited value of the executing message call.
For every transaction, there is an associated gas-limit and gas-price which make up the fees of an EVM execution. These fees are used to facilitate the necessary resources of a transaction, such as computation and memory. Gas is also used for the creation of accounts and smart-contracts.
In simple terms, a system or programming language is Turing complete if it can solve any problem you feed into it. This is discussed in the Ethereum Yellow Paper:
It is a quasi-Turing complete machine; the quasi qualification comes from the fact that the computation is intrinsically bounded through a parameter, gas, which limits the total amount of computation done.
ETHEREUM: A SECURE DECENTRALISED GENERALISED TRANSACTION LEDGER
While the EVM can theoretically solve any problem it receives, gas is what might prevent it from doing so. This could occur in a few ways:
1) Blocks that get mined in Ethereum have a gas limit associated with them; that is, the total gas used by all the transactions inside the block can not exceed a certain limit. 2) Since gas and gas price go hand-in-hand, even if the gas limit restrictions were lifted, highly complex transactions may be economically infeasible.
For the majority of use cases, however, the EVM can solve any problem provided to it.
When compiling a contract, you can either get the contract bytecode or the runtime bytecode.
The contract bytecode contains the bytecode of what will actually end up sitting on the blockchain plus the bytecode needed to place that bytecode on the blockchain and run the contract’s constructor.
The runtime bytecode, on the other hand, is only the bytecode that ends up sitting on the blockchain. This does not include the bytecode needed to initialize the contract and place it on the blockchain.
Let’s take the simple Faucet.sol contract we created earlier as an example.
// Version of Solidity compiler this program was written for
pragma solidity ^0.4.19;
// Our first contract is a faucet!
contract Faucet {
// Give out ether to anyone who asks
function withdraw(uint withdraw_amount) public {
// Limit withdrawal amount
require(withdraw_amount <= 100000000000000000);
// Send the amount to the address that requested it
msg.sender.transfer(withdraw_amount);
}
// Accept any incoming amount
function () public payable {}
}
To get the contract bytecode, we would run solc --bin Faucet.sol. If we instead wanted just the runtime bytecode, we would run solc --bin-runtime Faucet.sol.
If you compare the output of these commands, you will see that the runtime bytecode is a subset of the contract bytecode. In other words, the runtime bytecode is entirely contained within the contract bytecode.
Disassembling EVM bytecode is a great way to understand how high-level Solidity acts in the EVM. There are a few disassemblers you can use to do this:
-
Porosity is a popular open-source decompiler: https://github.com/comaeio/porosity
-
Ethersplay is an EVM plugin for Binary Ninja, a disassembler: https://github.com/trailofbits/ethersplay
-
IDA-Evm is an EVM plugin for IDA, another disassembler: https://github.com/trailofbits/ida-evm
In this section, we will be using the Ethersplay plugin for Binary Ninja.
After getting the runtime bytecode of Faucet.sol, we can feed it into Binary Ninja (after importing the Ethersplay plugin) to see what the EVM instructions look like.
When you send a transaction to a smart contract, the transaction first interacts with that smart contract’s dispatcher. The dispatcher reads in the data field of the transaction and sends it to the appropriate function.
After the familiar MSTORE instruction, we see the following insturctions in our compiled Faucet.sol contract:
PUSH1 0x4 CALLDATASIZE LT PUSH1 0x3f JUMPI
"PUSH1 0x4" places 0x4 onto the top of the stack, which is otherwise empty. "CALLDATASIZE" gets the size in bytes of the received transaction’s calldata and pushes it onto the stack. The current stack looks like as follows:
| Stack |
|---|
0x4 |
length of calldata from tx (msg.data) |
This next instruction is "LT", short for “less than”. The LT instruction checks whether the top item on the stack is less than the next item on the stack. In our case, it checks to see if the result of CALLDATASIZE is less than 4 bytes.
Why does the EVM check to see that the calldata of the transaction is at least 4 bytes? Because of how function identifiers work. Each function is identified by the first four bytes of its keccak256 hash. By placing the function’s name and what arguments it takes into a keccak256 hash function, we can deduce its function identifier. In our contract, we have:
keccak256("withdraw(uint256)") = 0x2e1a7d4d...Thus, the function identifier for the "withdraw(uint256)" function is 0x2e1a7d4d, since these are the first four bytes of the resulting hash. A function identifier is always 4 bytes long, so if the entire data field of the transaction sent to the contract is less than 4 bytes, then there’s no function with which the transaction could possibly be communicating, unless a fallback function is defined. Because we implemented such a fallback function in Faucet.sol, the EVM jumps to this function when the calldata’s length is less than 4 bytes.
If the msg.data field is less than 4 bytes, LT pops off the top two values of the stack and pushes 1 onto it. Otherwise, it pushes 0. In our example, let’s assume the msg.data field of the transaciton sent to our contract was less than 4 bytes.
The "PUSH1 0x3f" instruction pushes the byte "0x3f" onto the stack. After this instruction, the stack looks as follows:
| Stack |
|---|
1 |
0x3f |
The next instruction is "JUMPI", which stands for "jump if". It works like so:
jumpi(label, cond) // Jump to "label" if "cond" is true
In our case, "label" is 0x3f, which is where our fallback function lives in our smart contract. The "cond" argument is 1, which was from the result of the LT instruction earlier. To put this entire sequenece into words, the contract jumps to the fallback function if the transaction data is less than 4 bytes.
At 0x3f, only a "STOP" instruction follows, because though we declared a fallback function, we kept it empty. Had we not implemented a fallback function, the contract would throw an exception instead.
Let’s examine the central block of the dispatcher. Assuming we received calldata that was greater than 4 bytes in length, the "JUMPI" instruction would not jump to the fallback function. Instead, code execution would follow with the next instructions:
PUSH1 0x0 CALLDATALOAD PUSH29 0x1000000... SWAP1 DIV PUSH4 0xffffffff AND DUP1 PUSH4 0x2e1a7d4d EQ PUSH1 0x41 JUMPI
"PUSH1 0x0" pushes 0 onto the stack, which is otherwise empty. "CALLDATALOAD" accepts as an argument an index within the calldata sent to the smart contract and reads 32 bytes from that index, like so:
calldataload(p) // call data starting from position p (32 bytes)
Since 0 was the index passed to it from the PUSH1 0x0 command, CALLDATALOAD reads 32 bytes of calldata starting at byte 0, and then pushes it to the top of the stack (after popping the original 0x0). After the "PUSH29 0x1000000…" instruction, the stack looks as follows:
| Stack |
|---|
32 bytes of calldata starting at byte 0 |
0x1000000… (29 bytes in length) |
"SWAP1" switches the top element on the stack with the ith element after it. In this case, it swaps 0x1000000… with the calldata. The new stack looks as follows:
| Stack |
|---|
0x1000000… (29 bytes in length) |
32 bytes of calldata starting at byte 0 |
The next instruction is "DIV", which works as follows:
div(x, y) // x / y
In this case, x = 32 bytes of calldata starting at byte 0, and y = 0x100000000… (29 bytes total). Can you think of why the dispatcher is doing the division? Here’s a hint: we read 32 bytes from calldata earlier starting at index 0. The first four bytes of that calldata is the function identifier.
The 0x100000000…. we pushed earlier is 29 bytes long, consisting of a 1 at the beginning, followed by all 0s. Dividing our 32 bytes of calldata by this 0x100000000…. will leave us only the topmost 4 bytes of our calldataload starting at index 0. These four bytes – the first four bytes in calldataload starting at index 0 – are the function identifier, and this is how the EVM extracts that field.
If this part isn’t clear to you, think of it like this: in base10, 1234000 / 1000 = 1234. In base16, this is no different. Instead of every place being a multiple of 10, it is a multiple of 16. Just as dividing by 103 (1000) in our smaller example kept only the topmost digits, dividing our 32 byte base16 value by 1629 does the same.
The result of the DIV (the function identifier) gets pushed on the stack, and our new stack is as follows:
| Stack |
|---|
function identifier sent in msg.data |
Since the "PUSH4 0xffffffff" and "AND" instructions are redundant, we can ignore them entirely, as the stack will remain the same after they are done. The "DUP1" instruction duplicates the 1st item on the stack, which is the function identifier. The next instruction, "PUSH4 0x2e1a7d4d", pushes the calculated function identifier of the withdraw(uint256) function onto the stack. The stack now looks as follows:
| Stack |
|---|
function identifier sent in msg.data |
function identifier sent in msg.data |
0x2e1a7d4d |
The next instruction, "EQ", pops off the top two items of the stack and compares them. This is where the dispatcher does its main job: it compares whether the function identifier sent in the msg.data field of the transaction matches that of withdraw(uint256). If they are equal, EQ pushes 1 onto the stack, which will ultimately get used to jump to the withdraw function. Otherwise, EQ pushes 0 onto the stack.
Assuming the transaction sent to our contract indeed began with the function identifier for withdraw(uint256), our new stack looks as follows:
| Stack |
|---|
function identifier sent in msg.data |
1 |
Next, we have "PUSH1 0x41", which is the address at which the withdraw(uint256) function lives in the contract. After this instruction, the stack looks as follows:
| Stack |
|---|
function identifier sent in msg.data |
1 |
0x41 |
The JUMPI instruction is next, and it once again accepts the top two elements on the stack as arguments. In this case, we have "jumpi(0x41, 1)", which tells the EVM to execute the jump to the location of the withdraw(uint256) function.
-
[ByteCode To Opcode Disassembler](https://etherscan.io/opcode-tool) (Useful to check/debug if compilation ran with integrity and for reverse-engineering purposes if the source code wasn’t published)