Releases: ethereum/tests
Tests update 14.1 Legacy Tests
The recent tests changes:
From now on there would be no new test vectors in this repository and existing ones will slowly be converted into https://github.com/ethereum/execution-spec-tests. NOTE: Pyspecs doesn't cover all the vectors from this repo, but has many new vectors that are not covered in ethereum/tests
Eventually there will be only ethereum/legacytests repo with all .json tests from Frontier to Cancun.
And ethereum/execution-spec-tests will handle all the official ethereum testing vectors for all the froks.
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: https://tinyurl.com/y525ezs7
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v14.0
Test format
Tests Fillers
GeneralStateTests/BlockchainTests Filled
Test Regeneration
Update all:
- Documentation
TESTS
Converted to python Tests
- Test all opcodes
PR #1399 - A few eip1153 tests
PR #1383 - EOF return contract tests removed
PR #1391 - EIP eof tests
PR #1395
PR #1397
PR #1400
Cosmetic
Upcoming changes:
Convert .json .yml yo .py
Pyspecs filling support
Tests update 14 Legacy Tests
The recent tests changes:
Make all existing tests snapshot into LegacyTests folder Cancun
Make all recent test vectors >=Cancun only
Add PYSPEC style exceptions to develop tests
Begin convertion of json fillers to .py fillers (migrate to pyspecs repo)
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: https://tinyurl.com/y525ezs7
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v13.3
Test format
Tests Fillers
New folder LegacyTests/Cancun
GeneralStateTests/BlockchainTests Filled
New folder LegacyTests/src/LegacyTests/Cancun
Test Regeneration
Update all:
PR #1377 (make all tests snapshot)
PR #1380 (refill only cancun vectors)
PR #1382 (update pyspec tests)
- Documentation
PR #1376 (list KEVM in readme)
TESTS
-
Rename exceptions to PYSPEC style
PR #1380 -
Replace addresses in tests 0x00..0100 to 0x00..01000
PR #1372 -
Enable ecparing tests for cancun and beyond
PR #1375
EIPTests/EOFTests
Converted to python Tests
The following tests will now be managed by pyspecs test generator
PR #1383 (Cancun/stEIP1153)
Cosmetic
-
Fix invalid json fields from pyspecs
PR #1385 -
Mark bigInt in test ValueOverflow when generated from state test
PR #1387 -
Fix pyspec test filled with bug
PR #1388
Upcoming changes:
Convert .json .yml yo .py
Pyspecs filling support
Tests update 13.3 Cancun tests
The recent tests changes:
Fix the issue in blockchain tests with beacon root precompile, cosmetic updates
EIP7610 legacy tests fix
EIP tests for BLS
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: https://tinyurl.com/y525ezs7
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v13.2
Test format
Tests Fillers
GeneralStateTests/BlockchainTests Filled
Test Regeneration
Update all:
- Documentation
- PR #1356 (fix typos)
TESTS
-
Fix legacy tests to revert on create collision with non empty storage dest (EIP7610)
PR #1367 -
**Fix the blockchain tests logic with beacon root in genesis **
PR #1357 -
Add more ecpairing test vectors
PR #1358 -
Update bad opcode tests for Cancun
PR #1362 -
Update pyspec tests, rename Merge to Paris
PR #1363
EIPTests/EOFTests
-
Reintroduce BLS precompile tests
PR #1366 -
Fix and update EOF tests
PR #1349 -
More EOF validation tests from evmone
PR #1364
Upcoming changes:
Convert .json .yml yo .py
Pyspecs filling support
Tests update 13.2 Cancun tests
The recent tests changes:
Cosmetic update
Insert beacon precompile into bc tests's pre >= Cancun
Fix genesis excessBlobGasUsed for first block excessBlobGasUsed to be = 0 or x calculation
Remove empty storage records from pre state
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: https://tinyurl.com/y525ezs7
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v13.1
Test format
Tests Fillers
GeneralStateTests/BlockchainTests Filled
Insert beacon precompile into tests pre if not exist for BlockchainTests >= Cancun
"0x000f3df6d732807ef1319fb7b8bb8522d0beac02" : {
"balance" : "0x00",
"code" : "0x3373fffffffffffffffffffffffffffffffffffffffe14604d57602036146024575f5ffd5b5f35801560495762001fff810690815414603c575f5ffd5b62001fff01545f5260205ff35b5f5ffd5b62001fff42064281555f359062001fff015500",
"nonce" : "0x01",
"storage" : {
}
},
Test Regeneration
Update all:
-
Pyspec update: remove beacon contract from tests, remove empty storage records
PR #1345 -
Refill all blockchain tests, add beacon root on cancun
PR #1351 -
Documentation
TESTS
- --
PR --
EIPTests/EOFTests
-
update for relaxed stack validation
PR #1334 -
**Move EOF creation tests into EOF test format **
PR #1337
Cosmetic
-
Move random blockhash calling state tests to bc format
PR #1354 -
Change exception name in PYSPEC
PR #1342 -
Remove empty storage keys (0x00) in pre state
PR #1347
Upcoming changes:
Convert .json .yml yo .py
Pyspecs filling support
Tests update 13.1 Cancun tests
The recent tests changes:
Rename Merge to Paris (note: Pyspecs tests still have Merge keyword)
Remove empty accounts from post Paris
Update cancun rules, state test format, update pyspecs vectors
Update filled state test format
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: https://tinyurl.com/y525ezs7
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v13
Test format
Tests Fillers
Pyspec tests folder moved from /src/BlockchainTestsFiller/Pyspecs to /src/GeneralStateTestsFiller/Pyspecs
and the generated tests moved accordingly. as .py tests now could have state and blockchain tests.
--filltests flag will generate all state tests out of .py files
--filltests --fillchain flag will generate all blockchain tests out of .py files.
GeneralStateTests/BlockchainTests Filled
rename forkname in all of the tests: "Merge" => "Paris"
empty accounts (zero balance, no code, zero nonce, with or without storage) in pre state are not allowed post Paris.
In state tests remove blockchain related fields:
"currentBeaconRoot" : "0x0000000000000000000000000000000000000000000000000000000000000000",
"currentWithdrawalsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",`
"previousHash" : "0x5e20a0453cecd065ea59c37ac63e079ee08998b6045136a8ce6635c7912ec0b6"
- currentBeaconRoot although not set, the value could still be found in
prestate in beacon root contract.
(ethereum/execution-spec-tests#413)
In blockchain tests if invalid block rlp is unreadable, or supposed to fail during rlp parsing, rlp_decoded field no longer provided
Test Regeneration
Update all:
-
All tests regeneration with Cancun rules
PR #1332
PR #1333 (PYSPEC UPDATE) -
Adapt tests and remove empty accounts from state/bc tests post Paris/Merge
PR #1335
PR #1336 -
Documentation
PR #1323
TESTS
EIPTests/EOFTests
- Update EOF tests to not contain unreachable code sections
PR #1319
Cosmetic
- ---
PR ---
Upcoming changes:
Convert .json .yml yo .py
Pyspecs filling support
Tests update 13 Cancun tests
The recent tests changes:
Regenerate all tests using cancun rules
Move Cancun eip tests from EIPTests folder into normal test folders
Test diagram updated: https://tinyurl.com/y525ezs7
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: https://tinyurl.com/y525ezs7
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v12.4
Test format
New test folder GeneralStateTests/Cancun
New test folder /src/GeneralStateTestsFiller/Cancun
Tests Fillers
defining solidity yul now allows to switch optimisations and select evm version:
--evm-version shanghai, use default solc optimisations
code: |
:yul shanghai optimise {
sstore(0, 1)
}
--evm-version london, no optimisations (default)
code: |
:yul london {
sstore(0, 1)
}
This is because each new version of solc introduce default optimisations that change the compiled code bytes. to avoid this affecting the tests by default solc optimisations in tests are disabled
GeneralStateTests/BlockchainTests Filled
Invalid blocks are now transcripted. new format of invalid bc block:
{
"blocknumber" : "3",
"chainname" : "default",
"expectException" : "InvalidGasLimit2",
"rlp" : "0x...",
"rlp_decoded" : {
"blockHeader" : {
...
},
"transactions" : [
{
...
}
],
"uncleHeaders" : [
{
...
}
],
"withdrawals" : [
]
}
}
Keep in mind that this are debug info fields: blocknumber, chainname, expectException
Test Regeneration
Update all:
TESTS
-
Tests for contract creation
PR #1306 -
Regenerate transaction sequence into invalid blocks
PR #1316 -
Move Cancun tests from EIP Tests
PR #1313
EIPTests/EOFTests
- Pyspecs Cancun tests
PR #1299 Update Pyspecs to 1.0.5
Cosmetic
- Fix fillers json with missing commas
PR #1310
Upcoming changes:
Support pyspec .py test filling with different clients using retesteth
Nimbus --vmtrace support
Tests update 12.4 Preparations to fill Cancun
The recent tests changes:
Add Cancun blob fields and transactions to json tests
Update pyspec tests
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v12.3
Test format
Tests Fillers
new cancun fields in env section
parentExcessBlobGas: 0x1000
parentBlobGasUsed: 0x2000
currentBeaconRoot: 0x45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8
withdrawals root explicitly exported in state tests
"currentWithdrawalsRoot" : "0x56e81f171bcc55a6ff8345e692c0f86e5b48e01b996cadc001622fb5e363b421",
new lllc instruction
[[0]] (BLOBHASH 0)
new transaction type (blob transaction)
transaction:
data:
- data: :label declaredKeyWrite :raw 0x00
accessList:
- address: 0x095e7baea6a6c7c4c2dfeb977efac326af552d87
storageKeys:
- 0x00
- 0x01
maxFeePerGas: '0x12A05F200'
maxPriorityFeePerGas: '2'
gasLimit:
- '4000000'
nonce: '0'
to: 095e7baea6a6c7c4c2dfeb977efac326af552d87
value:
- '100000'
secretKey: "45a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
maxFeePerBlobGas: '10'
blobVersionedHashes:
- "0x01a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
GeneralStateTests/BlockchainTests Filled
new block header fields
"blobGasUsed" : "0x00",
"excessBlobGas" : "0x00",
"parentBeaconBlockRoot" : "0x01a915e4d060149eb4365960e6a7a45f334393093061116b197e3240065ff2d8"
Test Regeneration
Update all:
-
Refill solc/yul tests using solc evm version flag
PR #1280 PR #1279 PR #1278 PR #1277
PR #1276 PR #1275 PR #1274 PR #1273
PR #1272 PR #1271 PR #1270 PR #1269
PR #1268 PR #1267 PR #1266 PR #1265
PR #1264 PR #1263 PR #1262 PR #1261
PR #1260 PR #1259 PR #1258 PR #1257
PR #1256 PR #1255 PR #1254 PR #1253
PR #1252 PR #1251 PR #1250 -
Documentation
PR #1283 -
State tests transactions from non existing account
PR #1290
PR #1291
EIPTests/EOFTests
-
Refill EIPTests with new blob names
PR #1224
Cosmetic
PR #1247
PR #1248
PR #1281
PR #1289
Upcoming changes:
Fill all tests with cancun rules
Support pyspec .py test filling with different clients using retesteth
Nimbus --vmtrace support
Tests update 12.3 EIP tests and Pyspecs support
The recent tests changes:
Update .py tests to the latest version (but not Cancun)
Update EIPTests to new specs and add more test cases
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v12.2
Test format
Tests Fillers
GeneralStateTests/BlockchainTests Filled
Test Regeneration
Update all:
-
Refill invalid tx state tests into invalid block bc tests
PR #1225
EIPTests/EOFTests
-
More EIP1153 (Transient storage) tests
PR #1220
PR #1235
PR #1237 -
CALLF Stack Overflow test
PR #1214 -
Update EOF tests
PR #1219 (Forbid DELEGATECALL EOF → legacy)
Cosmetic
Upcoming changes:
Support pyspec .py test filling with different clients using retesteth
Nimbus --vmtrace support
Tests update 12.2: EOF Tests
The recent tests changes:
Update EOFTests (EIPTests)
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v12.1
Test format
Tests Fillers
GeneralStateTests/BlockchainTests Filled
Test Regeneration
Update all:
- Documentation
EIPTests/EOFTests
-
EIP-5450: State Tests And EOF Tests
PR #1163 (EOF Stack validation) -
EIP-4750: State Tests And EOF Tests
PR #1115 (EOF Functions) -
EIP4200: EOF - Static relative jumps
(3540-3670-4200
PR #1103 (EOF Jumps) -
EIP3540: EOF - Ori's tests
PR #1206 (change to Cancun frokword)
Cosmetic
- Change ttWrongRLP tx exception
PR #1211
Upcoming changes:
SSZ tests
Implement malicious rlp tests in StateTest or BlockchainTest format (need more debugging with t8n)
Besu retesteth RPC support (web vmtracing support)
Random state test script
Tests update 12.1: EIPTests separate, Pyspecs
The recent tests changes:
Move EIPTests into a separate folder so not to confuse
Introduce Pyspecs test vectors and support .py fillers
Announcement
New docker instructions for all t8n supported clients: https://github.com/ethereum/retesteth/tree/develop#docker-instructions
The test process diagram: http://tinyurl.com/2ctak8yn
Reminder of a brand new web interface for test debugging: http://retesteth.ethdevops.io/web/
Tests stats on latest builds: http://retesteth.ethdevops.io/
(implement test RPC support or t8ntool protocol to apply for the page. https://ethereum-tests.readthedocs.io/en/latest/t8ntool-ref.html)
Previous release link: https://github.com/ethereum/tests/releases/tag/v12
Test format
Tests Fillers
new folder /src/BlockchainTestsFiller/Pyspecs
new folder /src/EIPTestsFiller/StateTests
new folder /src/EIPTestsFiller/BlockchainTests
GeneralStateTests/BlockchainTests Filled
new folder /BlockchainTests/Pyspecs
new folder /EIPTests/StateTests
new folder /EIPTests/BlockchainTests/StateTests (EIP state tests generated in blockchain form)
Test Regeneration
Update all:
PR #1197 (Update Transaction and Difficulty tests)
PR #1203 (Update Legacy Tests)
-
Documentation
-
Pyspec test vectors
PR #1209 -
Move EIPTests into separate folder from all of the common tests
PR #1192 -
EIP1153 - Transient Storage Tests
PR #1196 -
More withdrawals tests
PR #1198
PR #1199 (Merge era comes block with withdrawals body)
EIPTests
- Update EIP-3540 and EIP-3670 Tests to Cancun
PR #1193 (EIPTests) - EOFTests add exception checks
PR #1208
Upcoming changes:
Implement malicious rlp tests in StateTest or BlockchainTest format (need more debugging with t8n)
Besu retesteth RPC support (web vmtracing support)
Random state test script