tests: update tests

[holiman]

This PR builds on #26299, but also updates the tests to the most recent version, which includes tests regarding TheMerge.

This update contains an interesting error:

--- FAIL: TestBlockchain (0.01s)
    --- FAIL: TestBlockchain/TransitionTests/bcArrowGlacierToMerge/powToPosBlockRejection.json (0.01s)
        block_test.go:51: test without snapshotter failed: block (index 11) insertion should have failed due to: 3675PoWBlockRejected
        block_test.go:54: test with snapshotter failed: block (index 11) insertion should have failed due to: 3675PoWBlockRejected

Here, we try to import some blocks, and one of the blocks goes over the TTD. The test
expects us to reject the last one, however, we have this little thing in the
beacon/consensus.go:

func (beacon *Beacon) VerifyHeaders(chain consensus.ChainHeaderReader, headers []*types.Header, seals []bool) (chan<- struct{}, <-chan error) {
	if !beacon.IsPoSHeader(headers[len(headers)-1]) {
		return beacon.ethone.VerifyHeaders(chain, headers, seals)
	}
	var (
		preHeaders  []*types.Header
		postHeaders []*types.Header

That is, we check the very last header -- if it looks like a PoW-header, we ship all
headers to the ethone engine for verification, and done.

In the other path, where we go through the work of separating them up in pre- and post- headers,
we're much more careful about ensuring that the PoW headers do not exceed the TTD:

		// Verify that pre-merge headers don't overflow the TTD
		if index, err := verifyTerminalPoWBlock(chain, preHeaders); err != nil {

This is something we need to investigate, I'm not sure exactly what is The Correct Fix.

[MariusVanDerWijden]

Hmm you're right I think. If we get a long pow chain, we only verify it with the inner engine and in there we don't verify that the block isn't post ttd afaict

[rjl493456442]

If the header batch contains all PoW style headers and some of them are above the TTD, it means they come from the legacy PoW network which should be rejected in the first place.

There are some logics in the fetcher https://github.com/ethereum/go-ethereum/blob/master/eth/handler.go#L279

I mean these logics are not embedded in the consensus engine but we hold the assumption that they should be rejected.

[holiman]

I mean these logics are not embedded in the consensus engine but we hold the assumption that they should be rejected.

IMO these checks should be embedded in the (beacon) consensus engine.. So I'll make a stab at that. Also, we could merge this PR as is, with the test commented-out, and fix that test in a follow-up PR. If someone approves this ...

[holiman]

I have now applied a fix, please take a look

[holiman]

When I was fixing the failing tests, I also got around to fix the broken test which is also touched in #26272 .

[holiman]

Note for reviewers: the changes look large, but most of it is in tests. The core change is here: https://github.com/ethereum/go-ethereum/pull/26314/files#diff-89272f61e115723833d498a0acbe59fa2286e3dc7276a676a7f7816f21e248b7

[rjl493456442]

Shouldn't we sum up the difficulty of past blocks?

[holiman]

IIUC, they are all using the same difficulty of params.GenesisDifficulty. So n*GenesisDifficulty menas that n-1 reaches ttd, since genesis also has a difficulty.
If there's an off by one here, it seems that the test ignore it.
... It was humongously erroneous before this change...

[rjl493456442]

@karalabe @MariusVanDerWijden Please also check the change in consensus/beacon package.

[MariusVanDerWijden]

Tried this on hive and the following tests break:

INFO[12-19|16:19:10] API: test started                        suite=0 test=1 name="Invalid Terminal Block in ForkchoiceUpdated (go-ethereum)"
INFO[12-19|16:19:12] API: client go-ethereum started          suite=0 test=1 container=1c54ed85
INFO[12-19|16:19:12] API: test ended                          suite=0 test=1 pass=false
INFO[12-19|16:19:12] API: test started                        suite=0 test=2 name="Invalid GetPayload Under PoW (go-ethereum)"
INFO[12-19|16:19:14] API: client go-ethereum started          suite=0 test=2 container=6c74bb76
INFO[12-19|16:19:14] API: test ended                          suite=0 test=2 pass=false
INFO[12-19|16:19:14] API: test started                        suite=0 test=3 name="Invalid Terminal Block in NewPayload (go-ethereum)"
INFO[12-19|16:19:16] API: client go-ethereum started          suite=0 test=3 container=8978a659
INFO[12-19|16:19:16] API: test ended                          suite=0 test=3 pass=false

[holiman]

Tried this on hive

Which simulator is that?

[holiman]

Ah, it's these ones: https://github.com/ethereum/hive/blob/master/simulators/ethereum/engine/README.md#engine-api-negative-test-cases

Invalid Terminal Block in ForkchoiceUpdated:
Client should reject ForkchoiceUpdated directives if the referenced HeadBlockHash does not meet the TTD requirement.

[holiman]

It's still running, but so far (115 tests) all tests pass for me

INFO[12-20|09:19:09] API: test started                        suite=0 test=1 name="Invalid Terminal Block in ForkchoiceUpdated (go-ethereum_latest)"
INFO[12-20|09:19:13] API: client go-ethereum_latest started   suite=0 test=1 container=a9d15988
INFO[12-20|09:19:16] API: test ended                          suite=0 test=1 pass=true

[MariusVanDerWijden]

LGTM