triedb/pathdb: fix flaky test in pathdb

[lilasxie]

Refer to #29830 comment

Larger maxDiffLayers causes a higher probability of failing. If maxDiffLayers >= 7, it always fails.

I think it's due to the round times of init data in newTester.

After adjusting round times to 12 and stress test TestNewTester for 2 minutes, no failed again

// TestNewTester is a temp test for quickly check
func TestNewTester(t *testing.T) {
	// Redefine the diff layer depth allowance for faster testing.
	maxDiffLayers = 4
	defer func() {
		maxDiffLayers = 128
	}()
	tester := newTester(t, 0)
	defer tester.release()
	if ok, err := tester.db.diskdb.Has(rawdb.TrieNodeAccountPrefix); !ok {
		panic(fmt.Sprintf("missing key %v %v", ok, err))
	}
}

stress ./pathdb.test TestNewTester

5s: 0 runs so far, 0 failures
10s: 16 runs so far, 0 failures
15s: 32 runs so far, 0 failures
20s: 48 runs so far, 0 failures
25s: 64 runs so far, 0 failures
30s: 80 runs so far, 0 failures
35s: 96 runs so far, 0 failures
40s: 112 runs so far, 0 failures
45s: 128 runs so far, 0 failures
50s: 144 runs so far, 0 failures
55s: 160 runs so far, 0 failures
1m0s: 176 runs so far, 0 failures
1m5s: 192 runs so far, 0 failures
1m10s: 208 runs so far, 0 failures
1m15s: 224 runs so far, 0 failures
1m20s: 240 runs so far, 0 failures
1m25s: 256 runs so far, 0 failures
1m30s: 272 runs so far, 0 failures
1m35s: 288 runs so far, 0 failures
1m40s: 304 runs so far, 0 failures
1m45s: 320 runs so far, 0 failures
1m50s: 336 runs so far, 0 failures
1m55s: 352 runs so far, 0 failures
2m0s: 368 runs so far, 0 failures

[lilasxie]

And for TestDisable stress test 3 minutes

stress ./pathdb.test TestDisable

5s: 0 runs so far, 0 failures
10s: 16 runs so far, 0 failures
15s: 32 runs so far, 0 failures
20s: 64 runs so far, 0 failures
25s: 80 runs so far, 0 failures
30s: 96 runs so far, 0 failures
35s: 112 runs so far, 0 failures
40s: 128 runs so far, 0 failures
45s: 160 runs so far, 0 failures
50s: 176 runs so far, 0 failures
55s: 192 runs so far, 0 failures
1m0s: 208 runs so far, 0 failures
1m5s: 224 runs so far, 0 failures
1m10s: 240 runs so far, 0 failures
1m15s: 256 runs so far, 0 failures
1m20s: 256 runs so far, 0 failures
1m25s: 272 runs so far, 0 failures
1m30s: 272 runs so far, 0 failures
1m35s: 288 runs so far, 0 failures
1m40s: 304 runs so far, 0 failures
1m45s: 320 runs so far, 0 failures
1m50s: 336 runs so far, 0 failures
1m55s: 352 runs so far, 0 failures
2m0s: 368 runs so far, 0 failures
2m5s: 384 runs so far, 0 failures
2m10s: 400 runs so far, 0 failures
2m15s: 416 runs so far, 0 failures
2m20s: 432 runs so far, 0 failures
2m25s: 448 runs so far, 0 failures
2m30s: 464 runs so far, 0 failures
2m35s: 480 runs so far, 0 failures
2m40s: 496 runs so far, 0 failures
2m45s: 512 runs so far, 0 failures
2m50s: 528 runs so far, 0 failures
2m55s: 544 runs so far, 0 failures
3m0s: 560 runs so far, 0 failures

[rjl493456442]

Larger maxDiffLayers causes a higher probability of failing. If maxDiffLayers >= 7, it always fails.

Could you provide the logs of failing tests?

maxDiffLayers should not be larger than 7. We only initialize 8 layers for testing, and a larger maxDiffLayers value could prevent the merging of underlying layers.

[lilasxie]

Larger maxDiffLayers causes a higher probability of failing. If maxDiffLayers >= 7, it always fails.

Could you provide the logs of failing tests?

maxDiffLayers should not be larger than 7. We only initialize 8 layers for testing, and a larger maxDiffLayers value could prevent the merging of underlying layers.

@rjl493456442 Sorry sir, I thought I had referred to the issue before.

From CI https://ci.appveyor.com/project/ethereum/go-ethereum/builds/49926124/job/2nw6ofdur8xt4ox6

?   	github.com/ethereum/go-ethereum/triedb	[no test files]
?   	github.com/ethereum/go-ethereum/triedb/database	[no test files]
?   	github.com/ethereum/go-ethereum/triedb/hashdb	[no test files]
--- FAIL: TestDisable (0.16s)
    database_test.go:482: Invalid activation should be rejected
FAIL
FAIL	github.com/ethereum/go-ethereum/triedb/pathdb	2.345s
FAIL
util.go:47: exit status 1
exit status 1

[holiman]

    database_test.go:482: Invalid activation should be rejected

That error is different, I think it stems from the test state progressions winding up with an empty state

[holiman]

That error is different,

My bad. You are right, having 8 there somehow makes it so that when we later try to do rawdb.ReadAccountTrieNode(db.diskdb, nil), we get nothing. I suspect 8 is insufficient to trigger a flush to "disk" ?

Regarding progressions winding up with an empty state, that is also a problem which causes flaky failures, of the type panic: failed to update state changes, err: layer cycle. This can be fixed if we modify the generate function a bit, e.g. we could remove 'delete', always do a 'create', and then randomly do some mutations.

[lilasxie]

@rjl493456442 @holiman Hi sir, how is this PR going? Anything else need I to update?