build: tests: add execution spec tests #26985
Conversation
|
This PR A random other PR (numbers taken from Appveyor ubuntu run) Are you sure it is actually running the tests? Or are there just not a lot of them? |
|
There's not a lot of them right now. I think on the order of 50 or so |
build/ci.go
Outdated
| csdb := build.MustLoadChecksums("build/checksums.txt") | ||
| ext := ".tar.gz" | ||
| base := "fixtures" // TODO(MariusVanDerWijden) rename once the version becomes part of the filename | ||
| url := fmt.Sprintf("https://github.com/ethereum/execution-spec-tests/releases/download/v%s/%s%s", version, base, ext) | ||
| archivePath := filepath.Join(cachedir, base+ext) |
There was a problem hiding this comment.
If you take the version as a parameter, e.g. go run ./build/ci.go -test --test.spec "0.2.4". Then we don't have to muck around with making version part of the filename. In order to update the spec, we just bump the version and update the hash.
And then this PR would be good to go, without any TODO's left..(?)
There was a problem hiding this comment.
Actually, I guess it is already good to go. TO update for a new release of the tests, we have to update the const version string and the hash. Doing it the way I proposed doesn't simplify anything
build/ci.go
Outdated
| coverage = flag.Bool("coverage", false, "Whether to record code coverage") | ||
| verbose = flag.Bool("v", false, "Whether to log verbosely") | ||
| race = flag.Bool("race", false, "Execute the race detector") | ||
| stateTestDir = "tests/spec-tests" |
There was a problem hiding this comment.
Is there any reason for definining it here? If you move it into downloadSpecTestFixtures, you will have all related to spec-tests collected there, which might be nicer?
|
It would be nice to have a script/tool like
|
This makes it possible to run the execution-spec-tests (a.k.a. pyspec) in CI. --------- Co-authored-by: Felix Lange <[email protected]>
This reverts commit 22c5567.
This reverts commit 22c5567.
This PR adds the execution spec tests to our test suite.
Doesn't work under windows yet