Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Significantly long GnoVM test runtime #1406

Open
zivkovicmilos opened this issue Dec 4, 2023 · 4 comments
Open

Significantly long GnoVM test runtime #1406

zivkovicmilos opened this issue Dec 4, 2023 · 4 comments
Assignees
Labels
📦 🤖 gnovm Issues or PRs gnovm related

Comments

@zivkovicmilos
Copy link
Member

Description

Running the gnovm test suite takes an extraordinarily long amount of time to execute:

time make --no-print-directory -C gnovm test  1005.06s user 23.11s system 160% cpu 10:42.20 total

For CI runs, take a look at an example GnoVM CI run:
https://github.com/gnolang/gno/actions/runs/7031801537/job/19134177743

Notice that pkg0, pkg1, pkg2take ~13mins to execute in the CI.

An additional problem is that we cannot execute the VM tests in parallel (t.Parallel()), since there are primitives in the VM preventing parallel execution (global vars).

This long test suite execution time takes a large toll on local development and CI workflows.

@ajnavarro @gfanton please add more info to this task if I've missed something, @petar-dambovaliev will look into the issue

@zivkovicmilos zivkovicmilos added the 📦 🤖 gnovm Issues or PRs gnovm related label Dec 4, 2023
@zivkovicmilos zivkovicmilos moved this to 🚀 Needed for Launch in 🚀 The Launch [DEPRECATED] Dec 4, 2023
@zivkovicmilos zivkovicmilos changed the title Significantly lower GnoVM test runtime Significantly long GnoVM test runtime Dec 4, 2023
@ajnavarro
Copy link
Contributor

Maybe we should make the VM thread-safe, to make it possible to run several of them simultaneously.

@zivkovicmilos
Copy link
Member Author

to make it possible to run several of them simultaneously.

This is actually something that is required down the line, we can't do gas estimations (through an RPC call, cc @MichaelFrazzy) and state transitions at the same time with the current VM

@petar-dambovaliev
Copy link
Contributor

petar-dambovaliev commented Dec 4, 2023

Maybe we should make the VM thread-safe, to make it possible to run several of them simultaneously.

I also lean towards this idea.
I made a profile on the slowest tests and the distribution of runtime is kind of equal across the board.
In other words, there is no single function bottleneck.
The only one that takes more than 2% of the runtime is

// NOTE: for allocation, use *Allocator.NewBlock.
func NewBlock(source BlockNode, parent *Block) *Block

It takes about 7% of the runtime.
It is worth improving it. It wouldn't fix the tests runtime, we still need to make them thread safe but it will help a bit with that and also make the general runtime faster.

@MichaelFrazzy
Copy link

to make it possible to run several of them simultaneously.

This is actually something that is required down the line, we can't do gas estimations (through an RPC call, cc @MichaelFrazzy) and state transitions at the same time with the current VM

Is this test similar in function to the benchmarking framework that Dylan set up or something that we've had for a while?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 🤖 gnovm Issues or PRs gnovm related
Projects
Status: 🚀 Needed for Launch
Status: Backlog
Development

No branches or pull requests

4 participants