Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance for test generation and execution #359

Merged
merged 3 commits into from
Feb 12, 2024

Conversation

MatheusFranco99
Copy link
Contributor

@MatheusFranco99 MatheusFranco99 commented Feb 1, 2024

Overview

The purpose of this PR is to optimize the performance of test generation and execution with the purpose of improving work quality on future changes.

The ssv module is the one that takes the most time. After running a CPU profiling test, we noticed that the NewTestingKeyManager function consumes 80% of the total time, which is explained by the fact that it's called multiple times and takes 26 milliseconds to run (according to benchmark tests).

Nonetheless, the NewTestingKeyManager function returns a testingKeyManager object that is only used to validate and sign data but does not change state throughout a test execution. Thus, we propose adding a singleton pattern for this object.

Metrics

To illustrate the number of times the NewTestingKeyManager function is called, take a look at the following table

Note: the number after the dash (/) is the number of different instances created, which depends on a slashableDataRoots parameter.

Module Test generation Test execution
QBFT 368/1 390/1
SSV 17058/1 20470/2
Types 48/1 48/1

Performance comparison

With the singleton pattern, if the appropriate object already has been created, the function takes 5 microseconds, rather than 26 milliseconds.

All time values shown below are in seconds (or in minutes:seconds if it has ":" ).

For test generation, we got the following performance results.

QBFT SSV Types
Old 15 7:36 3
New 5 20 3

For test execution, we got the following performance results.

QBFT SSV Types
Old 16 1:22 3
New 4 5 2

Copy link
Contributor

@GalRogozinski GalRogozinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

)

func getHash(data [][]byte) string {
hasher := sha256.New()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can consider using MuHash or something else which is faster

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fnv is easy fast and simple for tasks like this. built in the go stdlib

Comment on lines 49 to 52
mu.Lock()
defer mu.Unlock()

hash := getHash(slashableDataRoots)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better approach would be getHash before locking, so we keep it locked as short as possible

Comment on lines 30 to 33
var (
instancesMap = make(map[string]*testingKeyManager)
mu sync.Mutex
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally I would say global state is a bad idea, but since this is a test generation code and you are protecting it properly, its seems ok to me.

@GalRogozinski GalRogozinski merged commit 7635881 into main Feb 12, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants