Skip to content
This repository has been archived by the owner on Aug 23, 2020. It is now read-only.

Allow to configure benchmark from CLI #1663

Merged
merged 2 commits into from
Dec 2, 2019

Conversation

GalRogozinski
Copy link
Contributor

@GalRogozinski GalRogozinski commented Nov 14, 2019

Description

We have added the following flags to benchmark runner:
-Dforks - How may time should the jvm fork. Running more than one fork will cause more iterations but should reset JVM optimization profiles. May reduce variance. Default Value: 1.
-DwarmUpIterations - How many warm up iterations we should run before we stop measuring. Default value: 5
-DmeasurementIterations - How many measurement iterations we should have. Default value: 10
-Dthreads - How many threads will run the test (concurrency). Default value: # of cores.

Example of how to run from CLI:
mvn -Dtest=BenchmarkRunner#launchDbBenchmarks -Dforks=2 -DmeasurementIterations=6 -DwarmUpIterations=2 -Dthreads=3 test

Fixes #1651

Type of change

  • Test Enhancement (a non-breaking change which adds functionality)

How Has This Been Tested?

Locally run the example line. The following was printed:

# Warmup: 2 iterations, 10 s each
# Measurement: 6 iterations, 10 s each
# Timeout: 10 min per iteration
# Threads: 3 threads, will synchronize iterations
# Benchmark mode: Average time, time/op
# Benchmark: com.iota.iri.benchmarks.dbbenchmark.RocksDbBenchmark.deleteBatch
# Parameters: (numTxsToTest = 10)

Checklist:

  • My code follows the style guidelines for this project
  • I have performed a self-review of my own code

Copy link
Contributor

@kwek20 kwek20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not actually faster now right? Youre just able to change settings to make it faster :D
Or does the .threads() change anything? I assumed it would always take all processors

@GalRogozinski
Copy link
Contributor Author

True,
Devops said that they want to accelerate the test by utilizing more cpus

I am still waiting for their feedback

@GalRogozinski GalRogozinski merged commit 80b7834 into iotaledger:dev Dec 2, 2019
@GalRogozinski GalRogozinski mentioned this pull request May 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make benchmarks faster
3 participants