chore(storage): add warmup option [benchmarks] #8418
Conversation
product-auto-label
bot
added
size: m
|
|
||
| for deadline := time.Now().Add(opts.warmup); time.Now().Before(deadline); { | ||
| warmupGroup.Go(func() error { | ||
| benchmark := &w1r3{opts: opts, bucketName: opts.bucket} |
There was a problem hiding this comment.
So, I guess this uses the same level of parallelism as was specified for the main workload? I am wondering if it makes more sense to have a fixed/higher level of parallelism, or to be able to specify this separately.
There was a problem hiding this comment.
It does; I am open to changing it to a fixed parallelism level - say 16? I think being able to specify this separately may create unnecessary confusion - unless we determine it would be beneficial to have different levels of parallelism for warmups for different workloads, in which case we can always add this in.
The way it is now gives consistency in terms of bucket traffic: the bucket always sees the same level of parallelism; but ultimately for the long term a fixed parallelism makes sense.
There was a problem hiding this comment.
Yeah I think 16 (or numCPUs) is appropriate. We could make it configurable if/when we refactor all the CLI opts into some kind of config file eventually...
There was a problem hiding this comment.
Good idea - I changed it to num cpus
Extra telemetry set up to test SSB - will revert those changes.
This runs w1r3 benchmarks (discarding output) to warm up.
It does write/read from disk to warmup - we can change that in the future but this should do for now