You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Scenario: I want to run a lot (thousands) of benchmarks for .NET framework itself. The first step that BDN does in pilot stage is finding the perfect invocation count.
My goal: add an option to remember this value (+ iteration count + unroll factor) to do it only once for new benchmarks and then read it from some storage.
My initial idea is following:
add new interface IStorage to BenchmarkDotNet.dll
add a nullable property to the IConfig which allows configuring IStorage
Implement two classes that implement the interface. One for some lightweight file-db (like LiteDb) and one for some popular and cheap cloud db (like AzureTableStorage). Add it to new packages BenchmarkDotNet.Storage.$NameOfTheDb
@AndreyAkinshin what do you think about this idea? It would another step to improve the perf (#550)
The text was updated successfully, but these errors were encountered:
@adamsitnik, sorry for the late reply. Here are some thoughts:
We can't completely skip the pilot change. E.g., if you make a 100x perf improvement or just run a benchmark on a very slow computer, we should calibrate the invocation count. However, it's a pretty good idea to automatically configure the initial invocation count. In simple cases, the pilot stage should be finished after the first iteration.
We should keep the benchmark source code, config, and predefined values together (in the repository). If you have any settings in an external storage, it means that your benchmarks are not completely reproducible. Also, you can't test such benchmarks offline.
A plain text format is preferred. Each predefined value contain 3 values: benchmark FQN, property name (e.g. InitialInvocationCount), property value. Even if you have thousands of benchmark, we can instantly load a file with thousands of lines (e.g. one line per benchmark) and parse them. I think that we don't need any db here (unless you are going to run billions of benchmarks). Also, it's much simpler to debug and experiment with plain text files.
We can introduce an additional boolean property in the config which automatically generates such file.
Scenario: I want to run a lot (thousands) of benchmarks for .NET framework itself. The first step that BDN does in pilot stage is finding the perfect invocation count.
My goal: add an option to remember this value (+ iteration count + unroll factor) to do it only once for new benchmarks and then read it from some storage.
My initial idea is following:
IStorage
to BenchmarkDotNet.dllIConfig
which allows configuringIStorage
@AndreyAkinshin what do you think about this idea? It would another step to improve the perf (#550)
The text was updated successfully, but these errors were encountered: