Skip to content

Commit

Permalink
test: update printing results
Browse files Browse the repository at this point in the history
For getting stable results we need to run performance tests many times
and then calculate mean, median and standard deviation for results.
luatest has an option `-r N` that allows to run testcase N times, but it
don't execute setup/teardown on each iteration [1] and due to this our
testcases fails due to duplicated id's in a space.
For running tests many times I made an external loop in a shell script:

```
for i in `seq 1 1 30`; do
luatest -v -c test/performance/select_perf_test.lua 2>&1 | grep -E 'INSERT|SELECT'`;
done
```

For more convenient extracting numbers from log I changed format of
printing on each iteration - all numbers that changed from run to run
printed in a single row, and removed printing numbers that are always
static.

1. tarantool/luatest#161
  • Loading branch information
ligurio committed Sep 3, 2021
1 parent f47a9d4 commit f0445ea
Showing 1 changed file with 4 additions and 14 deletions.
18 changes: 4 additions & 14 deletions test/performance/select_perf_test.lua
Original file line number Diff line number Diff line change
Expand Up @@ -130,13 +130,8 @@ g.test_insert = function()
fibers[i]:join()
end

log.error('INSERT')
log.error('Fibers count - %d', fiber_count)
log.error('Connection count - %d', connection_count)
log.error('Timeout - %f', timeout)
log.error('Requests - %d', report.count)
log.error('Errors - %s', #report.errors)
log.error('RPS - %f', report.count / timeout)
log.error('\nINSERT: requests %d, rps %d, errors %d',
report.count, report.count / timeout, #report.errors)
end

g.test_select = function()
Expand Down Expand Up @@ -167,11 +162,6 @@ g.test_select = function()
fibers[i]:join()
end

log.error('SELECT')
log.error('Fibers count - %d', fiber_count)
log.error('Connection count - %d', connection_count)
log.error('Timeout - %f', timeout)
log.error('Requests - %d', report.count)
log.error('Errors - %s', #report.errors)
log.error('RPS - %f', report.count / timeout)
log.error('\nSELECT: requests %d, rps %d, errors %d',
report.count, report.count / timeout, #report.errors)
end

0 comments on commit f0445ea

Please sign in to comment.