Skip to content

Commit

Permalink
hijack: add a quick write test to monitor storage perf. after each test
Browse files Browse the repository at this point in the history
We've experienced a fair number of test failures that seem to point to
storage performance issues:

thesofproject/linux#3387
thesofproject/linux#3669

This (temporary?) addition runs a quick write test after each audio test
to monitor storage sanity.

As a bonus feature the "sync" could help us collect more logs.

On our (slowest) BYT devices the test adds 3s per test; much less on
newer devices.

Sample output:

```
2022-05-24 23:20:41 UTC [INFO] pkill -TERM sof-logger
2022-05-24 23:20:42 UTC [INFO] nlines=1132 /home/mherber2/SOF/sof-test/logs/BOGUS-check-playback/2022-05-24-16:20:35-3288/slogger.txt
+ timeout -s CONT 5 sudo sync

real	0m0.062s
user	0m0.005s
sys	0m0.019s
+ timeout -s CONT 10 dd if=/dev/zero of=/home/mherber2/HD_TEST_DELETE_ME bs=1M count=200 conv=fsync
200+0 records in
200+0 records out
209715200 bytes (210 MB, 200 MiB) copied, 2.0893 s, 100 MB/s
+ timeout -s CONT 5 sudo sync

real	0m0.037s
user	0m0.004s
sys	0m0.018s
2022-05-24 23:20:44 UTC [INFO] Test Result: PASS!
```

Signed-off-by: Marc Herbert <[email protected]>
  • Loading branch information
marc-hb committed May 24, 2022
1 parent 75f59b7 commit 045912c
Show file tree
Hide file tree
Showing 2 changed files with 21 additions and 0 deletions.
2 changes: 2 additions & 0 deletions case-lib/hijack.sh
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,8 @@ function func_exit_handler()

fi

storage_checks || exit_status=1

if [[ "$KERNEL_CHECKPOINT" =~ ^[0-9]{10} ]]; then
# Do not collect the entire duration of the test but only the
# last iteration.
Expand Down
19 changes: 19 additions & 0 deletions case-lib/lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,25 @@ poll_wait_for()
$pass
}


storage_checks()
{
local max_sync_duration=5

( set -x
# Thanks to CONT this does not actually timeout; it only returns a
# non-zero exit status when taking too long.
time timeout -s CONT "$max_sync_duration" sudo sync || return $?
# Spend a few seconds to test and show the current write speed
timeout -s CONT 6 dd if=/dev/zero of=~/HD_TEST_DELETE_ME bs=1M count=200 conv=fsync ||
return $?
time timeout -s CONT "$max_sync_duration" sudo sync
) || return $?

rm ~/HD_TEST_DELETE_ME
}


setup_kernel_check_point()
{
# Make the check point $SOF_TEST_INTERVAL second(s) earlier to avoid
Expand Down

0 comments on commit 045912c

Please sign in to comment.