tests/kernel/timer/timer_api failed on reel_board and mec15xxevb_assy6853.

[chen-png]

Describe the bug
for mec15xxevb_assy6853, it was the same error log as reel_board.

To Reproduce
Steps to reproduce the behavior:

1. west build -b reel_board -p auto tests/kernel/timer/timer_api/
2. west flash
3. See error

Screenshots or console output
*** Booting Zephyr OS build zephyr-v2.2.0-1006-g41712204059e ***
Running test suite timer_api

starting test - test_time_conversions
PASS - test_time_conversions

starting test - test_timer_duration_period
PASS - test_timer_duration_period

starting test - test_timer_period_0
PASS - test_timer_period_0

starting test - test_timer_expirefn_null
PASS - test_timer_expirefn_null

starting test - test_timer_periodicity
PASS - test_timer_periodicity

starting test - test_timer_status_get
PASS - test_timer_status_get

starting test - test_timer_status_get_anytime
PASS - test_timer_status_get_anytime

starting test - test_timer_status_sync
PASS - test_timer_status_sync

starting test - test_timer_k_define
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:64: duration_expire: interval >= 100 is false
FAIL - test_timer_k_define

starting test - test_timer_user_data
PASS - test_timer_user_data

starting test - test_timer_remaining
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:548: test_timer_remaining: (exp_ticks - now) <= dur_ticks / 2 is false
FAIL - test_timer_remaining

starting test - test_timeout_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:561: test_timeout_abs: t2.ticks == t.ticks is false
FAIL - test_timeout_abs

Test suite timer_api failed.

PROJECT EXECUTION FAILED

Environment (please complete the following information):

• OS: fedora28
• Toolchain: zephyr-sdk-0.11.1
• Commit ID: 4171220

[andyross]

The issue with test_timeout_abs is my messup. The test is doing an imprecise double-conversion. Will fix.

[chen-png]

for mec board, this test passed, but for reel board, it still failed.

starting test - test_timer_remaining
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:547: test_timer_remaining: rem_ticks <= dur_ticks / 2 is false
FAIL - test_timer_remaining

starting test - test_timeout_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:591: test_timeout_abs: k_timer_remaining_ticks(&remain_timer) + k_uptime_ticks() + 1 == exp_ticks is false
FAIL - test_timeout_abs

Test suite timer_api failed.

[andyross]

Can someone recheck the reel_board? There was an unrelated fix on nRF that went in a week or three ago that I'm guessing is what was reported here.

[chen-png]

tested it with the latest commit ed2d263 on reel_board, it still was the same error.

[andyross]

Oh wait, I'm sorry, I was confusing fixes. This failure on my nrf5 board was the one tracked in #24784, and it's not fixed yet. Can you check to see that one of the workarounds listed there resolves this?

I think we can close this as a dup, if so.

[chen-png]

tested the patch #24817 and #24972 in #24784, the test unit 'test_timer_k_define' and 'test_timer_remaining' passed, but the 'test_timeout_abs' still failed like below.

starting test - test_timeout_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:591: test_timeout_abs: k_timer_remaining_ticks(&remain_timer) + k_uptime_ticks() + 1 == exp_ticks is false
FAIL - test_timeout_abs

[pabigot]

I can confirm that the above and other checks in this test fail on reel_board with varying frequency.
As @andyross suggests this is likely due to #24784. All the failing tests are comparing durations calculated from HFCLK (k_busy_wait()) against durations calculated from LFCLK (sys_clock i.e. timeout/k_timer). In all cases the actual error is one tick the wrong side of the check.

Using a 1pps signal from a Maxim DS3231 TCXO the reel board 32 KiHz LFXT runs about 60 ppm fast on both the reel boards I have, compared to between 25-30 ppm for most Nordic dev boards I've tested. (@jfischer-phytec-iot fyi, not sure this is known.) This is probably why the out-of-spec accuracy checks fail more frequently on this platform.

I still feel that Zephyr cannot assume that the clock used for k_busy_wait() maintains a stable and precise relationship with the clock used for sys_clock, and that as a result these tests are inherently not portable.

If the tests must pass, there are several paths:

• The expedient and misleading one is to test Nordic hardware with a non-standard Nordic clock configuration that hides the error by locking LFCLK to HFCLK. Thid can be done quickly and should only affect Nordic hardware, but it misrepresents the behavior of that hardware in normal applications.
• The justifiable but complex and fragile one is to refactor the tests to compare the measured error against a target-specific tolerance, which for Nordic could be up to 13%. This is very complex and might break on other platforms that use significantly higher or lower ticks-per-second values that complicate accurate calculation of the clock error.
• The expedient and justifiable one is to manually tweak the thresholds again.

If the goal is to get medium bugs below a threshold a fourth path is to close this as a duplicate of #24784, and leave that unresolved for now.

Unless I hear otherwise I'll tweak the thresholds again until the test passes reliably on the two reel boards I have. @carlescufi

[jfischer-no]

Using a 1pps signal from a Maxim DS3231 TCXO the reel board 32 KiHz LFXT runs about 60 ppm fast on both the reel boards I have, compared to between 25-30 ppm for most Nordic dev boards I've tested. (@jfischer-phytec-iot fyi, not sure this is known.) This is probably why the out-of-spec accuracy checks fail more frequently on this platform.

Try to find out what is wrong. It should not be far from 20ppm. I can confirm that the test is failing on the reel board, even on the first version, where the layout is similar to nRF52804 DK.