[TRACKING] Fixes for automatic tests of ESP32 boards.

[gschorcht]

Description

Automatic tests for ESP32 didn't work because of problems to reset ESP32 boards using esptool.py. With PR #12752 it will become possible to reset ESP32 boards and to perform automatic tests.

However, a number of tests are not working due to several open problems. This issue tracks the tests that are not working for ESP32 and their fixes.

Open Test Problems

• tests/cond_order (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/heap_cmd (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/log_color (PR cpu/esp32: improvements and cleanup of log_module #12044)
• tests/log_printfnoformat (PR cpu/esp32: improvements and cleanup of log_module #12044)
• tests/mutex_order (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/netstats_l2 (PR cpu/esp*: define esp_now as default netdev #12787)
• tests/periph_timer (PR cpu/esp32: fix xtimer dependency #12757)
• tests/pkg_spiffs (PR cpu/esp32: increase timeout for spiffs and littlefs tests #12758)
• tests/pkg_littlefs (PR cpu/esp32: fix to be able to use SPI flash drive with pkg_littlefs #12803)
• tests/posix_semaphore (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/rmutex_cpp (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/rmutex (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/shell (PR cpu/esp32: fix of thread priorities levels #12753)
• tests/ssp (PR [TRACKING] Fixes for automatic tests of ESP32 boards. #12763)
• tests/sys_arduino (PR +cpu/esp32: workaround for UART problems #12794)

Test stability issues

• sporadic problems with UART clock (PR cpu/esp32: workaround for UART problems #12794)
• sporadic problems system locks with tests/bench_runtime_coreapis

[aabadie]

@gschorcht, there's only one item left in this tracking issue. Do you think this is fixable ? Or maybe it's already fixed ?

[gschorcht]

Unfortunately, the reason is still not really clear. It seems that the system locks when the hardware timer is read while the interrupts are disabled in BENCHMARK_FUNC. When I remove the irq_disable/irq_enable in BENCHMARK_FUNC, everything is fine. Maybe it's a race condition between the hardware timer read operation and a timer interrupt that is pending. I tried to debug it with OCD but when the system locks, OCD is not working anymore because interrupts are not handled anymore, even not high priority interrupts.

BTW, when I was investigating the problem I was wondering whether it is really a good idea to disable all interrupts during the benchmark test. There are platforms that require at least the timer interrupts to get benchmark tests working. For example, the results you get with ATmega boards and the tests/periph_gpio benchmark tests are completely nonsense because timer is not running during the benchmark tests. I know, ISRs must not affect the benchmark results. But if the timer interrupt is required to keep the timer working, disabling the interrupts is counter productive.