tests/cpu_esp: tests for ESP specific modules #12971

gschorcht · 2019-12-17T06:09:06Z

Contribution description

A number of optional features of ESP boards have to be explicitly activated by enabling the corresponding modules. However, because these modules are not enabled by default, the CI compilation test cannot detect potential compilation problems. As a result, PRs can be merged that cause compilation problems when an optional ESP feature is enabled through the corresponding module.

To make it possible for CI compilation tests to recognize potential compilation problems when optional ESP features are enabled, a number of specific tests are provided by this PR. Some of them are only for compilation tests, but others can also be used to test the optional feature.

cpu_esp_eth:
Compile test for module esp_eth using a shell-based GNRC networking application for ESP32 boards that provides feature esp_eth.
cpu_esp_hw_counter:
Uses tests/periph_timer with hardware counters on ESP32 boards (module esp_hw_counter).
cpu_esp_i2c_hw:
Uses tests/periph_i2c with hardware I2C on ESP32 boards (module esp_i2c_hw).
cpu_esp_heap:
Uses tests/malloc with SDK heap on ESP8266/ESP32 boards (module esp_idf_heap) and the SPI RAM (module esp_spi_ram) for ESP32 boards that provide the feature esp_spi_ram.
cpu_esp_log:
Uses tests/shell with enabled startup information (module esp_log_startup) and tagged output format (module esp_log_tagged).
cpu_esp_wifi:
Compile test for module esp_wifi using a shell-based GNRC networking application .

Testing procedure

Compilation and automatic tests should succeed on Murdock.

Issues/PRs references

Tracked in issue #12960

gschorcht · 2019-12-17T06:15:38Z

@kaspar030 @benpicco What do you think about such tests? That would allow Murdock to detect compilation or even runtime problems for optional ESP features. I know, it is the only CPU that defines so many specific tests 😎 But these tests are necessary to cover all optional features.

benpicco

Sorry, forgot about this.
But yea we should definitely have a compile-test for each driver.

I'm not sure about copying entire tests just to try out another config option, we might get away with less code duplication by just having a (test) board that has these options enabled by default.

benpicco · 2020-03-23T10:39:23Z

tests/cpu_esp_i2c_hw/README.md

+The application corresponds exactly to the application `tests/periph_i2c`.
+It is only for testing the timer implementation using hardware counters.
+In fact, the application is nothing more than the `tests/periph_i2c`
+application compiled with the following command:


What timer implementation gets used on the esp for tests/periph_i2c?

The hardware implementation esp_i2c_hw doesn't require a timer. The bit-banging software implementation uses busy waiting.

Why would you default to bit-banging when there is a hardware i2c?

Because it is buggy, instable and slower than the software implementation.

benpicco · 2020-03-23T10:42:22Z

tests/cpu_esp_hw_counter/README.md

+The application corresponds exactly to the application `tests/periph_timer`.
+It is only for testing the timer implementation using hardware counters.
+In fact, the application is nothing more than the `tests/periph_timer`
+application compiled with the following command:


Instead of copying every test to have a version with hardware counters, why not add a test-esp board that has

USEMODULE += esp_hw_counter

What is used by default instead? When would the user chose to use hardware counters?

ESP32 has two counters and four 64-bit timers that can be used as periph_timer. By default, the 64-bit timers are used.

Not a single test application - just use the existing tests tests/periph_i2c, tests/periph_timer but have a board that uses the other timer / i2c implementation by default.
At least that's how I understand the problem.

But I wouldn't like to expose this test board in the boards directory.

Maybe adding an external board that lives inside the test works.
I'm not sure what the latest state of compatibility with external boards and in-tree boards was, but if that works it would probably be the cleanest solution.

benpicco · 2020-03-23T10:45:01Z

Also please rebase.

gschorcht

I'm not sure about copying entire tests just to try out another config option, we might get away with less code duplication by just having a (test) board that has these options enabled by default.

You mean we should have a single test application that contains a test board definition with all required functions and all modules enabled? Tricky, but really a good idea. I really like hacks like this 🕶️

benpicco · 2020-03-23T12:05:31Z

Not a single test application - just use the existing tests tests/periph_i2c, tests/periph_timer but have a board that uses the other timer / i2c implementation by default.
At least that's how I understand the problem.

benpicco · 2020-08-07T14:22:05Z

Please rebase

boards/common/esp8266/Makefile.features

The test is just a copy of tests/periph_timer but uses the hardware counter for timers (module esp_hw_counter).

The test is just a copy of tests/periph_i2c but uses the hardware module for I2C (module esp_i2c_hw).

The test is just a copy of tests/malloc but uses the heap of the SDK (module esp_idf_heap) instead of the heap provided by the newlib. If feature esp_spi_ram is provided, the module esp_esp_ram is enabled. Thus the test is good for testing modules esp_idf_heap and esp_spi_ram.

The application corresponds exactly to the application `tests/shell`. It is only for testing the ESP-specific logging functions enabled by the `esp_log_startup` and `esp_log_tagged` modules.

gschorcht · 2020-08-07T14:35:05Z

Rebased, but I just saw that the discussion about a virtual test board got stuck when the online semester came. I'm sorry, the PR seems to be not finished yet. esp_wifi_enterprise is also not covered yet.

fjmolinas · 2020-09-11T07:17:50Z

Rebased, but I just saw that the discussion about a virtual test board got stuck when the online semester came. I'm sorry, the PR seems to be not finished yet. esp_wifi_enterprise is also not covered yet.

I don't think we should block this because esp_wifi_enterprise is not covered. Regarding the virtual board, you could do the same as tests/external_boards_dir, I can help with that if you need help :).

stale · 2021-03-19T23:33:09Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If you want me to ignore this issue, please mark it with the "State: don't stale" label. Thank you for your contributions.

gschorcht · 2021-03-20T05:36:27Z

Still in progress.

benpicco · 2021-12-01T13:24:45Z

Since there seems to be no good other way right now, let's get this in as is.
Please rebase to update the CI configuration.

gschorcht · 2021-12-01T13:37:40Z

I have removed the CI: ready for build label because I think I can further reduce the number of these applications a bit. If this is the only way for now, I will rework the PR a bit.

gschorcht · 2021-12-13T19:33:35Z

Closed in favor of #17314.

gschorcht force-pushed the cpu/esp/tests branch from a00a72c to cc3d03e Compare December 17, 2019 06:32

gschorcht removed the CI: run tests If set, CI server will run tests on hardware for the labeled PR label Dec 17, 2019

gschorcht force-pushed the cpu/esp/tests branch 2 times, most recently from 140c359 to 26fecc9 Compare December 17, 2019 08:17

gschorcht added the CI: run tests If set, CI server will run tests on hardware for the labeled PR label Dec 17, 2019

gschorcht mentioned this pull request Jan 26, 2020

[TRACKING] cpu/esp*: compile tests for ESP specific modules required #12960

Open

19 tasks

gschorcht force-pushed the cpu/esp/tests branch from 26fecc9 to 8f05d30 Compare March 10, 2020 13:12

gschorcht requested review from aabadie, fjmolinas, leandrolanzieri, MichelRottleuthner, miri64 and smlng as code owners March 10, 2020 13:12

gschorcht mentioned this pull request Mar 23, 2020

cpu/esp32: support for light/deep sleep and pm_layered #13416

Merged

4 tasks

benpicco reviewed Mar 23, 2020

View reviewed changes

gschorcht commented Mar 23, 2020

View reviewed changes

gschorcht mentioned this pull request Aug 7, 2020

cpu/esp32: remove the dependency of vendor code on BOARDDIR #14729

Merged

benpicco reviewed Aug 7, 2020

View reviewed changes

boards/common/esp8266/Makefile.features Outdated Show resolved Hide resolved

benpicco reviewed Aug 7, 2020

View reviewed changes

boards/common/esp8266/Makefile.features Outdated Show resolved Hide resolved

boards/esp8266: add common features for tests

01aeff7

gschorcht added 6 commits August 7, 2020 16:29

tests/cpu_esp: compile test for esp_wifi netdev

387f9d4

tests/cpu_esp32: compile test for esp_eth netdev

d8f12c1

tests/cpu_esp32: test for periph/timer using esp_hw_counter

9d874cd

The test is just a copy of tests/periph_timer but uses the hardware counter for timers (module esp_hw_counter).

tests/cpu_esp32: test for periph/i2c using esp_i2c_hw

c10b0f6

The test is just a copy of tests/periph_i2c but uses the hardware module for I2C (module esp_i2c_hw).

tests/cpu_esp: test for esp_log_startup and esp_log_tagged

10d6a21

The application corresponds exactly to the application `tests/shell`. It is only for testing the ESP-specific logging functions enabled by the `esp_log_startup` and `esp_log_tagged` modules.

gschorcht force-pushed the cpu/esp/tests branch from 8f05d30 to 10d6a21 Compare August 7, 2020 14:34

stale bot added the State: stale State: The issue / PR has no activity for >185 days label Mar 19, 2021

gschorcht added State: WIP State: The PR is still work-in-progress and its code is not in its final presentable form yet and removed State: stale State: The issue / PR has no activity for >185 days labels Mar 20, 2021

MrKevinWeiss added this to the Release 2021.07 milestone Jun 22, 2021

MrKevinWeiss removed this from the Release 2021.07 milestone Jul 15, 2021

gschorcht mentioned this pull request Dec 1, 2021

cpu/esp32: esp_wifi_enterprise compilation error #17305

Closed

benpicco added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Dec 1, 2021

benpicco requested a review from maribu December 1, 2021 13:22

gschorcht removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR CI: run tests If set, CI server will run tests on hardware for the labeled PR labels Dec 1, 2021

gschorcht mentioned this pull request Dec 1, 2021

cpu/esp: compile optional modules in CI #17314

Merged

gschorcht closed this Dec 13, 2021

gschorcht deleted the cpu/esp/tests branch December 13, 2021 19:33

gschorcht mentioned this pull request Jan 6, 2022

board/*/kconfig: Fix boards on kconfig blocklist #17467

Merged

fjmolinas mentioned this pull request Jan 17, 2022

tests/external-boards: add esp compile test boards #17525

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests/cpu_esp: tests for ESP specific modules #12971

tests/cpu_esp: tests for ESP specific modules #12971

gschorcht commented Dec 17, 2019

gschorcht commented Dec 17, 2019

benpicco left a comment

benpicco Mar 23, 2020

gschorcht Mar 23, 2020

benpicco Mar 23, 2020

gschorcht Mar 23, 2020

benpicco Mar 23, 2020

gschorcht Mar 23, 2020

gschorcht Mar 23, 2020

benpicco Mar 23, 2020

benpicco commented Mar 23, 2020

gschorcht left a comment

benpicco commented Mar 23, 2020

benpicco commented Aug 7, 2020

gschorcht commented Aug 7, 2020

fjmolinas commented Sep 11, 2020

stale bot commented Mar 19, 2021

gschorcht commented Mar 20, 2021

benpicco commented Dec 1, 2021

gschorcht commented Dec 1, 2021

gschorcht commented Dec 13, 2021

tests/cpu_esp: tests for ESP specific modules #12971

tests/cpu_esp: tests for ESP specific modules #12971

Conversation

gschorcht commented Dec 17, 2019

Contribution description

Testing procedure

Issues/PRs references

gschorcht commented Dec 17, 2019

benpicco left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benpicco commented Mar 23, 2020

gschorcht left a comment

Choose a reason for hiding this comment

benpicco commented Mar 23, 2020

benpicco commented Aug 7, 2020

gschorcht commented Aug 7, 2020

fjmolinas commented Sep 11, 2020

stale bot commented Mar 19, 2021

gschorcht commented Mar 20, 2021

benpicco commented Dec 1, 2021

gschorcht commented Dec 1, 2021

gschorcht commented Dec 13, 2021