[Review wanted] Async greentea-client support

[PrzemekWirkus]

Description

Changes related to async feature bundle:

• test_env macros simplified.
• Added custom lexer/parser for KiVi (key-value) protocol, see: {{key;value}}.
  • Used to capture master (host) responses or protocol flow (e.g. handshake is started by master/host).
• Added handshake: sync support.
• Added slave side API to send KiVi messages to master (host).
• Ported most of the tests:
  • timers, tickers, rtc, echo etc.

Corresponding changes

Corresponding changes in mbedhtrun are here: ARMmbed/htrun#65
Note that for now Greentea doesn't support new mbedhtrun scheme but I will add it.

Key-value protocol (KiVi)

• Format {{KEY;VALUE}}}.
• Master-slave mode where host is master and DUT is slave.
• Transport protocol KEYs in use:
  • sync - handshake, sync event.
  • end - end of test suite execution event.
  • exit - last event from DUT (Between end and exit LCOV events may appear).
  • coverage_start - LCOV events.
  • testcase_start & testcase_finish - test case mark events.

New test_env implementation

• See test_env.cpp
• See test_env.h

New macros

• See test_env.h

Note: This will be in the future extracted to separate module e.g. greentea-client. C++ Wrapper will be added.

Naming convention

• test suite - a binary containing test cases we execute on hardware.
• test case - equivalent of a test case in utest.
  • New macros introduced for test case mark:

DUT test case idiom

void app_start(int, char*[]) {
    bool result = true;
    GREENTEA_START();
    GREENTEA_SETUP(15, "wait_us_auto");

    // Test suite body

    GREENTEA_TCASE_START("MINAR001");
    // MINAR0001 test case body
    int success = 0;  // success == 0
    GREENTEA_TCASE_FINISH("MINAR001", success);

    GREENTEA_TSUITE_RESULT(result);
}

TODO

• Add TIMEOUT support
• Bindings with Greentea (lcvo, test cases, and --digest support)
• Port more tests like: serial_interrupt, detect etc.
• Port socket-test (e.g. tcp-client) with new async APIs.

Example implementation

• Basic basic test and default_auto.
• Timers ticker_2 and wait_us_auto.
• RTC rtc test and rtc_auto.

[0xc0170]

Note: This will be in the future extracted to separate module e.g. greentea-client. C++ Wrapper will be added.

Before we go further with a review, are these changes considered to be breaking? I am asking mainly because the test env files should be moved as a note quoted above.

Looking at the changes they are breaking. We should sync about this, how to proceed (move test files plus this update).

[autopulated]

How does this relate to https://github.com/ARMmbed/utest?

[niklarm]

The default greentea handlers in utest will need to be updated to reflect the changes here.
All written test should then just work fine.

[niklarm]

Macros? There are no macros where we're going.
They obscure type information.

[niklarm]

Also, could we please spare the three more letters and call this GREENTEA_TESTCASE_*?

[niklarm]

Apart from some cosmetic changes, 👍 from me.
The above summary has some mistakes:

• {{{key;value}}} six braces?
• GREENTEA_TCASE_START(MINAR001); no strings?

[PrzemekWirkus]

@0xc0170

Before we go further with a review, are these changes considered to be breaking? I am asking mainly because the test env files should be moved as a note quoted above.

This is breaking everything, tests in yotta modules and host tests.

[PrzemekWirkus]

I'm closing this PR because it is obsolete and was used to present Async feature idea.
Please refer to ARMmbed/greentea#78 for more details.