Unit testing impossible for frameworks including Unity themselves #3980

maxgerhardt · 2021-06-19T11:09:50Z

What kind of issue is this?

[X ] PlatformIO Core.
If you’ve found a bug, please provide an information below.

You can erase any parts of this template not applicable to your Issue.

Configuration

Operating system: Windows 10 x64

PlatformIO Version (platformio --version): version 5.2.0a6

Description of problem

The PlatformIO "test" procedure is such that PlatformIO adds its own Unity library to the build.

platformio-core/platformio/builder/tools/piomisc.py

Lines 353 to 361 in 834c7b0

    
           def ConfigureTestTarget(env): 
        
               env.Append( 
        
                   CPPDEFINES=["UNIT_TEST", "UNITY_INCLUDE_CONFIG_H"], 
        
                   CPPPATH=[os.path.join("$BUILD_DIR", "UnityTestLib")], 
        
               ) 
        
               unitylib = env.BuildLibrary( 
        
                   os.path.join("$BUILD_DIR", "UnityTestLib"), get_core_package_dir("tool-unity") 
        
               ) 
        
               env.Prepend(LIBS=[unitylib])

this however directly conflicts with any framework which already has Unity compiled into it. Such is the case with the https://github.com/arduino/ArduinoCore-mbed in which the precompiled mbed-os library (libmbed.a) has the Unity object files.

Thus it is impossible to e.g. run unit tests with a board = nano33ble framework = arduino configuration without modifications.

Refer to the community topic here

Steps to Reproduce

Create a new Arduino Nano 33 BLE project
Add test\main.cpp from below
Run the "Test" project task

Actual Results

A bazillion linker errors due a conflict with .pio\build\nano33ble\libUnityTestLib.a and .platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a

c:/users/max/.platformio/packages/[email protected]/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o): in function `UnityBegin':
unity.c:(.text.UnityBegin+0x0): multiple definition of `UnityBegin'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):unity.c:(.text.UnityBegin+0x0): first defined here
c:/users/max/.platformio/packages/[email protected]/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o): in function `UnityEnd':
unity.c:(.text.UnityEnd+0x0): multiple definition of `UnityEnd'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):unity.c:(.text.UnityEnd+0x0): first defined here
c:/users/max/.platformio/packages/[email protected]/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o):(.rodata.UnityStrErr64+0x0): multiple definition of `UnityStrErr64'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):(.rodata.UnityStrErr64+0x0): first defined here
c:/users/max/.platformio/packages/[email protected]/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o):(.rodata.UnityStrErrDouble+0x0): multiple definition of `UnityStrErrDouble'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):(.rodata.UnityStrErrDouble+0x0): first defined here
c:/users/max/.platformio/packages/[email protected]/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o):(.rodata.UnityStrErrFloat+0x0): multiple definition of `UnityStrErrFloat'; .pio\build\nano33ble\libUnityTestLib.a(unity.c.o):(.rodata.UnityStrErrFloat+0x0): first defined here
c:/users/max/.platformio/packages/[email protected]/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld.exe: warning: size of symbol `Unity' changed from 132 in .pio\build\nano33ble\libUnityTestLib.a(unity.c.o) to 160 in C:\Users\Max\.platformio\packages\framework-arduino-mbed\variants\ARDUINO_NANO33BLE\libs\libmbed.a(unity.o)
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\nano33ble\firmware.elf] Error 1

Expected Results

Running unit tests is possible

If problems with PlatformIO Build System:

The content of platformio.ini:

[env:nano33ble]
platform = nordicnrf52
board = nano33ble
framework = arduino

Source file to reproduce issue:
test\main.cpp

#include <Arduino.h>
#include <unity.h>

void simple_test(void) {
    TEST_ASSERT_EQUAL(33, 33);
}

void setup() {
    delay(2000);

    UNITY_BEGIN();
    RUN_TEST(simple_test);
    UNITY_END();
}

void loop() {
    delay(1000);
}

Additional info

The text was updated successfully, but these errors were encountered:

valeros · 2021-09-28T10:39:50Z

Hi @maxgerhardt ! Thanks for reporting. Indeed, the reasons for this behavior are obvious, but I'm not sure how we can solve this problem in a reasonable way. We cannot simply omit our .pio\build\nano33ble\libUnityTestLib.a in favor of the Unity located in mbed.a. We need a complete control over the Unity configuration, this is the main reason why PlatformIO also adds several additional files that implement standard routines (like redirecting test output, etc.) when compiling the Unity sources. On the other hand, we a bit reluctant to dynamically manipulate the libmbed.a archive and remove all references of the Unity code. I believe the best way out here is to ask the maintainers of ArduinoCore-mbed to exclude Unity from their framework package.

maxgerhardt · 2021-09-28T11:42:17Z

We need a complete control over the Unity configuration, this is the main reason why PlatformIO also adds several additional files that implement standard routines (like redirecting test output, etc.)

There might be special cases where e.g the pre-compiled Unity version is "functionally correct" and would not differ from what PlatformIO would use. Or when there are frameworks which won't remove their internal Unity version. It would make sense to add a fallback option to the platformio.ini to disable using PlatformIO's Unity version completely, like test_use_builtin_unity = no -- the user has to of course understand the implications of this then.

In the Arduino-mbed case (and probably for framework = mbed too?), no adverse consequences were observed by the user when removing the Unity object files from libmbed.a during normal firmware compilation, since the normal firmware does not use Unity, only the unit-testing firmware does. So again, an option to really make PlatformIO remove known Unity object files from the build (while backing up the old file) via an explicit "please do so" configuration option may come in handy if frameworks with precompiled blobs won't remove Unity.

The ArduinoCore-mbed people will surely see a problem with removing Unity since then no Arduino IDE user can use Unity (if they did). They might resolve that though be refactoring Unity into a library and compile it from source. That would be a good alternative.

ivankravets · 2022-05-03T19:07:28Z

Thanks for the report! See docs https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/custom/examples/custom_unity_library.html

pratiksurgeonslab98 · 2022-09-02T12:12:06Z

Custom unity framework not working with mbedOS

I need help with running unit tests on platformio with mbedOS framework, I am using the custom unity test framework.
I am referring to the following links:
custom unity library
#3342 and following the steps mentioned in this issue.

Below is the platformio.ini config file
[env:nucleo_f411re] framework = mbed platform = ststm32 board = nucleo_f411re build_flags = -DPIO_FRAMEWORK_MBED_RTOS_PRESENT -std=gnu++11 build_unflags = -std=gnu++98 test_framework = custom lib_deps = throwtheswitch/Unity@^2.5.2
The test folder looks like this:
/project
├── platformio.ini
└── test
├── test_blink
│ └── test_main.cpp
└── test_custom_runner.py

While running the pio test
I am getting the following errors:

Collected 1 tests (test_blink)
Error: Traceback (most recent call last):
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/main.py", line 102, in main
cli() # pylint: disable=no-value-for-parameter
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/cli.py", line 71, in invoke
return super().invoke(ctx)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/cli.py", line 138, in cli
runner = TestRunnerFactory.new(
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/runners/factory.py", line 67, in new
return runner_cls(test_suite, project_config, options)
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/runners/unity.py", line 191, in init
self._tmp_pre_upgrade()
File "/home/roy/.platformio/penv/lib/python3.10/site-packages/platformio/test/runners/unity.py", line 201, in _tmp_pre_upgrade
pkg = lm.get_package(self.EXTRA_LIB_DEPS[0])
TypeError: 'NoneType' object is not subscriptable

============================================================

An unexpected error occurred. Further steps:

Verify that you have the latest version of PlatformIO using
pip install -U platformio command
Try to find answer in FAQ Troubleshooting section
https://docs.platformio.org/page/faq/index.html
Report this problem to the developers
https://github.com/platformio/platformio-core/issues

============================================================

@valeros @ivankravets @maxgerhardt please help me out.

ivankravets · 2022-09-03T09:14:48Z

@pratiksurgeonslab98, thanks, we fixed the development version of PlatformIO Core. Could you re-test with pio upgrade--dev?

pratiksurgeonslab98 · 2022-09-03T09:42:20Z

@ivankravets Thanks for the prompt response.
The above error is solved but the test cases are not getting printed on the terminal of platformio.

This is the present output:

roy@roy-XPS-15-9510:~/SG_Firmware_Gen_I/master/SG_Firmware_Gen_I/Blink_LED$ pio test
Verbosity level can be increased via -v, -vv, or -vvv option
Collected 1 tests

Processing test_blink in nucleo_f411re environment

Building & Uploading...
xPack OpenOCD x86_64 Open On-Chip Debugger 0.11.0+dev (2021-10-16-21:15)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
debug_level: 1

srst_only separate srst_nogate srst_open_drain connect_deassert_srst

target halted due to debug-request, current mode: Thread
xPSR: 0x01000000 pc: 0x080038a0 msp: 0x20020000
** Programming Started **
** Programming Finished **
** Verify Started **
** Verified OK **
** Resetting Target **
shutdown command invoked
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

None of test case results are printed.

maxgerhardt · 2022-09-03T12:25:39Z

Did you press the reset button on the board? What's the exact project?

pratiksurgeonslab98 · 2022-09-03T12:28:05Z

@maxgerhardt yes, i pressed the reset button but still it didn't show any test result.
The test main.cpp file has this code
`#include <mbed.h>
#include <unity.h>

void setUp(void)
{
// set stuff up here
}

void tearDown(void)
{
// clean stuff up here
}

void simple_test(void)
{
TEST_ASSERT_EQUAL(33, 33);
}

int main(int argc, char **argv)
{
printf("Unity Hello\r\n");
UNITY_BEGIN();
RUN_TEST(simple_test);
UNITY_END();
}`

Nothing was printed

maxgerhardt · 2022-09-03T12:32:44Z

Do regular printf()s work in the firmware code? Without adding the code for creating the serial on the right pins? Are you running on a regular Nucleo F411RE? Can you upload the whole project somewhere for inspection?

pratiksurgeonslab98 · 2022-09-03T16:50:59Z

@maxgerhardt

Do regular printf()s work in the firmware code? Without adding the code for creating the serial on the right pins? Are you running on a regular Nucleo F411RE? Can you upload the whole project somewhere for inspection?

Yes regular printf() works on fine on firmware. I am running on Nucleo f411re board

After opening the serial terminal on putty to read stlink serial data, the test is running. The results are displayed, it is coming in a jumbled way like this.

To replicate the steps,

Keep the particular port open on putty
and run the pio -test command on platformio terminal

On platform io terminal, this is the output I am getting:
`Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

Please specify test_port for environment or use global --test-port option.
--------------------- nucleo_f411re:test_blink [ERRORED] Took 5.66 seconds ---------------------

============================================ SUMMARY ============================================
Environment Test Status Duration

nucleo_f411re test_blink ERRORED 00:00:05.658
=========================== 1 test cases: 0 succeeded in 00:00:05.658 ===========================
`

I am also attaching my project file in zipped format for further inspection.
Please note that the fourth test case is expected to fail since

void test_function_calculator_division(void) { TEST_ASSERT_EQUAL(31, 32); }
addition, multiplication, and subtraction is supposed to PASS.

Blink_LED.zip

Thank You.

dudeperf3ct · 2022-10-18T10:59:54Z

I followed the exact steps listed in Custom Unity Library — PlatformIO and tests don't pass on Arduino Nano 33 BLE board.

PlatformIO version: 6.1.5a4

Project structure and content is same for all files as the documentation. The following error comes from running pio test

*********************************************************************************************************************************************************************************************************************************************************************************
If you like PlatformIO, please:
- follow us on Twitter to stay up-to-date on the latest project news > https://twitter.com/PlatformIO_Org
- star it on GitHub > https://github.com/platformio/platformio
- try PlatformIO IDE for embedded development > https://platformio.org/platformio-ide
*********************************************************************************************************************************************************************************************************************************************************************************

Verbosity level can be increased via `-v, -vv, or -vvv` option
Collected 1 tests
Platform Manager: Installing nordicnrf52
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Platform Manager: [email protected] has been installed!

Processing test_blink in nano33ble environment
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Building & Uploading...
Tool Manager: Installing platformio/toolchain-gccarmnoneeabi @ ~1.80201.0
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: [email protected] has been installed!
Tool Manager: Installing platformio/framework-arduino-mbed @ ~3.1.1
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%          
Tool Manager: [email protected] has been installed!
Tool Manager: Installing platformio/tool-sreccat @ ~1.164.0
Downloading  [####################################]  100%          
Unpacking  [####################################]  100%
Tool Manager: [email protected] has been installed!
Tool Manager: Installing platformio/tool-openocd @ ~2.1100.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: [email protected] has been installed!
Tool Manager: Installing platformio/tool-bossac-nordicnrf52 @ ~1.10901.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: [email protected] has been installed!
Tool Manager: Installing platformio/tool-scons @ ~4.40400.0
Downloading  [####################################]  100%
Unpacking  [####################################]  100%
Tool Manager: [email protected] has been installed!



unity.c:(.text.UnityDefaultTestRun+0x0): multiple definition of `UnityDefaultTestRun'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):unity.c:(.text.UnityDefaultTestRun+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o): in function `UnityBegin':
unity.c:(.text.UnityBegin+0x0): multiple definition of `UnityBegin'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):unity.c:(.text.UnityBegin+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o): in function `UnityEnd':
unity.c:(.text.UnityEnd+0x0): multiple definition of `UnityEnd'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):unity.c:(.text.UnityEnd+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o):(.rodata.UnityStrErr64+0x0): multiple definition of `UnityStrErr64'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):(.rodata.UnityStrErr64+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o):(.rodata.UnityStrErrDouble+0x0): multiple definition of `UnityStrErrDouble'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):(.rodata.UnityStrErrDouble+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o):(.rodata.UnityStrErrFloat+0x0): multiple definition of `UnityStrErrFloat'; .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o):(.rodata.UnityStrErrFloat+0x0): first defined here
/home/dudeperf3ct/.platformio/packages/toolchain-gccarmnoneeabi/bin/../lib/gcc/arm-none-eabi/8.2.1/../../../../arm-none-eabi/bin/ld: warning: size of symbol `Unity' changed from 132 in .pio/build/nano33ble/lib0fc/libUnity.a(unity.c.o) to 160 in /home/dudeperf3ct/.platformio/packages/framework-arduino-mbed/variants/ARDUINO_NANO33BLE/libs/libmbed.a(unity.o)
collect2: error: ld returned 1 exit status
*** [.pio/build/nano33ble/firmware.elf] Error 1
Uploading stage has failed, see errors above. Use `pio test -vvv` option to enable verbose output.
--------------------------------------------------------------------------------------------------------------- nano33ble:test_blink [ERRORED] Took 1.29 seconds ---------------------------------------------------------------------------------------------------------------

==================================================================================================================================== SUMMARY ====================================================================================================================================
Environment    Test        Status    Duration
-------------  ----------  --------  ------------
nano33ble      test_blink  ERRORED   00:00:01.285
=================================================================================================================== 1 test cases: 0 succeeded in 00:00:01.285 ===================================================================================================================

maxgerhardt · 2022-10-18T11:15:43Z

Unable to reproduce.

$ pio test --without-uploading --without-testing -vvv
Collected 1 tests (test_blink)

Processing test_blink in nano33ble environment
------------------------------------------------------------------------------------------------------------------------------------------------------------
Building...
CONFIGURATION: https://docs.platformio.org/page/boards/nordicnrf52/nano33ble.html
PLATFORM: Nordic nRF52 (9.4.0) > Arduino Nano 33 BLE
HARDWARE: NRF52840 64MHz, 256KB RAM, 960KB Flash
DEBUG: Current (blackmagic) External (blackmagic, cmsis-dap, jlink)
PACKAGES: 
 - framework-arduino-mbed @ 3.1.1 
 - tool-sreccat @ 1.164.0 (1.64) 
 - toolchain-gccarmnoneeabi @ 1.80201.181220 (8.2.1)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 37 compatible libraries
Scanning dependencies...
No dependencies
Building in test mode
MethodWrapper(["checkprogsize"], [".pio/build/nano33ble/firmware.elf"])
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [==        ]  16.2% (used 42576 bytes from 262144 bytes)
Flash: [=         ]   8.2% (used 81012 bytes from 983040 bytes)

Core: 6.1.5a4, Nordic nRF52: 9.4.0.

Grab the project file, unpack it, and run the same above as me in the CLI. Does it still show an error?

unity_tests.zip

dudeperf3ct · 2022-10-18T12:55:42Z

I was able to run the test using unity_tests.zip.

Is it possible to perform the tests for TFLite library using platformIO? A sample test can be found here.

maxgerhardt · 2022-10-18T17:07:28Z

I was able to run the test using unity_tests.zip.

Then something is wrong in your project. Adding the TFLite library also shouldn't magically pull Unity in. The error message is showing a conflict from the compiled libraries (.pio/libdeps or lib/ folder) and mbed-os (.pio/build/nano33ble/lib0fc/libUnity.a vs ARDUINO_NANO33BLE/libs/libmbed.a.

Do you have Unity in the lib_deps of the platformio.ini or in the lib/ folder of the project? Have you 100% added the custom test runner script at the correct place with the correct content, and additional placed test_framework = custom in the platformio.ini? Please upload the whole project you're having problems with otherwise.

dudeperf3ct · 2022-10-18T17:31:31Z

Removing .pio folder seemed to do the trick. Thanks! It works in my project as well. I have the same content as mentioned in the documentation for both platformio.ini and test folder. I use 2 libraries TFLite and TinyMLx for running ML models on Arduino.

Is it possible to perform the unit testing for TFLite library using platformIO? A sample unit test can be found here or here for reference.

I wonder if GoogleTest Runner can be of help in this case.

nicklasb · 2023-04-07T23:17:48Z

Hm. I am trying running unity as a custom task, however, the tests never "end", in that the results are never returned even through UNITY_END(); has been called.
I am following the example here:
https://docs.platformio.org/en/latest/advanced/unit-testing/frameworks/custom/examples/custom_unity_library.html

I am running the tests on a RasperryPi Pico W.
Even if I remove all the tests, and only runs the test framework, nothing is printed to output.
It does seem as it sort of ends though, as it becomes possible to update the pico without disconnecting the USB, which is not possible if it has crashed.
But there is no print output at all except for the things my code outputs.

Thing is that it works on all other platforms, Native, pure arduino, ESP-IDF, and so on.
(interestingly the pico platform seems more picky when it comes to some pointer constructs, had to change to memcpy all over)

Config:
[env:RaspberryPi_Pico]
platform = raspberrypi
board = pico
framework = arduino
test_framework= custom

nicklasb · 2023-04-07T23:57:03Z

Tried with the Earle Philhower core as well, alas, no luck, no unity-output.

maxgerhardt · 2023-04-08T13:21:12Z

This issue is getting kind of flooded with different problems that may or not be related to the original problem. For the Pico, if you use the default USB serial, you might be running into the problem that PlatformIO choses the wrong serial port because the Pico has not yet initialized, like #3742.

I'll test if I can reproduce this.

maxgerhardt · 2023-04-08T13:41:13Z

A few observations: Using test code of

#include <Arduino.h>
#include <unity.h>

void setUp(void) { }
void tearDown(void) { }

int addition_function_under_test(int a, int b)
{
    return a + b;
}

void test_calculator_addition(void) {
    TEST_ASSERT_EQUAL(32, addition_function_under_test(25, 7));
}

void setup() {
    // wait until serial port has connected
    while(!Serial) {}
    Serial.println("UNIT TEST START!!");
    UNITY_BEGIN();
    RUN_TEST(test_calculator_addition);
    UNITY_END();
}

void loop() {
    digitalWrite(LED_BUILTIN, HIGH);
    delay(100);
    digitalWrite(LED_BUILTIN, LOW);
    delay(500);
}

and opening a CLI with pio test --without-testing -vvv, the resulting firmware does open a serial port, that when opened in e.g. hterm, and the DTR button is pressed (otherwise no output), the output "UNIT TEST START!!" does appear, but no further. However, the blink sequence in the loop() does trigger, meaning it did run through successfully.

This leads to the conclusion that the Unity version included in ArduinoCore-mbed for the Pico is not outputting its content to the USB serial port at least, maybe the hardware UART, or not all.

Not tested with Arduino-Pico yet, but in general that should be less of a headache since it shouldn't include Unity.

maxgerhardt · 2023-04-08T13:47:13Z

Solved for ArduinoCore-mbed: Their Unity version is working once you add a magic declaration that stdio output is supposed to go to Serial, the USB-serial.

See arduino/ArduinoCore-mbed#174

So, using

#include <Arduino.h>
#include <unity.h>

REDIRECT_STDOUT_TO(Serial);

at the top of the test file makes it work with ArduinoCore-mbed.

maxgerhardt · 2023-04-08T14:03:23Z

And as expected, Arduino-Pico (Earlephilhower) put up no fight at all, working instantly.

I've created a project that is compatible to be tested with both cores, just using a #ifdef ARDUINO_ARCH_MBED check in the ArduinoCore-mbed case.

https://github.com/maxgerhardt/pio-pico-unity-testing

==================== SUMMARY ==================== 
Environment          Test           Status    Duration
-------------------  -------------  --------  ------------
mbed_pico            test_embedded  PASSED    00:00:08.153
earlephilhower_pico  test_embedded  PASSED    00:00:08.138
=== 2 test cases: 2 succeeded in 00:00:16.291 ===

nicklasb · 2023-04-08T16:53:48Z

WRT:

REDIRECT_STDOUT_TO(Serial);
This worked for me, or rather it solve my problem because I use platform = raspberrypi.
However, if i changed to:

platform = https://github.com/maxgerhardt/platform-raspberrypi.git
board_build.core = earlephilhower

...I got a declared out of scope for REDIRECT_STDOUT_TO(Serial).

Uploading not working properly, that is also only a problem I have on earlephilhower.

Thanks!!

adowg67 · 2024-03-20T14:07:45Z

I sure would like this to work as " EASY as its says it is" _ am stuck and I am certain it is NOT becuase I do not want to learn this waste of time programming crap . I am not wasting my remaining time or life doing this dum shit. Especially when it is not me . this stupid vs code is what is so dysfunctional and insecure, and got all kinds of glitchs and malfunctions . thats why it never works the same was no matter how many times i open it. its always some different crap . also now why do i have to download this same crap 20 X per day ????? just make it so I can compile my firmware for my 3D printer without all the mental gymnaswtics would you please???_

nicklasb · 2024-03-20T19:49:23Z

I sure would like this to work as " EASY as its says it is" _ am stuck and I am certain it is NOT becuase I do not want to learn this waste of time programming crap . I am not wasting my remaining time or life doing this dum shit. Especially when it is not me . this stupid vs code is what is so dysfunctional and insecure, and got all kinds of glitchs and malfunctions . thats why it never works the same was no matter how many times i open it. its always some different crap . also now why do i have to download this same crap 20 X per day ????? just make it so I can compile my firmware for my 3D printer without all the mental gymnaswtics would you please???_

I am afraid we can't.
Must say, this was probably the most impressively out-of-topic outburst I have ever seen, and I've seen a few. :-)

Feeling your pain, though, I would recommend you to share this rant with the manufacturer of your 3D-printer instead of posting it here. It should not be that complicated.

SebastianVivoverse · 2024-05-23T20:08:30Z

I am still having the exact same issue as the OP for Arduino Nano 33 BLE.

ivankravets added help wanted unit testing labels Sep 13, 2021

ivankravets assigned valeros Sep 13, 2021

ivankravets added this to the 5.2.1 milestone Sep 13, 2021

ivankravets added feature and removed help wanted labels Sep 29, 2021

ivankravets modified the milestones: 5.2.1, 5.3.0 Sep 29, 2021

ivankravets modified the milestones: 5.4, Backlog Feb 9, 2022

torntrousers mentioned this issue Apr 9, 2022

rp2040 connect with unity tests fails to build platformio/platform-raspberrypi#33

Open

ivankravets mentioned this issue Apr 9, 2022

MBed tests do not print to serial #3342

Closed

1 task

ivankravets modified the milestones: Backlog, 5.3.0 Apr 9, 2022

ivankravets added the docs label Apr 22, 2022

ivankravets unassigned valeros Apr 22, 2022

sguarin mentioned this issue Apr 28, 2022

Multiple tests defined in platformio.ini not running on remote agent #4204

Closed

ivankravets closed this as completed in e81d83b May 3, 2022

ivankravets mentioned this issue May 16, 2022

platformio 6.0.0 Homebrew/homebrew-core#101681

Closed

6 tasks

ivankravets added a commit that referenced this issue Sep 3, 2022

Remove temporary code // Issue #3980

0b4aedb

maxgerhardt mentioned this issue May 3, 2023

undefined reference to `i2c_start' when trying to use mbed I2C maxgerhardt/platform-raspberrypi#28

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unit testing impossible for frameworks including Unity themselves #3980

Unit testing impossible for frameworks including Unity themselves #3980

maxgerhardt commented Jun 19, 2021 •

edited

Loading

valeros commented Sep 28, 2021

maxgerhardt commented Sep 28, 2021

ivankravets commented May 3, 2022

pratiksurgeonslab98 commented Sep 2, 2022

ivankravets commented Sep 3, 2022

pratiksurgeonslab98 commented Sep 3, 2022

maxgerhardt commented Sep 3, 2022

pratiksurgeonslab98 commented Sep 3, 2022

maxgerhardt commented Sep 3, 2022

pratiksurgeonslab98 commented Sep 3, 2022 •

edited

Loading

dudeperf3ct commented Oct 18, 2022 •

edited

Loading

maxgerhardt commented Oct 18, 2022

dudeperf3ct commented Oct 18, 2022 •

edited

Loading

maxgerhardt commented Oct 18, 2022

dudeperf3ct commented Oct 18, 2022 •

edited

Loading

nicklasb commented Apr 7, 2023

nicklasb commented Apr 7, 2023

maxgerhardt commented Apr 8, 2023

maxgerhardt commented Apr 8, 2023 •

edited

Loading

maxgerhardt commented Apr 8, 2023

maxgerhardt commented Apr 8, 2023 •

edited

Loading

nicklasb commented Apr 8, 2023

adowg67 commented Mar 20, 2024

nicklasb commented Mar 20, 2024 •

edited

Loading

SebastianVivoverse commented May 23, 2024

Unit testing impossible for frameworks including Unity themselves #3980

Unit testing impossible for frameworks including Unity themselves #3980

Comments

maxgerhardt commented Jun 19, 2021 • edited Loading

Configuration

Description of problem

Steps to Reproduce

Actual Results

Expected Results

If problems with PlatformIO Build System:

Additional info

valeros commented Sep 28, 2021

maxgerhardt commented Sep 28, 2021

ivankravets commented May 3, 2022

pratiksurgeonslab98 commented Sep 2, 2022

Custom unity framework not working with mbedOS

ivankravets commented Sep 3, 2022

pratiksurgeonslab98 commented Sep 3, 2022

Processing test_blink in nucleo_f411re environment

maxgerhardt commented Sep 3, 2022

pratiksurgeonslab98 commented Sep 3, 2022

maxgerhardt commented Sep 3, 2022

pratiksurgeonslab98 commented Sep 3, 2022 • edited Loading

dudeperf3ct commented Oct 18, 2022 • edited Loading

maxgerhardt commented Oct 18, 2022

dudeperf3ct commented Oct 18, 2022 • edited Loading

maxgerhardt commented Oct 18, 2022

dudeperf3ct commented Oct 18, 2022 • edited Loading

nicklasb commented Apr 7, 2023

nicklasb commented Apr 7, 2023

maxgerhardt commented Apr 8, 2023

maxgerhardt commented Apr 8, 2023 • edited Loading

maxgerhardt commented Apr 8, 2023

maxgerhardt commented Apr 8, 2023 • edited Loading

nicklasb commented Apr 8, 2023

adowg67 commented Mar 20, 2024

nicklasb commented Mar 20, 2024 • edited Loading

SebastianVivoverse commented May 23, 2024

maxgerhardt commented Jun 19, 2021 •

edited

Loading

pratiksurgeonslab98 commented Sep 3, 2022 •

edited

Loading

dudeperf3ct commented Oct 18, 2022 •

edited

Loading

dudeperf3ct commented Oct 18, 2022 •

edited

Loading

dudeperf3ct commented Oct 18, 2022 •

edited

Loading

maxgerhardt commented Apr 8, 2023 •

edited

Loading

maxgerhardt commented Apr 8, 2023 •

edited

Loading

nicklasb commented Mar 20, 2024 •

edited

Loading