cortexm_common/ldscript: simplify generation of firmwares in section of the ROM

[cladmi]

Contribution description

This adds/modifies two build system configuration variables for cortexm_common/ldscript/cortexm.ld.

It is now possible to generate during a build, different elf files with different rom offset and firmware rom length by setting ROM_OFFSET and FW_ROM_LEN variables.

• Modify ROM_OFFSET so it can be set as a target specific variable to be used during linking.
• Add a new FW_ROM_LEN variable that can restrict the firmware length to use when linking instead of using the whole rom after ROM_OFFSET.
  • The previous solution was to change ROM_LEN to be ROM_LEN = ROM_OFFSET + FW_ROM_LEN which was not as straightforward to use.

Use cases

The goal is firmwares that do not use the whole ROM for the over the air update scenario with a ROM split in 3 parts "Boot Loader - Firmware 1 - Firmware 2".

Current state

In the test, I currently only set the supported board with a whitelist to iotlab-xx and samr21-xpro nodes. But it should be expandable to all boards using cortexm_common/ldscript/cortexm.ld as linking script.
I can do it depending on your feedback.

There are compilation tests for:

• Generating firmware with 2 different offsets
• Generating a firmware with a size of half the rom
• Overflow detection when setting a too big FW_ROM_LEN

Issues/PRs references

Part of #9342
Depends on #9451

This is a modified version of things inspired from both following branches:

kYc0o https://github.com/kYc0o/RIOT/tree/ota_firmware_updates
kaspar030 https://github.com/kaspar030/RIOT/tree/ota_firmware_updates

I did not re-use the existing commits as it is a different implementation.
Tell me if you want to be mentioned in a commit.

[cladmi]

I re-added the main if to only test if all symbols are defined when one is defined.
It should fix boards that do not use the common cortexm.ld.

I tried doing it in cortexm.ld with ASSERT but the memory section complains about undefined symbol before the ASSERT so the message is invalid length for memory region rom instead of the assert message.

[cladmi]

Murdock broken build:

make: Entering directory '/tmp/dwq.0.9331483276232536/84bce6789fe2daca27b9d89f003b6097/tests/cortexm_common_ldscript'
make: *** No rule to make target '/tmp/dwq.0.9331483276232536/84bce6789fe2daca27b9d89f003b6097/build/auto_init.a', needed by '/tmp/dwq.0.9331483276232536/84bce6789fe2daca27b9d89f003b6097/build/tests_cortexm_common_ldscript_offset_0x1000.elf'.  Stop.
make: *** Waiting for unfinished jobs....

Is caused by #8913 as the rule defining BASELIBS is defined before it has its real value.

I did not see it before as I compile without parallel build in a clean environment.

[cladmi]

I rebased on commits that should handleBASELIBS definition.
It will do a separate PR for them next week.

I also added a debug test in Makefile.include to verify that BASELIBS is indeed stable in the file.

[smlng]

@cladmi this depends on #9451 right?

[cladmi]

Yes, my bad I forgot to update it.

[kYc0o]

I'd like to ACK this PR but I don't have the hardware to test.

Tests are needed for:

• arduino-mkr (any) @aabadie ?
• stm32f103c8 @MrKevinWeiss ?
• feather-m0
• opencm904 @astralien3000 ?
• spark-core

I'd say any of the tests on these boards should validate this PR.

[cladmi]

I will add them to the list of tested boards in tests/cortem_common_ldscript. It does not verify that the code is functionnal, but that at least, the test works as expected.

[MrKevinWeiss]

I tested on the bluepill (using swd flasher), tests/leds make LED0 flash. I also ran the tests/cortem_common_ldscript and got

Test compilation with offset 0x2000: [OK]
Test compilation with half ROM length: [OK]
Test ROM overflow detection: [OK]
Test ROM overflow detection: [OK]

@kYc0o Let me know if you would like anything more extensive.

[kYc0o]

I tested on the bluepill (using swd flasher)

Actually the goal is to keep the bootloader, this PR modifies a bit the way the offset is managed to move the code outside the space allowed for the bootloader, but I can imagine that if you flash with SWD you've already remove the bootloader. Did you change the ROM_OFFSET variable to do so?

[MrKevinWeiss]

Did you change the ROM_OFFSET variable to do so?

Nope, I will reflash the bootloader (sometimes a pain) then attempt to vary the ROM_OFFSET using the tests/cortem_common_ldscript... After the RIOT meeting.

[kYc0o]

then attempt to vary the ROM_OFFSET

The goal is not to modify it, but just test if RIOT flashed with the normal settings still works.

[MrKevinWeiss]

Actually the goal is to keep the bootloader

Hmmm seems like my hardware isn't happy (I cannot use the USB bootloader to program the bluepill). I will try some other ones Monday. I was able to flash with the st-link from the nucleo board or the UART setting it BOOT0 pin to 1.

[MrKevinWeiss]

So here are the things I found... Currently the bluepill, which uses the stm32f103c8 works with swd and RIOT. Using the bootloader to flash is very outdated and full of problems. The bootloader firmware generic_boot20_pc13.bin changed the bootmode pin to B2 (which means the boot1 jumper must be connected to 1). The dfu-util also seems to show 3 instances instead of 1, requiring a serial number to specify the bootloader.

dfu-util: Invalid DFU suffix signature
dfu-util: A valid DFU suffix will be required in a future dfu-util release!!!
dfu-util: More than one DFU capable USB device found! Try `--list' and specify the serial number or disconnect all but one device

Here are the instances with the most recent bootloader

Found DFU: [1eaf:0003] ver=0201, devnum=110, cfg=1, intf=0, alt=2, name="STM32duino bootloader v1.0  Upload to Flash 0x8002000", serial="LLM 003"
Found DFU: [1eaf:0003] ver=0201, devnum=110, cfg=1, intf=0, alt=1, name="STM32duino bootloader v1.0  Upload to Flash 0x8005000", serial="LLM 003"
Found DFU: [1eaf:0003] ver=0201, devnum=110, cfg=1, intf=0, alt=0, name="STM32duino bootloader v1.0  ERROR. Upload to RAM not supported.", serial="LLM 003"

The third thing is it appears to not produce a .bin file which the bootloader requires.

Finally the last problems (though it is more likely my problem) requires it to be run with sudo or the udev rules updated.

Note that I used the dfu-util 0.8 as this is the most recent version for Ubuntu 16.04 LTS

[cladmi]

The third thing is it appears to not produce a .bin file which the bootloader requires.

If you do make BOARD=bluepill PROGRAMMER=dfu-util flash it should create the binfile and use it for flashing.

RIOT/boards/common/stm32f103c8/Makefile.include

Lines 19 to 26 in 9f26d37

	ifeq ($(PROGRAMMER),dfu-util)
	export ROM_OFFSET ?= 0x2000 # Skip the space needed by the embedded bootloader
	export FLASHER = dfu-util
	export DEBUGGER = # no debugger
	export RESET = # dfu-util has no support for resetting the device
	HEXFILE = $(BINFILE)
	export FFLAGS = -d 1d50:6017 -s 0x08002000:leave -D "$(HEXFILE)"

[kYc0o]

I had a arduino-mkr1000 on my drawer, so I could test. Everything works and the bootloader is kept. I'd say then it's safe to merge.

[kYc0o]

However, discovered a bug specific of that target (sam0). It works just by chance but the Makefiles need to be adapted. Can you take a look @cladmi?

[kYc0o]

Basically, the --defsym=_rom_offset=0x2000 variable is correctly set, however there's no logic to subtract that amount from the total size of the ROM. This can lead to no error while compiling large images which wouldn't fit the available ROM.

[cladmi]

@bergzand I will add a README with the expected output.
In the main.c I put the /* The important rules are in the Makefile */ comment but maybe it is not clear enough.

[bergzand]

@bergzand I will add a README with the expected output.

Thanks, I'm afraid that mainly the negative tests might confuse people

[cladmi]

I added a compile-test sub target to mention it in the documentation and a README.md.

[bergzand]

ACK!

@cladmi Thank you for adding the readme. Please squash

[cladmi]

Thank you for adding the readme.

You were right to request having one in the first place :) I just tend to forget if it is not minor changes.

I am re-running the test in our docker image for verification (but pushed before testing of course…)

[cladmi]

Compilation and compile tests works on docker image and arm-none-eabi-gcc 8.1.0.

[bergzand]

@cladmi Your commits show up as unverified in the github webinterface. Are you sure that those are your commits? Should we trust these changes? 😆

[cladmi]

Oh it's because I am from my laptop and did not commit with my fu-berlin address. Will rebase.

[cladmi]

Oh just misconfiguration of the commit gpg key ID I think, will rebase and see.

[kYc0o]

Congrats! One step forward for easy OTA updates 😃