Mbed OS components as OBJECT libs #21

0xc0170 · 2021-02-23T22:32:10Z

Summary of changes

Open for discussion

The only target tested is MAX32620FTHR due to it's simple tree structure (easy to adjust).

Impact of changes

Migration actions required

Documentation

Pull request type

[] Patch update (Bug fix / Target update / Docs update / Test update / Refactor)
[] Feature update (New feature / Functionality change / New API)
[] Major update (Breaking change E.g. Return code change / API behaviour change)

Test results

[] No Tests required for this change (E.g docs only update)
[] Covered by existing mbed-os tests (Greentea or Unittest)
[] Tests / results supplied as part of this PR

Reviewers

multiplemonomials · 2021-02-24T04:49:21Z

Thanks for taking a look at this!

I'm not quite 100% up to date with the target refactor stuff but I think I get most of what's going on.

Is this how it currently is?

Target	Function	User app links to it?
mbed-core	Contains flags from Mbed baremetal. All Mbed modules link to this.	No
mbed-core-obj	Builds objects for Mbed baremetal	No
mbed-rtos	Contains flags that turn Mbed baremetal into Mbed OS	No
mbed-rtos-obj	Builds objects for Mbed baremetal plus Mbed OS	No
${MBED_TARGET_CONVERTED}	Contains interface flags and sources that adapt Mbed OS for a target	No
mbed-baremetal	Interface library that attaches Mbed baremetal to a target (objects from mbed-core)	Yes
mbed-os	Interface library that attaches Mbed OS to a target (objects from mbed-rtos)	Yes

As long as this is correct, maybe we should put this table in the comments somewhere because it's very confusing.

object library cyclic deps - not supported in CMake yet, known issue. we got mbed-core and Mbed OS target - both depend on each other. One way would be to use just interface libraries for Mbed OS targets - do some magic to copy sources and expose the rest to mbed-core that needs it and also an application. Or any idea how to break this?

Can't you just keep the targets as they currently are (interface libraries containing flags and sources)? Then just link the targets to both mbed-core-obj and mbed-os-obj. This does lead to more files built but only if someone is using both baremetal and RTOS in the same project, which is kinda rare.

0xc0170 · 2021-02-24T09:16:13Z

Yes the above is correct. The table is very usefull, I'll add it to our Readme for now.

Can't you just keep the targets as they currently are (interface libraries containing flags and sources)?

Yes, I came to the same conclusion. Once I refactored simple maxim target (consist of 3 simple subtargets with few files, compare to others we have). As result there were lot of dependencies. It become hard for me to follow what I should link where. The tree with interface libraries are easy to create and maintain. That is a win for a porting in-tree or custom targets.

This does lead to more files built but only if someone is using both baremetal and RTOS in the same project, which is kinda rare.

That should not happen. I'll make a note about this either in the docs or the toplevel CMake.

I'll continue developing this to test our tree structure if this is the best what we can get :)

ladislas · 2021-02-24T16:03:53Z

@0xc0170 let me know when yo want me to test in my setup, I'll report here.

0xc0170 · 2021-02-24T16:08:44Z

@0xc0170 let me know when yo want me to test in my setup, I'll report here.

I was planning to do. I was able to get blinky and events example snippets working with the maxim. It failed with K64F due symbols from linker file, I'll have to fix that. If you can test with anything and report back, I'll fix.

0xc0170 · 2021-02-25T09:25:44Z

I fixed linker errors, I was missing how linker is dependent on a target library. K64F builds now for both toolchains.

@ladislas Can you test?

ladislas · 2021-02-25T10:54:11Z

Yes, I'll do that after lunch

0xc0170 · 2021-02-25T10:56:30Z

Yes, I'll do that after lunch

I am trying it as well and hit the first problem. Symbols required by linker script are not defined :/ MBED_ROM_START for instance. Using master it builds, but I can't spot where they come from (I would expect from targets.json and would be in the mbed_config file), need to check.

ladislas · 2021-02-25T11:08:43Z

if you -DMBED_ROM_START=0x8000000 does it help?

ladislas · 2021-02-25T22:05:51Z

I'm hitting the same issue when trying to compile for the DISCO_F769.

[2/277] Link line:
FAILED: extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/mbed-stm32g031xx.link_script.ld
cd /Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx && /usr/local/bin/arm-none-eabi-gcc @/Users/ladislas/dev/ladislas/mbed-cmake-template/_build/DISCO_F769NI/extern/mbed-os/compile_time_defs.txt -E -x assembler-with-cpp -P /Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/TOOLCHAIN_GCC_ARM/stm32g031xx.ld -o /Users/ladislas/dev/ladislas/mbed-cmake-template/_build/DISCO_F769NI/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/mbed-stm32g031xx.link_script.ld
In file included from /Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/TOOLCHAIN_GCC_ARM/stm32g031xx.ld:18:
/Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/TOOLCHAIN_GCC_ARM/../cmsis_nvic.h:28:2: error: #error "MBED_ROM_SIZE not defined"
   28 | #error "MBED_ROM_SIZE not defined"
      |  ^~~~~
[3/277] Link line:
FAILED: extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G071xx/mbed-stm32g071xx.link_script.ld
cd /Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G071xx && /usr/local/bin/arm-none-eabi-gcc @/Users/ladislas/dev/ladislas/mbed-cmake-template/_build/DISCO_F769NI/extern/mbed-os/compile_time_defs.txt -E -x assembler-with-cpp -P /Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G071xx/TOOLCHAIN_GCC_ARM/stm32g071xx.ld -o /Users/ladislas/dev/ladislas/mbed-cmake-template/_build/DISCO_F769NI/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G071xx/mbed-stm32g071xx.link_script.ld
In file included from /Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G071xx/TOOLCHAIN_GCC_ARM/stm32g071xx.ld:18:
/Users/ladislas/dev/ladislas/mbed-cmake-template/extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G071xx/TOOLCHAIN_GCC_ARM/../cmsis_nvic.h:31:2: error: #error "MBED_ROM_SIZE not defined"
   31 | #error "MBED_ROM_SIZE not defined"
      |  ^~~~~
[19/277] Link line:
ninja: build stopped: subcommand failed.
make: *** [all] Error 1

ladislas · 2021-02-25T22:16:48Z

But it works for the K64F board.

It also works for multiple add_executable() but we need to change the following in the main CMakeLists.txt:

function(mbed_configure_app_target target)
    # Gcc Arm requires memap to be set with app name, equally to ARMClang
    # TODO: move this to toolchain and set properly
    if(MBED_TOOLCHAIN STREQUAL "GCC_ARM")
-        target_link_options(mbed-core
-            INTERFACE
+        target_link_options(${target}
+            PRIVATE
                "-Wl,-Map=${CMAKE_CURRENT_BINARY_DIR}/${target}${CMAKE_EXECUTABLE_SUFFIX}.map"
        )
    endif()
endfunction()

Without that, only the latest add_executable can be build with as the -Map=... is overridden each time.

I can make a PR just for this change, or do you prefer to make it here?

ladislas · 2021-02-25T22:43:57Z

I've updated my template and made a PR: ladislas/mbed-cmake-template#7

To try:

# clone repo
git clone https://github.com/ladislas/mbed-cmake-template

# checkout branch
git checkout --track origin/feature/move-to-vanilla-cmake+OBJLIB

# clone mbed
make mbed_clone BRANCH=cmake-object-libraries-with-interface

# config
make config TARGET_BOARD=K64F

# build
make TARGET_BOARD=K64F

To activate the second executable target, uncomment the following line:

https://github.com/ladislas/mbed-cmake-template/blob/d17515d7368c5eb66ba4ee99dc41d3d606733ae7/CMakeLists.txt#L45

ladislas · 2021-02-26T08:31:01Z

@0xc0170 maybe the MBED_ROM_SIZE issue is related to this: ARMmbed/mbed-tools#156

ladislas · 2021-02-26T12:51:18Z

I've made a PR for the memory map. ARMmbed#14355

0xc0170 · 2021-03-01T09:27:39Z

I've made a PR for the memory map. ARMmbed#14355

+1

@0xc0170 maybe the MBED_ROM_SIZE issue is related to this: ARMmbed/mbed-tools#156

I have to investigate today. There's g0 target for F7: extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/mbed-stm32g031xx.link_script.ld this is wrong. I'll fix soon.

ladislas · 2021-03-01T10:28:11Z

There's g0 target for F7: extern/mbed-os/targets/TARGET_STM/TARGET_STM32G0/TARGET_STM32G031xx/mbed-stm32g031xx.link_script.ld this is wrong. I'll fix soon.

Yes! I saw that too. It seems like all the linkers are being included for the ST target.

0xc0170 · 2021-03-01T11:06:46Z

Correct I made a change I should not have. I reverted it and trying to find a way to generate a linker script. I'll push a rebase once I get it working.

0xc0170 · 2021-03-01T16:28:09Z

I fixed the issue with linker. DISCO_F769 now builds. I'll clean up bit more, add docs. Any review appreciated. If this works and people agree it's way to go, we start refactoring Mbed OS libraries.

The current view is that we have to do some magic to build core library due to deps with rtos and targets. Internal details, people won't touch the magic, we will document what we are doing and why. We might fix one day.

What users touch - targets libs - INTERFACE libraries, same as after refactor - it should be straightforward to add/remove targets. No change there.

All user facing components are OBJECT libs. I ported events so far only. I'll wait for more feedback and refactor the entire tree once agreed.

Otherwise this would not be visible to an app.

User facing targets should be just object libraries

Link what we can link with CMake. We still need copy core sources to mbed-os-obj. And expose also link options to app as linker flags are provided via them. Normally, we would link to target library but as it's interface, we would copy the sources as well, this is not what we want.

0xc0170 · 2021-03-01T16:37:14Z

Rebased to the latest master.

We do not want an app to link core and also targets. As both actually create one core component. Therefore expose include dirs from targets to core. mbed-events for instance should just link to mbed-core and it should get all it needs.

ladislas · 2021-03-01T21:25:16Z

🎉 I can confirm it works! I can compile both executables for the DISCO with only 250 files/"steps" (248 + 2)

0xc0170 · 2021-03-02T09:09:24Z

I made only comments improvements and some clean up. This is readme section bout object libs in our tree: https://github.com/0xc0170/mbed-os/blob/1d3e74f658ff020dbbd4d16ec7f8c51afdfde21f/tools/cmake/README.md#mbed-libraries, worth reading to understand what is this about.

0xc0170 · 2021-03-02T09:09:47Z

Thanks @ladislas for testing. Glad that this works.

These are part of cmsis + core, they only extend them thus reverting the change back to INTERFACE lib.

Easier to understand

0xc0170 · 2021-03-02T11:20:58Z

I'll move this to feature-cmake-object-libraries under mbed-os repo where we continue refactoring the libraries and test them. I'll close this now.

I'll fix any comments even after the closure.

0xc0170 mentioned this pull request Feb 23, 2021

CMake: From USCRPL/mbed-cmake to Mbed OS first-party CMake ARMmbed/mbed-os#13981

Closed

0xc0170 force-pushed the cmake-object-libraries-with-interface branch from b39b230 to 3c3567c Compare February 23, 2021 22:35

0xc0170 requested review from hugueskamba and rajkan01 February 24, 2021 09:18

ladislas mentioned this pull request Feb 25, 2021

Move to vanilla Mbed OS CMake - 0xc0170's OBJ LIB ladislas/mbed-cmake-template#7

Open

0xc0170 force-pushed the cmake-object-libraries-with-interface branch from 21a3d7b to 8f60484 Compare March 1, 2021 13:07

0xc0170 added 7 commits March 1, 2021 16:36

CMake: turn mbed-core into interface + obj libraries

caa5f58

Events: create interface + obj library

afd2d1f

cmsis: fix private link for core

51e447a

Maxim: turn max32630 into interface + object libraries

8c33c43

CMake: fix deps on other libraries

3db33b2

CMake: mbed target converted is cached variable

cad59f7

CMake Maxim: fix private obj files

b1caf5e

0xc0170 added 8 commits March 1, 2021 16:36

CMake: copy compile defs and link options to core from targets

ec0e1c4

CMake linker: move linker to mbed-core - upstream it to an app

cd1475a

Otherwise this would not be visible to an app.

CMake readme: fix typos in the comment

a969d78

events: use OBJECT library

63d1935

User facing targets should be just object libraries

CMake linker: fix change from the latest rebase

cfddd24

CMake: add comments for the magic we do around core/rtos libraries

5c86b46

CMake: remove util function, link object libs regularly

f972aab

0xc0170 force-pushed the cmake-object-libraries-with-interface branch from a5ca838 to f972aab Compare March 1, 2021 16:36

0xc0170 changed the title ~~CMake object libraries with interface~~ Mbed OS components as OBJECT libs Mar 1, 2021

0xc0170 added 3 commits March 1, 2021 17:07

CMake: expose targets includes to mbed-core

fa85e39

We do not want an app to link core and also targets. As both actually create one core component. Therefore expose include dirs from targets to core. mbed-events for instance should just link to mbed-core and it should get all it needs.

CMake: core should expose also compile defs from target

7ba307d

CMake: fix baremetal, core-obj links to core and target

1896097

0xc0170 added 4 commits March 2, 2021 08:28

CMake: add comments to internal mbed-os libraries

a06f7a8

CMake readme: fix libraries details

03fe4b1

CMake rtos: we link rtos in mbed-os where it gets all flags

1d3e74f

CMake cmsis: remove unused linking

869ad21

0xc0170 added 5 commits March 2, 2021 09:19

CMake Maxim: remove util

98e05bf

CMake cmsis: revert object for cmsis-cortex-x libs

f9a4f01

These are part of cmsis + core, they only extend them thus reverting the change back to INTERFACE lib.

CMake: rename -obj to -objects libraries

1ebbd57

Easier to understand

CMake readme: split the table to user-facing and internal libs

6856558

CMake: add comment about OBJECT limitation and our source tree deps

d36dc4d

0xc0170 force-pushed the cmake-object-libraries-with-interface branch from 455c5c0 to d36dc4d Compare March 2, 2021 11:05

0xc0170 closed this Mar 2, 2021

0xc0170 mentioned this pull request Mar 2, 2021

Add mbed-create-distro, a function to make adding multiple targets easy ARMmbed/mbed-os#14274

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mbed OS components as OBJECT libs #21

Mbed OS components as OBJECT libs #21

0xc0170 commented Feb 23, 2021 •

edited

Loading

multiplemonomials commented Feb 24, 2021

0xc0170 commented Feb 24, 2021 •

edited

Loading

ladislas commented Feb 24, 2021

0xc0170 commented Feb 24, 2021

0xc0170 commented Feb 25, 2021

ladislas commented Feb 25, 2021

0xc0170 commented Feb 25, 2021

ladislas commented Feb 25, 2021

ladislas commented Feb 25, 2021

ladislas commented Feb 25, 2021 •

edited

Loading

ladislas commented Feb 25, 2021 •

edited

Loading

ladislas commented Feb 26, 2021

ladislas commented Feb 26, 2021

0xc0170 commented Mar 1, 2021 •

edited

Loading

ladislas commented Mar 1, 2021

0xc0170 commented Mar 1, 2021

0xc0170 commented Mar 1, 2021 •

edited

Loading

0xc0170 commented Mar 1, 2021

ladislas commented Mar 1, 2021 •

edited

Loading

0xc0170 commented Mar 2, 2021

0xc0170 commented Mar 2, 2021

0xc0170 commented Mar 2, 2021

Mbed OS components as OBJECT libs #21

Mbed OS components as OBJECT libs #21

Conversation

0xc0170 commented Feb 23, 2021 • edited Loading

Summary of changes

Impact of changes

Migration actions required

Documentation

Pull request type

Test results

Reviewers

multiplemonomials commented Feb 24, 2021

0xc0170 commented Feb 24, 2021 • edited Loading

ladislas commented Feb 24, 2021

0xc0170 commented Feb 24, 2021

0xc0170 commented Feb 25, 2021

ladislas commented Feb 25, 2021

0xc0170 commented Feb 25, 2021

ladislas commented Feb 25, 2021

ladislas commented Feb 25, 2021

ladislas commented Feb 25, 2021 • edited Loading

ladislas commented Feb 25, 2021 • edited Loading

ladislas commented Feb 26, 2021

ladislas commented Feb 26, 2021

0xc0170 commented Mar 1, 2021 • edited Loading

ladislas commented Mar 1, 2021

0xc0170 commented Mar 1, 2021

0xc0170 commented Mar 1, 2021 • edited Loading

0xc0170 commented Mar 1, 2021

ladislas commented Mar 1, 2021 • edited Loading

0xc0170 commented Mar 2, 2021

0xc0170 commented Mar 2, 2021

0xc0170 commented Mar 2, 2021

0xc0170 commented Feb 23, 2021 •

edited

Loading

0xc0170 commented Feb 24, 2021 •

edited

Loading

ladislas commented Feb 25, 2021 •

edited

Loading

ladislas commented Feb 25, 2021 •

edited

Loading

0xc0170 commented Mar 1, 2021 •

edited

Loading

0xc0170 commented Mar 1, 2021 •

edited

Loading

ladislas commented Mar 1, 2021 •

edited

Loading