CMake: toolchain files do not match the standard way of doing

[massonal]

Hello,

I have been studying the CMake files generated to build code with cbuild (CMakeLists.txt + GCC/IAR/AC6.cmake). I have found that the usage of the "toolchain files" does not match with the standard "way-of-doing" (cf. cmake-toolchains(7) — CMake 3.25.1 Documentation). You can read below detailed examples of some shortcomings I have found that relate to this issue.

My point is, is this at all intentional? If so, I think it should be documented somewhere, in order not to confuse readers expecting standard CMake practices. Or maybe there is room for improvement on this front?

Example shortcomings:

• toolchain file not used in a standard way (--toolchain= or CMAKE_TOOLCHAIN_FILE): confusing + potential side-effects (through CMake's built-in mechanisms...)
• it prevents having a single toolchain-agnostic CMakeLists.txt (as opposed one of the goals of CMake in the first place)
• variables shared between main file and toolchain: weaken the separation of purposes (describe the build vs describe the environment)

[brondani]

@massonal
Thanks for the comments.
Yes it is intentional that a generated CMakeLists.txt is tied to an environment and to a toolchain to avoid ambiguities. The portable files for building are the *.cprj and the future *.cbuild.yml files.

The toolchain files are significantly more complex than the cmake-toolchains.
In addition to the cmake cross compiling variables, the toolchain files bring variables, macros and dictionaries for several compiler/assembler/linker/archiver agnostic features such as device attribute flags, multiple defines syntaxes, pre-includes, system includes, etc. Most of them are not supported in the cmake upstream.

Here you can find some documentation concerning the variables in the toolchain files:
https://github.com/Open-CMSIS-Pack/devtools/blob/main/tools/projmgr/docs/Manual/Build-Operation.md#cmake-variables

[massonal]

Thanks for the reply, and the doc pointer!

Yes it is intentional that a generated CMakeLists.txt is tied to an environment and to a toolchain to avoid ambiguities.

Ok, so this is an actual build decision here 👍 I will accept that, although this is a major deviation from the CMake model I really don't approve of.
This springs a further question: CMakeLists.txt is tied to the toolchain -- fine, but the toolchain is also tied to CMakeLists.txt! Again, solid project decision or oversight? [Cf. comment footer and #649 on how it is tied and why I think it's a questionable design]

The portable files for building are the *.cprj and the future *.cbuild.yml files.

These files are the inputs to the CMakeLists.txt generation, right? So, shouldn't it be possible to extend the portability to the CMakeLists.txt as well? The farther the better IMO, unless there are some non-portable inputs as well?

The toolchain files are significantly more complex than the cmake-toolchains.

Yes, I can tell!
Now, I think it would be better to restrict their scope to stay true to the CMake model.
You could also consider creating a CMake library along with the toolchain, to help with all the operations you describe, or run these earlier in cbuild to generate a more refined CMakeLists.txt?

---

I also realize I have been quite terse in my third bullet point, so here are some more details (I also talk about that a bit in #649 ) :

CMakeLists.txt and the toolchain have very different purposes in the CMake model, and should not have to communicate with one another:

• The toolchain file uses the system environment to find a compiler, detect its features and syntax, and advertises them to CMake;
• CMakeLists.txt describes a project, in terms of structure, features and build process.

E.g., the toolchain should not care about #defines, the project name, or the target CPU (hint: it does now). Handling theses should be the responsibility of CMakeLists.txt, either directly or by using a library (with then gets called at project scope, not toolchain scope).

[brondani]

This springs a further question: CMakeLists.txt is tied to the toolchain -- fine, but the toolchain is also tied to CMakeLists.txt! Again, solid project decision or oversight?

The <toolchain>.<version>.cmake files should be seen as a sort of extension of generated CMakeLists.txt, not as a separate entity. This has been decided because the CMake upstream is not able to handle all the abstractions and use cases that are supported by cprj and now csolution project descriptions.

So, shouldn't it be possible to extend the portability to the CMakeLists.txt as well?

A CMSIS project is not self-contained under an umbrella directory as most CMake projects, but it is rather spread over different places. For clarity it has been decided to use absolute paths. It should be possible to extend the portability if there is a good reason for that.

You could also consider creating a CMake library along with the toolchain, to help with all the operations you describe, or run these earlier in cbuild to generate a more refined CMakeLists.txt?

It should be feasible, but it would just shift the features from one file or representation into another. I see more value introducing such features in the CMake upstream.