cc.get_supported_arguments doesn't seem to work when CFLAGS contains linker options

[evverx]

get_supported_arguments(possible_cc_flags) is used to get a list of arguments supported by the compiler, some of which are supposed to suppress several compiler warnings. When systemd is built with the scripts provided by the OSS-Fuzz project, -Wl,--no-as-needed -Wl,-ldl -Wl,-lm are added to CFLAGS, which, in turn, makes meson think that none of possible_cc_flags are supported and leads to a lot of spurious warnings. Below is the relevant part of the build log:

...
The Meson build system
Version: 0.48.2
...
Appending CFLAGS from environment: '-O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION  -fprofile-instr-generate -fcoverage-mapping -pthread -Wl,--no-as-needed -Wl,-ldl -Wl,-lm -Wno-unused-command-line-argument'
Appending LDFLAGS from environment: '-L/usr/lib/clang/8.0.0/lib/linux'
Native C compiler: clang (clang 8.0.0 "clang version 8.0.0 (trunk 346388)")
Build machine cpu family: x86_64
Build machine cpu: x86_64
...
Running compile:
Working directory:  /tmp/tmpkrtp44g7
Command line:  clang /tmp/tmpkrtp44g7/testfile.c -pipe -D_FILE_OFFSET_BITS=64 -c -o /tmp/tmpkrtp44g7/output.obj -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fprofile-instr-generate -fcoverage-mapping -pthread -Wl,--no-as-needed -Wl,--start-group -Wl,-ldl -Wl,-lm -Wl,--end-group -Wno-unused-command-line-argument -O0 -Werror=unknown-warning-option -Werror=unused-command-line-argument -Werror=ignored-optimization-argument -Wno-gnu-variable-sized-type-not-at-end -Wgnu-variable-sized-type-not-at-end

Code:
 int i;

Compiler stdout:

Compiler stderr:
 clang-8: error: -Wl,--no-as-needed: 'linker' input unused [-Werror,-Wunused-command-line-argument]
clang-8: error: -Wl,--start-group: 'linker' input unused [-Werror,-Wunused-command-line-argument]
clang-8: error: -Wl,-ldl: 'linker' input unused [-Werror,-Wunused-command-line-argument]
clang-8: error: -Wl,-lm: 'linker' input unused [-Werror,-Wunused-command-line-argument]
clang-8: error: -Wl,--end-group: 'linker' input unused [-Werror,-Wunused-command-line-argument]

Compiler for C supports arguments -Wno-gnu-variable-sized-type-not-at-end: NO

cc @Dor1s

[Dor1s]

Hm, why does it complain even though there was -Wno-unused-command-line-argument passed?

[evverx]

I assume that's because meson also passed -Werror=unused-command-line-argument that most likely took precedence over -Wno-unused-command-line-argument :-)

[Dor1s]

:( any chance this can be tolerated on meson's size? May the order of the arguments be changed (if it affects anything, of course)? Because I'm not sure what to do on OSS-Fuzz side to make sure all the other projects are still building fine.

[evverx]

Given that the options clang is complaining about have nothing to do with the options passed to get_supported_arguments, maybe, one way to make it work more or less smoothly would be to analyze the output of clang to see if it contains the names of the options checked with get_supported_arguments. That's far from ideal because it depends on something nobody should depend on, but I can't seem to come up with anything else.

[jpakkane]

All of those flags are very suspect. The code specifically says that the no-as-needed flag is passed so that the warning produced by a spurious -ldl flag is suppressed. Which begs the question why is it there in the first place? The documentation does not say.

The core problem here is that OSS-Fuzz seems to treat all build systems as dumb and just throws compiler flags en masse at them. Meson does not really work like that. We aim to provide higher level interface to our users. We also exploit that to do things internally. This meshes badly with random linker flags injected in from the outside, especially for things that we have an internal option for.

The correct solution for this is that we provide the knobs necessary (most or possibly all of which we already do) so OSS-Fuzz and everyone else can just use those. This means we have one proper working solution and everyone and their dog does not have to reinvent the wheel and do the magic guessing game of "what linker flags should I inject in this case".

[evverx]

@jpakkane I used the script provided by OSS-Fuzz as an example of the environment where linker options passed via CFLAGS render get_supported_arguments useless and that's what this issue is about. I'd of course be happy if the OSS-Fuzz team found another way to pass the linker options around, but, in the meantime, it would be great if at least the meson documentation mentioned that it's supposed to be run in some kind of ideal hermetic environment so that it would be clear that get_supported_arguments shouldn't be used in the real world which is far from ideal.

[nirbheek]

We should consider filtering out -Wl flags from CFLAGS when running this.

[Dor1s]

Meson does not really work like that. We aim to provide higher level interface to our users.

That's the thing. Every build system does not really work like something else from different perspectives. We have to build 150+ projects in ~5 build configurations, and these projects are using whichever build system they prefer.

So far, the most unified way to handle this was to provide certain CFLAGS and some other variables, and require the projects to use them. AFAIK, the vast majority of the build systems out there do not have any problems with it.

If you have a better idea on how to solve this problem, I'd really appreciate your advice. Feel free to modify https://github.com/google/oss-fuzz/tree/master/infra/base-images/base-builder image locally and then test with different projects by running build_image and build_fuzzers with different configurations followed by check_build (link). Thanks!

[jpakkane]

Why are you injecting linker flags via CFLAGS and not via LDFLAGS? Anything starting with -Wl should not be used when compiling, only when linking. Moving all those sanitizer linker flags from CFLAGS to LDFLAGS should fix this.

[nirbheek]

Why are you injecting linker flags via CFLAGS and not via LDFLAGS

Autotools often requires you to inject linker flags in both CFLAGS and LDFLAGS, so it's a common configuration and we need to handle it.

[jpakkane]

That is not the case here, though. The problematic args are only in CFLAGS, not in LDFLAGS.

Anyhoo, I made an experimental branch for this. Can you test if it works for you?

[evverx]

@jpakkane I modified the build script a bit so that it used meson from the PR and then built systemd with the "coverage" sanitizer. Everything went smoothly without any spurious warnings. Thank you!