consumers using CO.RE might need vmlinux.h generation from BTF: it might not be available OR be different compile/run time

[rafaeldtinoco]

If the build system is running a recent kernel without BTF enabled... the build might fail. We could (should ?) include a default minor vmlinux.h so the build succeeds. Thoughts ?

[yanivagman]

Why BTF is required?
Today we compile tracee (with libbpfgo) without requiring BTF to be enabled

[rafaeldtinoco]

But you're already including: ./tracee-ebpf/tracee/x86_64/vmlinux.h, right ?

At: https://github.com/rafaeldtinoco/libbpfgo/blob/main/Makefile#L87

we would be generating it automatically, but the building env might not be able to do so.

Example: HWE kernels in Ubuntu, for example, don't have BTF enabled and won't have /sys/kernel/btf/vmlinux available at build time (of course this would open the discussion in supported building environments).

This also opens the question: libbpfgo on top of older kernels, or having BPF objects (of consumers) being built with vmlinux.h files (generated during build time ? or previously ? what kernel code base if previously ?)

Changes in vmlinux.h will cause issues with data type changes when using CO.RE, for example.

That is why I started with:

#28

So libbpfgo consumers might be able to do static or dynamic compiling of libbpfgo, and, perhaps, decide wether new vmlinux.h should be generated or not (if bpftool is available, if build environment kernel supports BTF, etc).

And by having that building schema we will be able to test libbpfgo statically and dynamically compiled against libbpf under different kernels as well as different libbpf versions.

Makes sense ?

[yanivagman]

But you're already including: ./tracee-ebpf/tracee/x86_64/vmlinux.h, right ?

At: https://github.com/rafaeldtinoco/libbpfgo/blob/main/Makefile#L87

We don't use vmlinux.h if CORE define was not chosen.
So on older kernels, where BTF is not enabled, tracee can compile against the local kernel headers, without requiring vmlinux.h.
If libbpfgo will require it, It might be a problem to compile in such environments

[rafaeldtinoco]

But you're already including: ./tracee-ebpf/tracee/x86_64/vmlinux.h, right ?
At: https://github.com/rafaeldtinoco/libbpfgo/blob/main/Makefile#L87

We don't use vmlinux.h if CORE define was not chosen.

This is tricky:

• Consumers might use CORE or not.
• They might use stable kernel interfaces or not (in case of kprobes/kretprobes).

So on older kernels, where BTF is not enabled, tracee can compile against the local kernel headers, without requiring vmlinux.h. If libbpfgo will require it, It might be a problem to compile in such environments

If consumers are using libbpfgo as libbpf is supposed to be used, and CORE is 'enabled' in that usage, then the BPF objects don't necessarily need vmlinux.h included, but I assume consumers will use it in their BPF C code (just like they do for libbpf).

So perhaps I should rename this issue to reflect this discussion.

Flagging @grantseltzer since this is something that came up in our discussions.

[yanivagman]

Ok, now it is more clear.
One thing I'm still not sure I understand.
vmlinuxh is now a build target in the makefile, which is used for both libbpfgo static and dynamic.
What if the kernel doesn't have /sys/kernel/btf/vmlinux? (like when BTF is not enabled) - wouldn't this give an error?

[rafaeldtinoco]

Ok, now it is more clear.
One thing I'm still not sure I understand.
vmlinuxh is now a build target in the makefile, which is used for both libbpfgo static and dynamic.

Nice catch ;). It was meant for the libbpfgo-dynamic-test only (because of the .bpf.c dependencies) but libbpfgo-test-bpf is already depending on it. I'm fixing that there.

What if the kernel doesn't have /sys/kernel/btf/vmlinux? (like when BTF is not enabled) - wouldn't this give an error?

Yes, there is much room for improvement there. I'll fix that but, for now, will likely add an error message only. Then, when solving this issue, I might do something else I guess.

[rafaeldtinoco]

Okay, we assume that whoever is using libbpfgo will always have to build in a system that has BTF available (/sys/kernel/btf/vmlinux) and this issue does not apply any longer.