Support testing different kernel versions

[rafaelroquetto]

Virtual Machine Tests enables Beyla to be tested under different kernel versions and architectures. This works by exposing the path to the Beyla source code (i.e. this git repository) to the guest by means of mounting it to the '/build' directory. This allows Makefile targets to be invoked from within the guest, in particular those tied to integration tests. Analogous to a Docker container, changes to the '/build' directory are volatile and are neither reflected on the host nor preserved once the VM terminates.

The supported pre-compiled kernels can be found in the kernels/$arch/ subdirectory, alongside their respective kernel config files.

---

USAGE

Simply run 'make' to get started. This will build a new rootfs image and launch the VM using the default kernel specified in the Makefile. To select a different kernel version. The following environment variables can be used to specify the kernel version or architecture to run:

• ARCH: the target architecture (currently only x86_64 is supported)
• KERNEL_VER: the kernel version to use OR;
• KERNEL: path the kernel image to use
• TARGET_TEST: the target to pass to Beyla's toplevel Makefile (defaults to run-integration-test-vm). When set to empty, the VM will drop onto an interactive shell

For debugging purposes, the VM also exposes an SSH endpoint at port 2222. The
access credentials are root:root.

See test/vm/README for further information, including on how to deploy new kernels.

Resolves #878

Tip: reviewing individual commits might be simpler.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.38%. Comparing base (a3ffb2f) to head (3f8cada).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1100      +/-   ##
==========================================
- Coverage   79.44%   79.38%   -0.07%     
==========================================
  Files         140      140              
  Lines       11514    11514              
==========================================
- Hits         9147     9140       -7     
- Misses       1825     1829       +4     
- Partials      542      545       +3     

Flag	Coverage Δ
k8s-integration-test	59.27% <ø> (ø)
oats-test	37.07% <ø> (ø)
unittests	52.37% <ø> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[grcevski]

LGTM! Excellent work!

[marctc]

Amazing work @rafaelroquetto, very much needed!

My question here is, do we want to run these tests on every PR? The pipeline is already slow, some times flaky, and quite complex to debug if something fails. Adding more tests here might be a burden for some pull requests.

In this PR you added two tests, but what would happen if we want to test more kernel and different architectures? My suggestion would be to move these tests to be built only in main. This has the cost that we have to monitor the health of builds/test in main as some thing may broke, but hopefully what can use some tooling like this: https://grafana.com/blog/2023/11/20/ci-cd-observability-via-opentelemetry-at-grafana-labs/

[rafaelroquetto]

@marctc that sound pretty reasonable to me - it should also be straightforward to flick the switch back and forth if we decide down the road that building only on main is not cutting it. @mariomac @grcevski, any thoughts from your end?

[grcevski]

@marctc that sound pretty reasonable to me - it should also be straightforward to flick the switch back and forth if we decide down the road that building only on main is not cutting it. @mariomac @grcevski, any thoughts from your end?

I think limiting when these tests run makes a lot of sense, I wonder if the best approach is to run them only if we have changes in the BPF generated files? PRs that don't touch that part shouldn't need to run the VM tests. This is similar how testing of the documentation works, unless we touch docs, doc testing isn't running. WDYT?

[grcevski]

I personally would like to run them on PRs, because they catch failures which I can't catch myself with my main development kernel version.

[rafaelroquetto]

Good idea - I can look into running these workflows only when the ebpf files have changed. This should address @marctc's concerns whilst not compromising coverage.

[rafaelroquetto]

Tests now passing:

• https://github.com/grafana/beyla/actions/runs/10584749826/job/29329713040?pr=1100 (5.15)
• https://github.com/grafana/beyla/actions/runs/10584749822/job/29329713048?pr=1100 (6.10)

They aren't appearing on the latest update as they have been disabled when there are no changes to the ebpf files.

[grcevski]

Looks great!