BPF implementation does not support BPF to BPF function calls

[jackcmay]

Problem

In order to support multi-file BPF programs and linking in/with libraries, our BPF implementation must support BPF to BPF calls.

Proposed Solution

• BPF to BPF function calls
  • Add BPF symbol resolutions (intra ELF)
  • Handle BPF subroutine return
  • Support BPF subroutine stack frames

Depends on

• Build system support for multi-file projects
  BPF no support for multi-file projects in build system #2176
• Respect ELF entry point
  ELF entrypoint not respected #2177

[jackcmay]

The linker is not resolving symbols between ELFs. There are two approaches:

1. Add support to the linker (right solution but will take some time)
2. Modify rbpf to do the resolutions at run-time

Moving forward on #2 since should be able to get that working quickly

[mvines]

I support the right solution that takes longer, (1). It would be nice to get this into v0.11 but not at the expense of short-term work that we would immediately need to undo and redo.

[jackcmay]

Turns out this is more complicated than it originally presented. The right solution is a combination of both.

[jackcmay]

One pain point of the proposed work is that BPF does not interact with the stack in a typical way. Typically a stack pointer is managed that moves as things a pushed and popped from the stack. For example, when functions are called, zero or more of their parameters maybe be pushed and the return address is pushed. BPF programs don't manage a head pointer, instead, a function's stack is a fixed size buffer essentially used as a scratch buffer. BPF does not pass parameters via the stack (only registers) This comes historically from the fact that all BPF programs are a single function. This means that each BPF function has its own "stack" frame. Linux recently added support for BPF to BPF calls and they do this by providing a 512 byte stack frame for each function called with a maximum depth of 8 function calls.