Python BPF disassembler and map layout parser

[oriolarcas]

Debugging eBPF programs can be tricky. The clang debug flags are not
supported in all the code-loading branches yet - e.g., only load_prog()
supports BPF_DEBUG or DEBUG_BPF_REGISTER_STATE, but compiling a kprobe
with BPF(...) doesn't.

This built-in disassembler can disassemble and print the BPF code in a
similar syntax than the kernel, whenever and the number of times the
user needs it. The BPF ISA is relatively stable so it doesn't require
much maintenance.

In addition, this parser is agnostic from the original source language
(C, B, Go, etc.), and doesn't depend on a particular compiler.

Example output for trace_pid_start() in biotop:

Disassemble of BPF program trace_pid_start:
   0: (79) r1 = *(u64*)(r1 +112)
   1: (7b) *(u64*)(r10 -8) = r1
   2: (b7) r1 = 0
   3: (63) *(u32*)(r10 -16) = r1
   4: (7b) *(u64*)(r10 -24) = r1
   5: (7b) *(u64*)(r10 -32) = r1
   6: (bf) r1 = r10
   7: (07) r1 += -28
   8: (b7) r2 = 16
   9: (85) call bpf_get_current_comm#16
  10: (67) r0 <<= 32
  11: (77) r0 >>= 32
  12: (55) if r0 != 0 goto +10 <23>
  13: (85) call bpf_get_current_pid_tgid#14
  14: (63) *(u32*)(r10 -32) = r0
  15: (18) r1 = <map at fd #3>
  17:      (64-bit upper word)
  17: (bf) r2 = r10
  18: (07) r2 += -8
  19: (bf) r3 = r10
  20: (07) r3 += -32
  21: (b7) r4 = 0
  22: (85) call bpf_map_update_elem#2
  23: (b7) r0 = 0
  24: (95) exit

The fields, types and memory layouts of maps can also be printed, which
is something that can be really helpful when dealing with unaligned
accesses or packed vs unpacked structures, and currently not supported
by clang.

For a map with key struct {int a; short b; struct {int c:4; int d:8;};}); and value u64 the example output is:

Layout of BPF type HASH map test_map (ID 0):
  struct {
    [0 +4] int a;
    [4 +2] short b;
    [6 +2] char[2] __pad_2;
    [8 +4] struct {
      int c:4;
      int d:8;
    } __anon0;
  } key;
  unsigned long long value;

The [X +Y] is optional and denotes the offset and the size of each
field. Note that bit-fields and padding fields are shown.

[yonghong-song]

[buildbot, test this please]

[yonghong-song]

only load_prog() supports BPF_DEBUG or DEBUG_BPF_REGISTER_STATE, but compiling a kprobe with BPF(...) doesn't.
I haven't examined it. We should add debug support for BPF(...) as well.

Recently kernel and compiler (llvm trunk) starts to support jmp32 insns. The compiler support in llvm9. I guess this can be added later.

Could you add a test, e.g., in test_clang.py, to test a simple program and a simple map?

With BTF support in bcc is on its way, bpftool (built from kernel tree) will be able to print out source annotated byte/jit codes and pretty print maps.

The map structure print will be good for debugging just in case the structure envisioned by user is different from bcc. With extra debug flag, the compiler can already print out insns. But I guess the interface here is more python user friendly and can be called as needed. Furthermore, it does not need a lot future changes as BPF ISA is quite stable.

[yonghong-song]

actually, this is not related to test frontend functionalities. so a standalone test under tests/python will be more appropriate.

[oriolarcas]

Since BTF is available in 4.18, I think this feature will help bridging the gap between current kernels -e.g., 4.15 in Ubuntu 18.04, 4.9 in Debian Stretch- and the ones supporting it. Same with toolchain support for BTF.

[yonghong-song]

[buildbot, test this please]

[yonghong-song]

27: Test command: /home/fedora/jenkins/workspace/bcc-pr/label/fc26/build/tests/wrapper.sh "py_disassembler" "simple" "/home/fedora/jenkins/workspace/bcc-pr/label/fc26/tests/python/test_disassembler.py"
27: Test timeout computed to be: 10000000
27: .could not open bpf map: test_map, error: Operation not permitted
27: E
27: ======================================================================
27: ERROR: test_func (__main__.TestDisassembler)
27: ----------------------------------------------------------------------
27: Traceback (most recent call last):
27:   File "/home/fedora/jenkins/workspace/bcc-pr/label/fc26/tests/python/test_disassembler.py", line 146, in test_func
27:     }""")
27:   File "/usr/lib/python2.7/site-packages/bcc/__init__.py", line 325, in __init__
27:     raise Exception("Failed to compile BPF text")
27: Exception: Failed to compile BPF text
27: 
27: ----------------------------------------------------------------------
27: Ran 2 tests in 0.119s
27: 
27: FAILED (errors=1)
27: Failed
27/39 Test #27: py_test_disassembler .............***Failed    0.19 sec

Looks like in test cmake file, you need sudo.

[yonghong-song]

[buildbot, test this please]

[yonghong-song]

Looks like a legitimate failure on fc28, could you take a look?

[oriolarcas]

Sure. I confirmed that it happens if the test is run with Python 3. I'll fix it.

[yonghong-song]

[buildbot, test this please]