Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Module distribution mechanism and tooling #68

Open
nascentcore-eng opened this issue Feb 21, 2023 · 2 comments
Open

[RFC] Module distribution mechanism and tooling #68

nascentcore-eng opened this issue Feb 21, 2023 · 2 comments
Assignees
Labels
component:agent Agent, aka Tricorder, runs on each and every Kubernetes node and executes eBPF+WASM modules component:api-server API Server manages agents, database, and grafana; and supports UI and CLI feature request Adding new features to Starship

Comments

@nascentcore-eng
Copy link
Contributor

nascentcore-eng commented Feb 21, 2023

Is your feature request related to a problem? Please describe.
eBPF+WASM module is written by Starship end users. The learning curve of writing eBPF+WASM
needs to be minimal, and the binary format needs to be easy to distribute and use.

This mechanism needs to have 2 parts:

  • Content: How to organize contents, like source files, binary files, etc.
  • User-interface: How these contents should be made available to users, for example, as compressed binary packages, or text-based archive.

Describe the solution you'd like
Not sure

This is an exploratory task, needs brain store.

Describe alternatives you've considered

Additional context
#250

@nascentcore-eng nascentcore-eng added the feature request Adding new features to Starship label Feb 21, 2023
@nascentcore-eng nascentcore-eng added component:api-server API Server manages agents, database, and grafana; and supports UI and CLI component:agent Agent, aka Tricorder, runs on each and every Kubernetes node and executes eBPF+WASM modules labels Feb 21, 2023
@yunwei37
Copy link

yunwei37 commented Feb 21, 2023

The distribution approach of wasm-bpf is the same as most libbpf, cilium/ebpf programs, except in wasm executable file instead of native elf executable file. you can distribute the wasm binary what ever you want.

The distribut approach in eunomia-bpf also nearly the same as what bumblebee does, but in wasm module instead of native share libraries. Because wasm has It's own OCI image types, there is no need to create another type.

see https://github.com/eunomia-bpf/wasm-bpf
and https://github.com/eunomia-bpf/eunomia-bpf/tree/master/ecli

@nascentcore-eng
Copy link
Contributor Author

Bumblebee's OCI image has to be executed by bumblebee CLI, not by docker.

Eunomia-bpf's wasm exported pre-defined public functions, and ecc CLI needs to call these functions. JSON -> header file, included in wasm guest code, and compile into .wasm object file.

https://github.com/eunomia-bpf/wasm-bpf bpftool, libbpf
https://github.com/eunomia-bpf/wasm-bpf/tree/main/examples/bootstrap#the-compile-process-of-the-bootstrapwasm
Wasm includes ebpf byte code.

eunomia-bpf JSON, ecc can dynamic load ebpf program object file, and minimal set of BTF file, BTF consumes 10s of MB disk size.

nascentcore-eng added a commit that referenced this issue Mar 22, 2023
# Describe your changes
.bcc & .wasm are named after the module directory name .json is changed
to manifest.json
Gradually, we are making module directory more like a structured
package,
and closer to devise a more robust distribution format:
#68

## Issue
- [ ] Check this box if this PR fixes an issue, and then paste the link
below
- [ ] No associated issue

## Test:
- [ ] Check this box if this PR has tests
- [ ] No test needed

Signed-off-by: Yaxiong Zhao <[email protected]>
@nascentcore-eng nascentcore-eng changed the title Module distribution mechanism [RFC] Module distribution mechanism and tooling Mar 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:agent Agent, aka Tricorder, runs on each and every Kubernetes node and executes eBPF+WASM modules component:api-server API Server manages agents, database, and grafana; and supports UI and CLI feature request Adding new features to Starship
Projects
Status: No status
Development

No branches or pull requests

5 participants