-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Integrate Beyla Agent as an OpenTelemetry Collector Receiver #34321
Comments
related grafana/beyla#1072 |
Beyla can export data in Direct mode: You can follow our getting started tutorial for a quick introduction to auto-instrumentation in Direct mode, by using OpenTelemetry. The OTLP endpoint authentication credentials are provided by using the following environment variables:
To run in Direct mode by using the Prometheus scrape endpoint, please refer to the |
Do we have any plans to support otel collector receiver for Beyla? As of today is it possible to integrate Beyla with Otel collector? Or do we have to specifically have alloy collector only? |
A) The setup I envision involves running Beyla on the same host as the collector, with Beyla sending data to the collector using OTLP Direct mode. This setup is already working. B) What’s being requested here is the ability to manage Beyla's configuration as an "OTel Collector contrib receiver," allowing remote configuration through a supervisor. Currently, since Beyla runs as a separate process, managing its configuration via OPAMP isn't possible. However, integrating eBPF-based kernel tracing executed by a Beyla receiver and managing the configuration through OPAMP would be a game-changer. Not only would it eliminate the need for OTEL SDK (zero instrumentation), but it would also make eBPF far more efficient, offering greater detail from the kernel space and even enabling profiling. fyi @maryliag |
Hey folks, I just wanted to mention that we made a proposal to donate Beyla to OpenTelemetry (link open-telemetry/community#2406), which if it's accepted would make this a lot easier. Regardless, I think we'll add Beyla as a component to the OpenTelemetry collector, but I can't give exact timelines yet. We'll know soon. |
Component(s)
No response
Is your feature request related to a problem? Please describe.
I would like to request the integration of the Beyla agent as a receiver in the OpenTelemetry Collector. Beyla is an open-source, eBPF-based application auto-instrumentation tool that supports OpenTelemetry and Prometheus, allowing for zero-code automatic instrumentation of HTTP/S, gRPC services, SQL, Redis, and Kafka.
Integrating Beyla as a receiver in the OpenTelemetry Collector would provide seamless observability for applications without requiring any code modifications. This integration would leverage Beyla’s capabilities to capture essential application events and transform them into OpenTelemetry trace spans and RED (Rate-Errors-Duration) metrics. By incorporating this into the OpenTelemetry Collector, users can benefit from a unified observability solution that is easy to configure and deploy across various environments.
https://github.com/grafana/beyla/
eBPF stands for Extended Berkeley Packet Filter, and allows attaching applications to different points of the Linux Kernel. eBPF applications run in privileged mode and allow the runtime information of the Linux Kernel to be inspected: system calls, network stack, as well as inserting probes in user space applications.
The eBPF applications are safe, they are compiled for their own Virtual Machine instruction set and run in a sandboxed environment that verifies each loaded eBPF program for memory access safety and finite execution time. Unlike older technologies, such as the natively-compiled Kprobes and Uprobes, there is no chance that a poorly programmed probe will cause the Linux Kernel to hang.
After being the eBPF binaries have been verified they are compiled with a Just-In-Time (JIT) compiler for the native host architecture (x86-64, ARM64, etc). This allows for efficient and fast execution.
The eBPF code is loaded from ordinary applications running in user space. The kernel and the user space applications can share information through a set of well defined communication mechanisms, which are provided by the eBPF specification. For example: ring buffers, arrays, hash maps, etc.
Beyla eBPF architecture
Describe the solution you'd like
Configuration Options:
Specify which executable to monitor.
Define network ports, executable names, or process IDs for instrumentation.
Choose the exposition formats (OpenTelemetry metrics, Distributed Traces).
Compatibility:
Support for a wide range of programming languages (Go, Java, .NET, NodeJS, Python, Ruby, Rust, etc.).
Integration with existing OpenTelemetry Collector pipelines and exporters.
Automatic Instrumentation:
Utilize eBPF to inspect application executables and OS networking layers.
Capture HTTP/S, gRPC, SQL, Redis, and Kafka events automatically without code changes.
Security and Permissions:
Include detailed examples for running Beyla with the necessary permissions on various platforms (e.g., Kubernetes, Docker).
Describe alternatives you've considered
Benefits:
Simplified observability setup for applications.
Leverage Beyla’s eBPF-based instrumentation for high performance and low overhead.
Unified configuration and deployment with existing OpenTelemetry tools and practices.
Additional context
Here is an example of how the configuration might look for integrating Beyla as a receiver in the OpenTelemetry Collector:
The text was updated successfully, but these errors were encountered: