diff --git a/docs/sources/network/_index.md b/docs/sources/network/_index.md new file mode 100644 index 000000000..63d7b6328 --- /dev/null +++ b/docs/sources/network/_index.md @@ -0,0 +1,74 @@ +--- +title: Network metrics +menuTitle: Network +description: Configuring Beyla to observe point-to-point network metrics. +weight: 1 +keywords: + - Beyla + - eBPF + - Network +--- + +{{% admonition type="warning" %}} +Network metrics is an [experimental](/docs/release-life-cycle/) under development feature, expect breaking changes. +{{% /admonition %}} + +# Network metrics + +Grafana Beyla can be configured to provide network metrics between different endpoints. For example, between physical nodes, containers, Kubernetes pods, services, etc. + +{{% admonition type="note" %}} +Prometheus exporting for network metrics is not currently supported. +{{% /admonition %}} + +## Get started + +To get started using Beyla networking metrics, consult the [quickstart setup documentation]({{< relref "./quickstart" >}}), and for advanced configuration, consult the [configuration documentation]({{< relref "./config" >}}). + +## Metric attributes + +Network metrics provides a single **OpenTelemetry** metric `beyla.network.flow.bytes`, a counter of Number of bytes observed between two network endpoints, with the following attributes: + +| Attribute name | Description | +|----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `beyla.ip` | Local IP address of the Beyla instance that emitted the metric | +| `src.address` | Source IP address of Network flow | +| `dst.address` | Destination IP address of Network flow | +| `src.name` | Name of Network flow source: Kubernetes name, host name, or IP address | +| `dst.name` | Name of Network flow destination: Kubernetes name, host name, or IP address | +| `src.namespace` | Namespace of Network flow source. Could be empty in non-Kubernetes flows | +| `dst.namespace` | Namespace of Network flow destination. Could be empty in non-Kubernetes flows | +| `src.cidr` | If the [`cidrs` configuration section]({{< relref "./config" >}}) is set, the CIDR that matches the source IP address | +| `dst.cidr` | If the [`cidrs` configuration section]({{< relref "./config" >}}) is set, the CIDR that matches the destination IP address | +| `k8s.src.namespace` | Kubernetes namespace of the source of the flow | +| `k8s.dst.namespace` | Kubernetes namespace of the destination of the flow | +| `k8s.src.name` | Name of the source Pod, Service, or Node | +| `k8s.dst.name` | Name of the destination Pod, Service, or Node | +| `k8s.src.owner.name` | Name of the owner of the source Pod. If there is no owner, the Pod name is used | +| `k8s.dst.owner.name` | Name of the owner of the destination Pod. If there is no owner, the Pod name is used | +| `k8s.src.owner.type` | Type of the owner of the source Pod: `Deployment`, `DaemonSet`, `ReplicaSet`, `StatefulSet`, or `Pod` if there is no owner | +| `k8s.dst.owner.type` | Type of the owner of the destination Pod: `Deployment`, `DaemonSet`, `ReplicaSet`, `StatefulSet`, or `Pod` if there is no owner | +| `k8s.src.node.ip` | IP address of the source Node | +| `k8s.dst.node.ip` | IP address of the destination Node | +| `k8s.src.node.name` | Name of the source Node | +| `k8s.dst.node.name` | Name of the destination Node | +| `k8s.cluster.name` | Name of the Kubernetes cluster. Beyla can auto-detect it on Google Cloud, Microsoft Azure, and Amazon Web Services. For other providers, set the `BEYLA_KUBE_CLUSTER_NAME` property | + +### Allowed attributes + +If the metric with all the attributes is reported it might lead to a cardinality explosion, especially when including external traffic in the `src.address`/`dst.address` attributes. + +You can specify which attributes are allowed in the Beyla configuration. Allowed attributes and aggregates the metrics by them. For example: + +```yaml +network: + enable: true + allowed_attributes: + - k8s.src.owner.name + - k8s.src.namespace + - k8s.dst.owner.name + - k8s.dst.namespace +``` + +In this example, the bytes metric is the aggregated by the source and destination owners. This is, all the +pods from a given Deployment/StatefulSet/ReplicaSet/DaemonSet. diff --git a/docs/sources/network/config.md b/docs/sources/network/config.md new file mode 100644 index 000000000..ada270833 --- /dev/null +++ b/docs/sources/network/config.md @@ -0,0 +1,177 @@ +--- +title: Beyla Network Metrics configuration options +menuTitle: Configuration +description: Learn about the configuration options available for Beyla network metrics +weight: 2 +keywords: + - Beyla + - eBPF + - Network +--- + +{{% admonition type="warning" %}} +Network metrics is an [experimental](/docs/release-life-cycle/) under development feature, expect breaking changes. +{{% /admonition %}} + +# Beyla Network Metrics configuration options + +Network metrics are configured under the `network` property of the [Beyla Configuration YAML file]({{< relref "../configure/options" >}}) or with a set of environment variables prefixed as `BEYLA_NETWORK_`. + +Example YAML: + +```yaml +network: + enable: true + allowed_attributes: + - k8s.src.owner.name + - k8s.src.namespace + - k8s.dst.owner.name + - k8s.dst.namespace + - src.cidr + - dst.cidr + cidrs: + - 10.10.0.0/24 + - 10.0.0.0/8 + - 10.30.0.0/16 +``` + +## Network metrics configuration properties + +| YAML | Environment variable | Type | Default | +| -------- | ----------------------- | ------- | ------- | +| `enable` | `BEYLA_NETWORK_METRICS` | boolean | `false` | + +Enables network metrics reporting in Beyla. + +| YAML | Environment variable | Type | Default | +| -------------------- | ---------------------------------- | -------- | ------- | +| `allowed_attributes` | `BEYLA_NETWORK_ALLOWED_ATTRIBUTES` | []string | (empty) | + +Specifies which attributes are visible in the metrics. +Beyla aggregates the metrics by their common visible attributes. +For example, hiding the `k8s.src.name` and allowing `k8s.src.owner.name` would aggregate the metrics of all the pods under the same owner. + +This property won't filter some meta-attributes such as `instance`, `job`, `service.instance.id`, `service_name`, `telemetry.sdk.*`, etc. + +{{% admonition type="note" %}} +If left empty, Beyla reports all attributes; which might greatly increase the cardinality of your metrics. +Setting this value to list only the attributes you really need is highly recommended. +{{% /admonition %}} + +See the [network metrics documentation]({{< relref "./_index.md" >}}) for a detailed list of all the available attributes. + +If you set this property via environment variable each entry must be separated by a comma, for example: + +```sh +BEYLA_NETWORK_ALLOWED_ATTRIBUTES=src.name,dst.name +``` + +| YAML | Environment variable | Type | Default | +| ------- | --------------------- | -------- | ------- | +| `cidrs` | `BEYLA_NETWORK_CIDRS` | []string | (empty) | + +CIDRs list, to be set as the `src.cidr` and `dst.cidr` attribute with the entry that matches the `src.address` and `dst.address` respectively. + +The attribute as a function of the source and destination IP addresses. +If an IP address does not match any address here, the attributes won't be set. +If an IP address matches multiple CIDR definitions, the flow is decorated with the narrowest CIDR. +As a result, you can safely add a `0.0.0.0/0` entry to group all the traffic that does not match any of the other CIDRs. + +If you set this property via environment variable each entry must be separated by a comma, for example: + +```sh +BEYLA_NETWORK_CIDRS=10.0.0.0/8,192.168.0.0/16 +``` + +| YAML | Environment variable | Type | Default | +| ---------- | ------------------------ | ------ | --------- | +| `agent_ip` | `BEYLA_NETWORK_AGENT_IP` | string | (not set) | + +Allows overriding the reported `beyla.ip` attribute on each metric. +If not set, Beyla automatically detects its own IP address from the specified network interface (see next property). + +| YAML | Environment variable | Type | Default | +| ---------------- | ------------------------------ | ------ | ---------- | +| `agent_ip_iface` | `BEYLA_NETWORK_AGENT_IP_IFACE` | string | `external` | + +Specifies which interface Beyla should use to pick its own IP address to set the value of the `beyla.ip` attribute. +Accepted values are: `external` (default), `local`, or `name:` (e.g. `name:eth0`). + +If the `agent_ip` configuration property is set, this property has no effect. + +| YAML | Environment variable | Type | Default | +| --------------- | ----------------------------- | ------ | ------- | +| `agent_ip_type` | `BEYLA_NETWORK_AGENT_IP_TYPE` | string | `any` | + +Specifies which type of IP address (IPv4 or IPv6 or both) Beyla should report in the `beyla.ip` field of each flow. +Accepted values are: `any` (default), `ipv4`, `ipv6`. +If the `agent_ip` configuration property is set, this property has no effect. + +| YAML | Environment variable | Type | Default | +| ------------ | -------------------------- | -------- | ------- | +| `interfaces` | `BEYLA_NETWORK_INTERFACES` | []string | (empty) | + +The interface names where flows are collected from. +If empty, Beyla fetches all the interfaces in the system, excepting the ones listed in `excluded_interfaces` (see below). +If an entry is enclosed by slashes (e.g. `/br-/`), it is matched as regular expression, otherwise it is matched as a case-sensitive string. + +If you set this property via environment variable each entry must be separated by a comma, for example: + +```sh +BEYLA_NETWORK_INTERFACES=eth0,eth1,/^veth/ +``` + +| YAML | Environment variable | Type | Default | +| -------------------- | ---------------------------------- | -------- | ------- | +| `exclude_interfaces` | `BEYLA_NETWORK_EXCLUDE_INTERFACES` | []string | `lo` | + +The interface names to be excluded from network flow tracing. +Default: `lo` (loop-back). +If an entry is enclosed by slashes (e.g. `/br-/`), it is matched as a regular expression, otherwise it is matched as a case-sensitive string. + +If you set this property via environment variable each entry must be separated by a comma, for example: + +```sh +BEYLA_NETWORK_EXCLUDE_INTERFACES=lo,/^veth/ +``` + +| YAML | Environment variable | Type | Default | +| ----------------- | ------------------------------- | ------- | ------- | +| `cache_max_flows` | `BEYLA_NETWORK_CACHE_MAX_FLOWS` | integer | `5000` | + +Specifies how many flows can be accumulated in the accounting cache before being flushed for its later export. +Default value is 5000. +Decrease it if you see the "received message larger than max" error in Beyla logs. + +| YAML | Environment variable | Type | Default | +| ---------------------- | ------------------------------------ | -------- | ------- | +| `cache_active_timeout` | `BEYLA_NETWORK_CACHE_ACTIVE_TIMEOUT` | duration | `5s` | + +Specifies the maximum duration that flows are kept in the accounting cache before being flushed for its later export. + +| YAML | Environment variable | Type | Default | +| ----------- | ------------------------- | ------ | ------- | +| `direction` | `BEYLA_NETWORK_DIRECTION` | string | `both` | + +Allows selecting which flows to trace according to its direction in the interface where they are captured from. +Accepted values are `ingress`, `egress`, or `both` (default). + +{{% admonition type="note" %}} +In this context, _ingress_ or _egress_ are not related to incoming/outgoing traffic from outside the node or the cluster, but the network interface. +This means that the same network packet could be seen as "ingress" in a virtual network device and as "egress" in the backing physical network interface. +{{% /admonition %}} + +| YAML | Environment variable | Type | Default | +| ---------- | ------------------------ | ------- | -------------- | +| `sampling` | `BEYLA_NETWORK_SAMPLING` | integer | `0` (disabled) | + +The rate at which packets should be sampled and sent to the target collector. +For example, if set to 100, one out of 100 packets, on average, are sent to the target collector. + + +| YAML | Environment variable | Type | Default | +| ------------- | --------------------------- | ------- | ------- | +| `print_flows` | `BEYLA_NETWORK_PRINT_FLOWS` | boolean | `false` | + +If set to `true`, Beyla prints each network flow to standard output. +Note, this might generate a lot of output. diff --git a/docs/sources/network/quickstart.md b/docs/sources/network/quickstart.md new file mode 100644 index 000000000..f733545e0 --- /dev/null +++ b/docs/sources/network/quickstart.md @@ -0,0 +1,260 @@ +--- +title: Beyla network metrics quickstart +menuTitle: Quickstart +description: A quickstart guide to produce Network Metrics from Grafana Beyla +weight: 1 +keywords: + - Beyla + - eBPF + - Network +--- + +{{% admonition type="warning" %}} +Network metrics is an [experimental](/docs/release-life-cycle/) under development feature, expect breaking changes. +{{% /admonition %}} + +# Beyla network metrics quickstart + +Beyla can generate network metrics in any environment (physical host, virtual host, or container). While the feature is in experimental development, it is recommended to use a Kubernetes environment, as Beyla is able to decorate each metric with the metadata of the source and destination Kubernetes entities. + +## Deploy Beyla with network metrics + +To enable network metrics, set the following option in your Beyla configuration: + +```yaml +network: + enable: true +``` + +Or export the following environment variable + +```sh +export BEYLA_NETWORK_METRICS=true +``` + +Network metrics requires metrics to be decorated with Kubernetes metadata. To enable this feature, set the following option in your Beyla configuration: + +```yaml +attributes: + kubernetes: + enable : true +``` + +Or export the following environment variable + +```sh +export BEYLA_KUBE_METADATA_ENABLE=true +``` + +Finally, network metrics requires administrative `sudo` privileges with the following capabilities: + +- Full privileged access, `root`, `sudo`, or `privileged: true` for Kubernetes +- The following capabilities: `BPF`, `PERFMON`, `NET_ADMIN`, `SYS_RESOURCE` + +To learn more about Beyla configuration, consult the [Beyla configuration documentation]({{< relref "../configure/options.md" >}}). + +## Example configuration + +The following YAML configuration provides a simple Beyla deployment for network metrics: + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: beyla +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: beyla +rules: + - apiGroups: [ "apps" ] + resources: [ "replicasets" ] + verbs: [ "list", "watch" ] + - apiGroups: [ "" ] + resources: [ "pods", "services", "nodes" ] + verbs: [ "list", "watch" ] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: beyla +subjects: + - kind: ServiceAccount + name: beyla + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: beyla +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: beyla-config +data: + beyla-config.yml: | + attributes: + kubernetes: + enable: true + network: + enable: true + print_flows: true +--- +apiVersion: apps/v1 +kind: DaemonSet +metadata: + name: beyla +spec: + selector: + matchLabels: + instrumentation: beyla + template: + metadata: + labels: + instrumentation: beyla + spec: + serviceAccountName: beyla + hostNetwork: true + volumes: + - name: beyla-config + configMap: + name: beyla-config + containers: + - name: beyla + image: grafana/beyla:main + securityContext: + privileged: true + volumeMounts: + - mountPath: /config + name: beyla-config + env: + - name: BEYLA_CONFIG_PATH + value: "/config/beyla-config.yml" +``` + +Note the following requirements for this deployment configuration: + +- The container image uses the latest under-development `grafana/beyla:main` image. +- Beyla needs to run as a DaemonSet, as it is requires only one Beyla instance per node +- To listen to network packets on the host, Beyla requires the `hostNetwork: true` permission +- To decorate the network metrics with Kubernetes metadata, create a `ClusterRole` and `ClusterRoleBinding` with `list` and `watch` permissions for ReplicaSets, Pods, Services and Nodes + +The configuration does not set an endpoint to export metrics. Instead, the `print_traces: true` option outputs the captured network flows to standard output. + +Use `kubectl logs` to see network flow entries, for example: + +``` +network_flow: beyla.ip=172.18.0.2 iface= direction=255 src.address=10.244.0.4 dst.address=10.96.0.1 +src.name=local-path-provisioner-7577fdbbfb-g6b7d src.namespace=local-path-storage +dst.name=kubernetes dst.namespace=default k8s.src.node.ip=172.18.0.2 +k8s.src.node.name=kind-control-plane k8s.dst.namespace=default k8s.dst.name=kubernetes +k8s.dst.owner.type=Service k8s.src.namespace=local-path-storage +k8s.src.name=local-path-provisioner-7577fdbbfb-g6b7d k8s.src.type=Pod +k8s.src.owner.name=local-path-provisioner k8s.src.owner.type=Deployment +k8s.dst.type=Service k8s.dst.owner.name=kubernetes +``` + +For further information on the attributes used, consult the [network metrics documentation]({{< relref "./_index.md" >}}). + +## Export OpenTelemetry metrics + +After you have confirmed that network metrics are being collected, configure Beyla to export the metrics in OpenTelemetry +format to an OpenTelemetry endpoint. + +{{% admonition type="note" %}} +Prometheus exporting for network metrics is not currently supported. +{{% /admonition %}} + +Beyla works with any OpenTelemetry endpoint. This quickstart uses the OpenTelemetry endpoint in Grafana Cloud. You can get a [Free Grafana Cloud Account at Grafana's website](/pricing/). + +To get your stack's OpenTelemetry endpoint, login to the Grafana Cloud Portal, and click **Configure** under the **OpenTelemetry** section. + +![OpenTelemetry Grafana Cloud portal](https://grafana.com/media/docs/grafana-cloud/beyla/quickstart/otel-cloud-portal-box.png) + +Under **Password / API token**, click **Generate now** and follow the instructions to create an API token. + +The **Environment Variables** section is populated with a set of standard OpenTelemetry environment variables which provide the connection endpoint and credentials information for Beyla. + +![OTLP connection headers](https://grafana.com/media/docs/grafana-cloud/beyla/quickstart/otlp-connection-headers.png) + +Copy the value of `OTEL_EXPORTER_OTLP_HEADERS` environment variable and paste it as a Kubernetes secret (and deploy it): + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: grafana-secret +type: Opaque +stringData: + otlp-headers: "Authorization=Basic MzQ3NTp....." +``` + +Now add the `OTEL_EXPORTER_OTLP_HEADERS` and reference this secret as the variable value. + +Also Add `OTEL_EXPORTER_OTLP_ENDPOINT` and its value as an environment variable to the Beyla container in the Kubernetes manifest. The `env` section of the `beyla` container in the manifest from the start of this document should look like: + +```yaml + env: + - name: BEYLA_CONFIG_PATH + value: "/config/beyla-config.yml" + - name: OTEL_EXPORTER_OTLP_ENDPOINT + value: "https://otlp-gateway-prod-eu-west-0.grafana.net/otlp" + - name: OTEL_EXPORTER_OTLP_HEADERS + valueFrom: + secretKeyRef: + key: otlp-headers + name: grafana-secret +``` + +## Select metrics attributes to reduce cardinality + +Letting Beyla to include all the [attributes]({{< relref "./_index.md" >}}) in the reported metric might lead to +a [cardinality explosion](/blog/2022/02/15/what-are-cardinality-spikes-and-why-do-they-matter/) in +your metrics storage, especially if you are capturing external traffic and reporting their IP addresses in the +`src.address` or `dst.address` metric attribute. + + +The `allowed_attributes` YAML subsection under `network` (or the `BEYLA_NETWORK_ALLOWED_ATTRIBUTES` environment variable) +lets you selecting the attributes to report: + +```yaml +network: + enable: true + allowed_attributes: + - k8s.src.owner.name + - k8s.src.namespace + - k8s.dst.owner.name + - k8s.dst.namespace +``` + +The previous example would aggregate the `beyla.network.flow.bytes` value by source and destination Kubernetes owner +(Deployment, DaemonSet, StatefulSet, ReplicaSet), avoiding finer-grained attributes such as Pod name or IP addresses. + +### Group IP addresses by CIDR + +Reporting metric attributes containing IP addresses (`src.address` and `dst.address`) might lead to cardinality explosion, +however it might be a useful network-level information to get a better view about how networks and sub-networks communicate. + +The `cidrs` YAML subsection in `network` (or the `BEYLA_NETWORK_CIDRS` environment variable) accepts a list of +subnets in [CIDR notation](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing), in both IPv4 and IPv6 format. + +The existence of the `cidrs` section leaves the `src.address` and `dst.address` fields untouched, +and adds the `src.cidr` and `dst.cidr` attributes. Don't forget to add them to the `allowed_attributes` +section: + +```yaml +network: + enable: true + allowed_attributes: + - k8s.src.owner.name + - k8s.src.namespace + - k8s.dst.owner.name + - k8s.dst.namespace + - src.cidr + - dst.cidr + cidrs: + - 10.10.0.0/24 + - 10.0.0.0/8 + - 10.30.0.0/16 +``` diff --git a/examples/asserts/rules-entity-relation.yml b/examples/asserts/rules-entity-relation.yml index eed841af2..b77a3d75d 100644 --- a/examples/asserts/rules-entity-relation.yml +++ b/examples/asserts/rules-entity-relation.yml @@ -19,23 +19,23 @@ entities: - query: | group by (asserts_env, asserts_site, k8s_dst_namespace, k8s_dst_owner_name) (beyla_network_flow_bytes_total{k8s_dst_owner_type=~"Pod|Deployment|Node|DaemonSet|ReplicaSet|StatefulSet"}) - type: Node - name: k8s_src_host_name + name: k8s_src_node_name scope: - namespace: k8s_src_host_name + namespace: k8s_src_node_name env: asserts_env site: asserts_site definedBy: - query: | - group by (asserts_env, asserts_site, k8s_src_host_name) (beyla_network_flow_bytes_total{k8s_src_type=~"Pod|Node"}) + group by (asserts_env, asserts_site, k8s_src_node_name) (beyla_network_flow_bytes_total{k8s_src_type=~"Pod|Node"}) - type: Node - name: k8s_dst_host_name + name: k8s_dst_node_name scope: - namespace: k8s_dst_host_name + namespace: k8s_dst_node_name env: asserts_env site: asserts_site definedBy: - query: | - group by (asserts_env, asserts_site, k8s_dst_host_name) (beyla_network_flow_bytes_total{k8s_dst_type=~"Pod|Node"}) + group by (asserts_env, asserts_site, k8s_dst_node_name) (beyla_network_flow_bytes_total{k8s_dst_type=~"Pod|Node"}) - type: Namespace name: k8s_src_namespace scope: @@ -132,17 +132,17 @@ relations: endEntityType: Node definedBy: ! pattern: | - group by (asserts_env, asserts_site, k8s_src_host_name, k8s_dst_host_name) ( + group by (asserts_env, asserts_site, k8s_src_node_name, k8s_dst_node_name) ( rate(beyla_network_flow_bytes_total{k8s_src_type=~"Pod|Node",k8s_dst_type=~"Pod|Node"}[1m]) ) startEntityMatchers: - name: k8s_src_host_name - namespace: k8s_src_host_name + name: k8s_src_node_name + namespace: k8s_src_node_name env: asserts_env site: asserts_site endEntityMatchers: - name: k8s_dst_host_name - namespace: k8s_dst_host_name + name: k8s_dst_node_name + namespace: k8s_dst_node_name env: asserts_env site: asserts_site - type: ROUTES diff --git a/pkg/beyla/config.go b/pkg/beyla/config.go index 0e8b2c880..db8c5279a 100644 --- a/pkg/beyla/config.go +++ b/pkg/beyla/config.go @@ -172,7 +172,7 @@ func (c *Config) Validate() error { } if c.Enabled(FeatureNetO11y) && !c.Grafana.OTLP.MetricsEnabled() && !c.Metrics.Enabled() && !c.NetworkFlows.Print { - return ConfigError("enabling network observability requires to enable at least the OpenTelemetry" + + return ConfigError("enabling network metrics requires to enable at least the OpenTelemetry" + " metrics exporter: grafana or otel_metrics_export sections in the YAML configuration file; or the" + " OTEL_EXPORTER_OTLP_ENDPOINT or OTEL_EXPORTER_OTLP_METRICS_ENDPOINT environment variables. For debugging" + " purposes, you can also set BEYLA_NETWORK_PRINT_FLOWS=true") diff --git a/pkg/beyla/network_cfg.go b/pkg/beyla/network_cfg.go index c908d2040..c722f5cac 100644 --- a/pkg/beyla/network_cfg.go +++ b/pkg/beyla/network_cfg.go @@ -26,7 +26,7 @@ import ( ) type NetworkConfig struct { - // Enable network observability. + // Enable network metrics. // Default value is false (disabled) Enable bool `yaml:"enable" env:"BEYLA_NETWORK_METRICS"` @@ -109,7 +109,7 @@ type NetworkConfig struct { // If an IP matches multiple CIDR definitions, the flow will be decorated with the // narrowest CIDR. By this reason, you can safely add a 0.0.0.0/0 entry to group there // all the traffic that does not match any of the other CIDRs. - CIDRs cidr.Definitions `yaml:"cidrs" env:"BEYLA_NETWORK_GROUP_CIDRS" envSeparator:","` + CIDRs cidr.Definitions `yaml:"cidrs" env:"BEYLA_NETWORK_CIDRS" envSeparator:","` } var defaultNetworkConfig = NetworkConfig{ diff --git a/pkg/components/beyla.go b/pkg/components/beyla.go index 4bff54bbf..56fbac651 100644 --- a/pkg/components/beyla.go +++ b/pkg/components/beyla.go @@ -58,14 +58,14 @@ func setupAppO11y(ctx context.Context, config *beyla.Config) { } func setupNetO11y(ctx context.Context, cfg *beyla.Config) { - slog.Info("starting Beyla in Network Observability mode") + slog.Info("starting Beyla in Network metrics mode") flowsAgent, err := agent.FlowsAgent(cfg) if err != nil { - slog.Error("can't start network observability", "error", err) + slog.Error("can't start network metrics capture", "error", err) os.Exit(-1) } if err := flowsAgent.Run(ctx); err != nil { - slog.Error("can't start network observability", "error", err) + slog.Error("can't start network metrics capture", "error", err) os.Exit(-1) } } diff --git a/pkg/internal/netolly/ebpf/tracer.go b/pkg/internal/netolly/ebpf/tracer.go index e51f5712f..a48f43ab8 100644 --- a/pkg/internal/netolly/ebpf/tracer.go +++ b/pkg/internal/netolly/ebpf/tracer.go @@ -293,7 +293,7 @@ func (m *FlowFetcher) closeObjects() []error { // doIgnoreNoDev runs the provided syscall over the provided device and ignores the error // if the cause is a non-existing device (just logs the error as debug). -// If the agent is deployed as part of the Network Observability pipeline, normally +// If the agent is deployed as part of the Network Metrics pipeline, normally // undeploying the FlowCollector could cause the agent to try to remove resources // from Pods that have been removed immediately before (e.g. flowlogs-pipeline or the // console plugin), so we avoid logging some errors that would unnecessarily raise the diff --git a/pkg/internal/netolly/export/printer.go b/pkg/internal/netolly/export/printer.go index 7ab7edb5d..3d1c544e6 100644 --- a/pkg/internal/netolly/export/printer.go +++ b/pkg/internal/netolly/export/printer.go @@ -27,7 +27,7 @@ func FlowPrinterProvider(_ FlowPrinterEnabled) (node.TerminalFunc[[]*ebpf.Record func printFlow(f *ebpf.Record) { sb := strings.Builder{} - sb.WriteString("beyla.ip==") + sb.WriteString("beyla.ip=") sb.WriteString(f.Attrs.BeylaIP) sb.WriteString(" iface=") sb.WriteString(f.Attrs.Interface) diff --git a/pkg/internal/netolly/transform/k8s/kubernetes.go b/pkg/internal/netolly/transform/k8s/kubernetes.go index d052b67d4..c58de0121 100644 --- a/pkg/internal/netolly/transform/k8s/kubernetes.go +++ b/pkg/internal/netolly/transform/k8s/kubernetes.go @@ -39,8 +39,8 @@ const ( attrSuffixType = ".type" attrSuffixOwnerName = ".owner.name" attrSuffixOwnerType = ".owner.type" - attrSuffixHostIP = ".host.ip" - attrSuffixHostName = ".host.name" + attrSuffixHostIP = ".node.ip" + attrSuffixHostName = ".node.name" AttrClusterName = "k8s.cluster.name" diff --git a/test/integration/k8s/netolly/k8s_netolly_network_metrics_test.go b/test/integration/k8s/netolly/k8s_netolly_network_metrics_test.go index 39a38b872..7bb797819 100644 --- a/test/integration/k8s/netolly/k8s_netolly_network_metrics_test.go +++ b/test/integration/k8s/netolly/k8s_netolly_network_metrics_test.go @@ -63,8 +63,8 @@ func testNetFlowBytesForExistingConnections(ctx context.Context, t *testing.T, _ assert.Equal(t, "Pod", metric["k8s_src_owner_type"]) assert.Equal(t, "Pod", metric["k8s_src_type"]) assert.Equal(t, "test-kind-cluster-netolly-control-plane", - metric["k8s_src_host_name"]) - assertIsIP(t, metric["k8s_src_host_ip"]) + metric["k8s_src_node_name"]) + assertIsIP(t, metric["k8s_src_node_ip"]) assert.Equal(t, "default", metric["k8s_dst_namespace"]) assert.Equal(t, "testserver", metric["k8s_dst_name"]) assert.Equal(t, "Service", metric["k8s_dst_owner_type"]) @@ -92,16 +92,16 @@ func testNetFlowBytesForExistingConnections(ctx context.Context, t *testing.T, _ assert.Equal(t, "Pod", metric["k8s_src_owner_type"]) assert.Equal(t, "Pod", metric["k8s_src_type"]) assert.Equal(t, "test-kind-cluster-netolly-control-plane", - metric["k8s_src_host_name"]) - assertIsIP(t, metric["k8s_src_host_ip"]) + metric["k8s_src_node_name"]) + assertIsIP(t, metric["k8s_src_node_ip"]) assert.Equal(t, "default", metric["k8s_dst_namespace"]) assert.Regexp(t, regexp.MustCompile("^testserver-"), metric["k8s_dst_name"]) assert.Equal(t, "Deployment", metric["k8s_dst_owner_type"]) assert.Equal(t, "testserver", metric["k8s_dst_owner_name"]) assert.Equal(t, "Pod", metric["k8s_dst_type"]) assert.Equal(t, "test-kind-cluster-netolly-control-plane", - metric["k8s_dst_host_name"]) - assertIsIP(t, metric["k8s_dst_host_ip"]) + metric["k8s_dst_node_name"]) + assertIsIP(t, metric["k8s_dst_node_ip"]) assert.Contains(t, podSubnets, metric["src_cidr"], metric) assert.Contains(t, podSubnets, metric["dst_cidr"], metric) }) @@ -124,14 +124,14 @@ func testNetFlowBytesForExistingConnections(ctx context.Context, t *testing.T, _ assert.Regexp(t, regexp.MustCompile("^testserver-"), metric["k8s_src_name"]) assert.Equal(t, "Deployment", metric["k8s_src_owner_type"]) assert.Equal(t, "Pod", metric["k8s_src_type"]) - assert.Equal(t, "test-kind-cluster-netolly-control-plane", metric["k8s_src_host_name"]) - assertIsIP(t, metric["k8s_src_host_ip"]) + assert.Equal(t, "test-kind-cluster-netolly-control-plane", metric["k8s_src_node_name"]) + assertIsIP(t, metric["k8s_src_node_ip"]) assert.Equal(t, "default", metric["k8s_dst_namespace"]) assert.Equal(t, "internal-pinger", metric["k8s_dst_name"]) assert.Equal(t, "Pod", metric["k8s_dst_owner_type"]) assert.Equal(t, "Pod", metric["k8s_dst_type"]) - assert.Equal(t, "test-kind-cluster-netolly-control-plane", metric["k8s_dst_host_name"]) - assertIsIP(t, metric["k8s_dst_host_ip"]) + assert.Equal(t, "test-kind-cluster-netolly-control-plane", metric["k8s_dst_node_name"]) + assertIsIP(t, metric["k8s_dst_node_ip"]) assert.Contains(t, podSubnets, metric["src_cidr"], metric) assert.Contains(t, podSubnets, metric["dst_cidr"], metric) }) @@ -159,8 +159,8 @@ func testNetFlowBytesForExistingConnections(ctx context.Context, t *testing.T, _ assert.Equal(t, "internal-pinger", metric["k8s_dst_name"]) assert.Equal(t, "Pod", metric["k8s_dst_owner_type"]) assert.Equal(t, "Pod", metric["k8s_dst_type"]) - assert.Equal(t, "test-kind-cluster-netolly-control-plane", metric["k8s_dst_host_name"]) - assertIsIP(t, metric["k8s_dst_host_ip"]) + assert.Equal(t, "test-kind-cluster-netolly-control-plane", metric["k8s_dst_node_name"]) + assertIsIP(t, metric["k8s_dst_node_ip"]) assert.Contains(t, svcSubnets, metric["src_cidr"], metric) assert.Contains(t, podSubnets, metric["dst_cidr"], metric) })