Logging hostname might break logging providers hostname parsers and features

[brunoluiz]

In recent versions, hostname was included in falco logs. In logging providers such as Datadog, hostname is parsed by default and in general it maps to a k8s worker or EC2 instance. This enables log enrichment by correlating pod x k8s worker details based on hostname.

It seems this field is using the HOSTNAME available in the environment variable, which in general is the pod name. This breaks up correlations done on logging providers that rely on the hostname.

How to reproduce it

Running the latest version of falco, observe the JSON output logs. It will have something like

{
  "hostname": "falco-mnv9v",
  "output": "...",
  "priority": "Notice",
  "rule": "...",
  "source": "...",
  "tags": [...],
  "time": "2023-05-03T12:20:03.418079886Z",
  "output_fields": {...}
}

Expected behaviour

There should be an option to not log the hostname, which might conflict with logging providers, or to be able to pass which value should be used (eg: spec.nodeName).

Environment

• Falco version:

Wed May  3 16:42:30 2023: Falco version: 0.34.1 (x86_64)
Wed May  3 16:42:30 2023: Falco initialized with configuration file: /etc/falco/falco.yaml
Falco version: 0.34.1
Libs version:  0.10.4
Plugin API:    2.0.0
Engine:        16
Driver:
  API version:    3.0.0
  Schema version: 2.0.0
  Default driver: 4.0.0+driver

• System info:

{
  "machine": "x86_64",
  "nodename": "falco-7gft5",
  "release": "5.4.228-131.415.amzn2.x86_64",
  "sysname": "Linux",
  "version": "#1 SMP Tue Dec 20 12:51:02 UTC 2022"
}

• Cloud provider or hardware configuration: AWS

• OS:

PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

• Kernel: 5.4.228-131.415.amzn2.x86_64

• Installation method: Falco 3.1.4 helm chart

[brunoluiz]

Ok, it seems that this can be fixed by setting FALCO_HOSTNAME (ref), which in my case I tested by adding this in the Kubernetes daemonset (kubectl edit daemonset falco):

        - name: FALCO_HOSTNAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName

I still have to understand how to add this using the helm chart, but it fixes the issue pointed out here. I couldn't easily find this option, but probably mostly due to not be available on the Helm chart.

Should I open a similar issue on https://github.com/falcosecurity/charts?

[xixinILYA]

edit helm chart values.yaml :
extra:
args: []
env:
  - name: FALCO_HOSTNAME
    valueFrom:
        fieldRef:
          apiVersion: v1
          fieldPath: spec.nodeName

[incertum]

@brunoluiz all good, can this issue be closed out?

We have existing options like json_include_output_property etc is there a need for json_include_hostname or similar?