Access to more variables in output_fields

[wdoekes]

Motivation

Right now, we're parsing the jsonOutput: true json generated by falco and feeding it to Slack.

Instead of using the unformatted "output", we use the "output_fields". This way, we can do something like this.

We take this json:

{
  "output": "14:33:48.105952346: Error File below / or /root opened for writing (user=root user_loginuid=-1 command=touch /wjd-test parent=runc:[1:CHILD] file=/wjd-test program=touch container_id=c0a8e388a8c2 image=falcosecurity/falco) k8s.ns=kube-falco k8s.pod=falco-zjsqj container=c0a8e388a8c2",
  "priority": "Error",
  "rule": "Write below root",
  "source": "syscall",
  "tags": [
    "filesystem",
    "mitre_persistence"
  ],
  "time": "2022-07-12T12:33:48.105952346Z",
  "output_fields": {
    "container.id": "c0a8e388a8c2",
    "container.image.repository": "falcosecurity/falco",
    "evt.time": 1657629228105952300,
    "fd.name": "/wjd-test",
    "k8s.ns.name": "kube-falco",
    "k8s.pod.name": "falco-zjsqj",
    "proc.cmdline": "touch /wjd-test",
    "proc.name": "touch",
    "proc.pname": "runc:[1:CHILD]",
    "user.loginuid": -1,
    "user.name": "root"
  }
}

And we turn it into this:

We can create that nicely formatted message because all fields are separately accessible.

output: "File below / or /root opened for writing (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline parent=%proc.pname file=%fd.name program=%proc.name container_id=%container.id image=%container.image.repository)"

All %-fields mentioned in the output are available in output_fields, as seen above.

But, we do not have access to -- for instance -- proc.aname[2].

The easy solution sounds like amending the output: in the default rules. But alas, this does not work.

    - rule: Write below root
      output: "File below / or /root opened for writing (user=%user.name user_loginuid=%user.loginuid command=%proc.cmdline parent=%proc.pname file=%fd.name program=%proc.name container_id=%container.id image=%container.image.repository)"
      append: true
      condition: >
        and not (expected_write_below_root_behaviour)

The conditions are amended, but the output is not amended/overwritten: so I cannot get more values in the output_fields without copying the entire Write below root rule to something new.

Feature

I'd like to be able to get more values in the output_fields.

• This could be by overwriting the rule name and adding them in the output,
• or by setting a separate also_add_these_variables_in_output_fields: [proc.aname[2], ...] on the rule,
• or through some other means.

Cheers,
Walter Doekes
OSSO B.V.

[jasondellaluce]

This makes a lot of sense. Will look into it and hopefully get this in the next release!

/milestone 0.33.0

[robinlandstrom]

I just noticed the same thing. Parsing output_fields from k8s_audit alerts and wanted to add the source ip of the request to all alerts.

- rule: Attach/Exec Pod
  append: true
  output: src_ip=%json.value[sourceIPs/0]

- rule: K8s Namespace Deleted
  append: true
  output: src_ip=%json.value[sourceIPs/0]
...

Falco 0.32.2 fails to load it because it does not contain an exception or condition property

Error: Could not load rules file /etc/falco/rules.d/k8s_audit_rules.z.local.yaml: 1 errors:
Appended rule must have exceptions or condition property
---
- rule: Attach/Exec Pod
  append: true
  output: src_ip=%json.value[sourceIPs/0]
---

Would be great to see support for appending to output or even just adding extra fields to output_fields in 0.33.0 😃

[jasondellaluce]

Out of curiosity, was this supported in Falco <= 0.31.1, or is this a brand new feature request? I agree this can be useful for many.

@robinlandstrom Would you like to open. a feature request issue on this repo to support the cases you reported in these code examples?

[robinlandstrom]

I don't think it was supported in Falco <= 0.31.1, have not tried thou.

Just being able to append to rule output as shown by @wdoekes would be enough for my use case.
If condition is required one can always just add condition: and always_true, but sure I think it would be a sane default if the condition is missing from a rule append.

[jasondellaluce]

/milestone 0.33.0

[jasondellaluce]

/remove-milestone 0.33.0

/milestone 0.34.0

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[jasondellaluce]

/remove-lifecycle stale

[jasondellaluce]

/milestone 0.35.0

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[Andreagit97]

/remove-lifecycle stale

[FedeDP]

/milestone 0.36.0

Sorry we havent' got the time to work on this during this release cycle.

[sboschman]

In addition to defining extra output fields for a single rule, it would be nice to be able to define extra output fields for every rule with a certain tag (e.g. k8s) or source (e.g. k8s_audit).

[incertum]

@wdoekes @sboschman @robinlandstrom I am on it. There are other issues remotely realted to this, please allow for some time to triage them all and then decide on necessary patches to support all features requests. The first PR is just to expand append mode option to output and tags.

[sboschman]

@incertum if you want more details on the setup we are trying to run, to perhaps better decide if and how falco can better support such a scenario, let me know. It is of course just one of many scenarios described in this issue and all the related issues, but hopefully there is enough similarity to cover most/all of them.

[incertum]

@sboschman this would be amazing! And agreed let's collect all information. We can then see what could be the best solution for #2127 (comment).

By the way my setup is a custom script and I do exactly what you describe in there I add certain output fields when xyz tags are available. But of course I create a whole new patched up rules file where I also prioritize the ordering of rules given it's currently "first match wins". This is also something @jasondellaluce and @leogr are thinking about a lot how we can expose better options to end users.

Thank you!

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[leogr]

Sorry, I had lost track of this.
/remove-lifecycle stale

cc @jasondellaluce

[poiana]

Issues go stale after 90d of inactivity.

Mark the issue as fresh with /remove-lifecycle stale.

Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle stale

[leogr]

This issue should be addressed by #2981 cc @LucaGuerra correct? 🤔

[poiana]

Stale issues rot after 30d of inactivity.

Mark the issue as fresh with /remove-lifecycle rotten.

Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Provide feedback via https://github.com/falcosecurity/community.

/lifecycle rotten

[leogr]

This issue should be addressed by #2981 cc @LucaGuerra correct? 🤔

@LucaGuerra ping

[poiana]

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community.
/close

[poiana]

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.

Reopen the issue with /reopen.

Mark the issue as fresh with /remove-lifecycle rotten.

Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.