Check that a list contains a number of entries

[arlimus]

In MQL we sometimes want to say that the number of entries should be within a certain range, e.g. There should be between 1 and 3 root users (we know at least 1 will be there but if there are more than 3 then we have a problem).

users.length <= 3

[failed] 4 is not <= 3

The problem: we don’t get the list of users returned. Ideally we would want to know, if there are too many users, who they are. As soon as this check fails, we would want to to know more. The only thing that the check returns, however, is the number of entries found. This is fundamentally correct (and we don't want to change that), but we do want to offer a way to solve this problem.

Discussion

There are a few ways to tackle this problem:

1. Use the @msg tag

While originally meant as a workaround only, @scottford-io reminded me today that it’s a valid approach for all cases where the context cannot be expressed in the query alone. As much as we want default queries to output the right thing, there will be cases where additional contextual information is helpful. The @msg tag is ideal for those cases.

How well does it serve this use-case?

Applied to the problem above, we get:

# @msg There should be no more than ${$expected} users,
# however we detected ${$actual} users:
# ${users{*}}
users.length <= 3

There should be no more than 3 users,
however we detected 4 users:
[
  0: {
     name: "root",
     uid: 0
     ...

Tons of flexibility, however it also requires a lot of writing, the context only really lives in the message, and we still need to repeat a part of the query. This can be mitigated with variables:

list = users.where( sth ) + users.where( sth else ) ...
# @msg ... ${list{*}}
list.length <= 3

As an upside, we get the flexibility to do anything we want. It also allows us to attach messages to any of the assertions in a chain of checks (though note: those should probably be written separately; but that's for a different day...)

Let's call this approach the fallback: It is always there for us.

2. Introduce a new check for list entries

For example, we could do something like:

users.containsAtMost(3)

[failed] Expected users to contain at most 3 entries, but found 4 instead:
[
  0: {
     name: "root",
     uid: 0
     ...

This has all the benefits we get from other list assertions: print more or less entries depending on the printer config, get a nice message out of the box, and continue to work with the data (e.g. subfield extraction down the line). However, it comes at the cost of a very specific keyword that only applies to list (and maps + dicts).

The list of possible keywords are:

- containsAtMost( max number )
- containsAtLeast( min number )
(and / or)
- containsOutsideOfRange( min, max number )
- containsWithinRange( min, max number )

"Within range" could replace "at most" and "at least", to reduce the keyword pressure. However, it would mandate "outsie of range" as well.

Alternatively we could scrap both "outside of range" and "within range" in favor of "at most" and "at least" and ensure that the context extraction works inside of nested boolean statements (which it should anyway).

One more semantic downside is that this can be misunderstood if you deal with a list of numbers. In that case e.g. [1,4,8,9].containsAtMost(5) can easily be misunderstood as meaning the number in the list, instead of the number of entries. Better wording suggestions are welcome (looked across a few languages and didn't find anything good yet).

3. Introduce assert

This is a more far-reaching approach to this problem, which isn't limited to checking list entry numbers:

users.assert( length <= 3 )

[failed] Expected users length <= 3. Found:
[
  0: {
     name: "root",
     uid: 0
     ...

The assert keyword creates a way to write assertions for an object without focusing on the value that is being tested. When the assertion fails, we retrieve the entire object and return it. The function itself is boolean, but it returns secondary values much in the same way all and none work for lists. The only difference is that it's not limited to lists: it works on any resource.

For fun, translating between them:

users.none( uid == 0 )
users.where( uid == 0 ).assert( length == 0 ) 

The downside is that a lot of the added context is hidden within the round brackets. We'd want to extend the way we inspect the checks to print better messages in a follow-up:

[failed] Expected at most 3 users, found 4. 
[
  0: {
     name: "root",
     uid: 0
     ...

Thoughts?

[atomic111]

Package example

You want to know if xorg packages are installed on the system. Actual MQL:

packages.where( name == /^xserver-xorg.*/ ).length == 0

New MQL style:

packages.where( name == /^xserver-xorg.*/ ).assert ( length == 0 )

[failed] Expected at most 0 packages, found 2. 
[
  0: {
     name: "xserver-xorg-driver",
     version: 1.1.0
     ...

iptables example

you want to check if there are two iptables rules. old style:

iptables.input.length == 2

new style:

iptables.input.assert( length == 2 )

[failed] Expected at 2 iptables, found 4. 
[
  0: {
     source: "0.0.0.0/0",
     destination: "0.0.0.0/0",
     options: "udp dpt:323 state NEW",
     chain: "input",
     target: "ACCEPT"
     ...

[frozen425]

I'm assuming this would still be valid with "assert" usage:

iptables.input.assert( length == 2 ) { source destination }

[scottford-io]

@arlimus First of all, thank you for the deep dive into this problem. I really like a lot of the built-in function ideas you have above, and would expect those to be useful to some users.

I am looking at this through the lens of making results clear and actionable for users. I expect that the person responding to a failed check will likely not be the person that wrote the check. No matter if the check results are rendered in Mondoo Platform, or in STDOUT on the command line with -o full, I would hope it is very clear what has failed and what needs to be done.

I believe @msg is still very useful, and from your write-up above, it appears that with variables and string interpolation, we can create policies with very specific messages. I am a 👍 to that! 🙌

[mm-weber]

@arlimus Thanks for the detailed write-up and fresh set of ideas.

Regarding 1. Use the @msg tag:

I am a 100% with @scottford-io for having the @msg tag as a tool in your belt. When writing the NSA K8s policies I would sometimes fall-back to assigning static helper messages to variables. Explaining to the user how to interpret the output, without referring to the actual values produced by the query in question is sub-optimal.

For example, instead of only displaying a block of risky k8s.rolebindings as output, without any context:

         0: {
                subjects: [
                  0: {
                    apiGroup: "rbac.authorization.k8s.io"
                    kind: "User"
                    name: "jane"
                  }
                ]
                name: "pod"
                roleRef: {
                  apiGroup: "rbac.authorization.k8s.io"
                  kind: "Role"
                  name: "pod-reader"
                }
                if: {
                  false
                }
              }

        <snip>

In my opinion it would be preferable to add explanatory messages to the output such as:

k8s.rolebinding a is considered risky because it provides user x with access to role y which allows z.

Regarding 2. + 3.

I think the those two keywords would be sufficient to construct queries.

- containsAtMost( max number )
- containsAtLeast( min number )

Regarding semantics, how about something along the lines of:

- containsAtMostXItems( max number )
- containsAtLeastXItems( min number )

The example @atomic111 gave, makes a good showcase why we need assert.
Do we have to decide on one of those two suggestions?