allow custom masking of fields

[daniel-garcia]

Expected Behavior

Masking sensitive fields should allow the field to be mutated instead of just dropped.

Actual Behavior

Masking only allows sensitive fields to be dropped.

Steps to Reproduce the Problem

https://www.openpolicyagent.org/docs/v0.13.5/decision-logs/#masking-sensitive-data

Not possible to modify a field... only drop it.

Additional Info

My use case is:
I am passing the entire JWT and other information to evaluate a decision. The JWT contains a signature that is being verified in OPA. The decision logs, however, expose the entire JWT which can be used in a playblack attack. If I drop the field, i don't have the entire context that was using for evaluating the decision. What I really want to do is just drop the JWT signature.

[tsandall]

Thanks for filing this @daniel-garcia ! Just adding a note here that another similar use case would be additive changes like appending a signature based on the some fields (or all fields) in the decision log to ensure integrity.

[dkiser]

Any suggestions on where/how to start picking up this work? I see most of the logic is contained in /plugins/logs/plugin.go and /plugins/logs/ptr.go.

[tsandall]

@dkiser most (or all) of the logic is implemented inside of the decision log plugin like you found. I would start by figuring out the interface we want to expose to admins configuring OPA.

Currently admins can implement a rule/decision that defines a set of paths in the decision log event to remove. The paths must be prefixed with input/ or result/, i.e., they cannot remove other event fields. The generated paths are represented as slash-separated strings. For example:

package system.log

mask["/input/ssn"]  # mask out the ssn field in the input document

The decision path is configurable and it defaults to system/log/mask.

A few thoughts:

• We need to maintain backwards compatibility. The default needs to remain system/log/mask. Moreover deployments configured with a masking policy must continue to work which means the existing decision structure must be supported. In terms of the interface, this means we could either (i) overload the 'mask' decision to support non-string values (see below) or (ii) introduce a new decision path that admins can opt into (e.g., system/log/transform). My preference would be for (i) to avoid maintaining two versions or having to deprecate and eventually remove support for the mask version.

• Masking and other transforms add additional overhead to the decision path because the mask is applied to the event before the decision is returned to the caller. We want to reduce the latency impact as much as possible. Ideally we can gather all of the transforms in one shot, i.e., we don't want to execute multiple queries if possible.

I propose we extend the implementation to support structured values in the mask set. For example:

mask[{"op": "remove", "path": "/input/ssn"}]

mask[{"op": "upsert", "path": "/input/decision_signature", "value": x}] {
   # logic to compute signature value
}

This way the mask can generate a set of instructions for OPA. Each instruction specifies op (the operation type), path (the value to apply the transform to), and optionally value (some operations may require a value. The remove op is treated the same as the strings in the mask set today. The upsert op inserts or updates the path with the specified value (creating any parent nodes required in the event.)

Internally, I'd modify the implementation to map old mask strings into the structure above and then refactor the rest of the implementation accordingly.

cc @timothyhinrichs

[timothyhinrichs]

Something like this has been on my wish list for a while, so thanks for picking it up!

I could see it being valuable to create new fields in the decision log outside of result and input. I'd suggest we require the user to put new fields under something like custom to ensure forwards compatibility for when we want to extend the decision-log format in the future.

[dkiser]

@timothyhinrichs Maybe we can talk through this on our call today, and we'll take a stab at knocking this one out!