Partial evaluation understanding

[thomascoquet]

Hello,

I am having issue understanding how partial evaluation works. I thought we could prepare a query for further evaluation given what is known at the time (query and inputs).
For example, I have a policy to grant access and it will be used at two different code paths. Do I need to create two different policies?

package main

import (
	"context"
	"fmt"

	"github.com/open-policy-agent/opa/rego"
)

func main() {
	ctx := context.Background()
	module := `
		package example

		allow {
			input.subject.user = "alice"
			input.auth_type = "hmac"
		}

		allow {
			input.subject.user = "john"
			input.auth_type = "api_key"
		}
	`

	pre := map[string]interface{}{"auth_type": "hmac"}

	r := rego.New(
		rego.Query("data.example.allow"),
		rego.Module("example.rego", module),
		rego.Input(pre),
	)

	pr, err := r.PartialResult(ctx)
	if err != nil {
		panic(err.Error())
	}

	input := map[string]interface{}{
		"subject": map[string]interface{}{
			"user": "alice",
		},
	}
	rr := pr.Rego(rego.Input(input))
	rs, err := rr.Eval(ctx)
	if err != nil {
		panic(err.Error())
	} else {
		fmt.Printf("allowed: %v\n", rs.Allowed())
	}
}

Expected Behavior

allowed: true

Actual Behavior

allowed: false

Thanks

[srenatus]

By default, PE will run with unknowns set to input. Since your input is not unknown, but only partially, you could pass rego.Unknowns([]string{"input.subject"}).

I often find it simpler to play with PE on the command line, with input.json as {"auth_type": "hmac"}, and example.rego as your policy, we find:

$ opa eval -fsource --partial -d ex.rego -u input.subject -i input.json 'data.example.allow'
# Query 1
input.subject.user = "alice"

which strikes me as exactly what you'd like, isn't it? 🤔 Not we'll just need to figure out how to translate it into using the rego golang interface...

[thomascoquet]

Thanks for the feedback, but I am not sure it is exactly what I am looking for.
What you are describing is "partial evaluation", meaning that the policy is evaluated and returned as a set of ast rules.
Looks like that in golang:

package main

import (
	"context"
	"fmt"

	"github.com/open-policy-agent/opa/rego"
)

func main() {
	ctx := context.Background()
	module := `
		package example

		allow {
			input.subject.user = "alice"
			input.auth_type = "hmac"
		}

		allow {
			input.subject.user = "john"
			input.auth_type = "api_key"
		}
	`

	pre := map[string]interface{}{"auth_type": "hmac"}

	r := rego.New(
		rego.Query("data.example.allow"),
		rego.Module("example.rego", module),
		rego.Input(pre),
		rego.Unknowns([]string{"input.subject"}),
	)

	pr, err := r.Partial(ctx)
	if err != nil {
		panic(err.Error())
	}

	for _, b := range pr.Queries {
		fmt.Println(b.String())
	}
}

What I want would rather be "partial compilation", i.e. an object I can reuse afterwards for evaluation. It is precisely the goal of PartialResult (https://github.com/open-policy-agent/opa/blob/a98c0f98a01b998104bcc69a99c0e6a68e461bd5/rego/rego.go#L62):

PartialResult represents the result of partial evaluation. The result can be used to generate a new query that can be run when inputs are known.

As stated in the doc (https://www.openpolicyagent.org/docs/latest/policy-performance/#use-indexed-statements), it can also be used for performance (linearizing policy fragments).

My overall workflow is:

1. I have a set of policies,
2. I use PartialResult with a specific query to create a decision point at application init time: pr, err := rego.New(rego.Query("data.example.allow"), opts...).PartialResult(ctx)
3. At run time, when I receive a request:
   • I use pr.Rego(rego.Input(...)).Eval(ctx) if I don't have any unknowns,
   • I use pr.Rego(rego.Input(...), rego.Unknowns(...)).Partial(ctx) if I have unknowns. In which case, I'll transform what is remaining into a SQL query.

The thing is that I have "constant inputs", for example the type of auth, the type of objects, etc., depending on the code path. I want api_key in a mux middleware, and hmac in a message bus.
There are three options:

a. Create several policies data.example.allow_hmac, data.example.allow_api_key, etc. But those grant the same access.
b. Use the constant input when building the partial policy at step 2, what seems logical: pr, err := rego.New(rego.Query("data.example.allow"), rego.Input(map[string]interface{}{"auth_type": "hmac"}), opts...).PartialResult(ctx).
c. Inject them at step 3, with the request-time input.

My question was about option b.

Hope it makes things clearer!

[fredmaggiowski]

Hi, @thomascoquet we have the exact same use case of yours, in your "overall workflow" change SQL with MongoDB and we're basically implementing the same solution.

With that said, we managed to make it work using PartialResult and then Eval, the only difference I see between your originally proposes solution and ours is that in the initial rego.New instance we do not provide any input for the PartialResult as we only provide them in the final Eval.. Honestly I don't see how that should be a problem.

What we can't figure out right now is how to perform PartialResult and then invoke Partial in order to obtain the data on which we then create our query.

[fredmaggiowski]

Here is a sample snippet of what is not working, suppose the following policy:

allow {
	true   
	resource := data.resources[_]
	resource.name == "john"
}

and the following Go snippet:

r := rego.New(
	rego.Query(`data.example.allow`),
	rego.Module("module.rego", module),
	rego.Capabilities(ast.CapabilitiesForThisVersion()),
	rego.Unknowns([]string{"data.resources"}),
	rego.Store(store),
	rego.EnablePrintStatements(true),
)
pr, err := r.PartialResult(ctx)
if err != nil {
	panic(err)
}

r1 := pr.Rego(
	rego.Input(input),
	rego.Unknowns([]string{"data.resources"}),
)
r1.Partial(ctx)

This does not produce any output (I guess the policy is failing?).

However if we directly use Partial (as in the snippet below) it works as expected:

r := rego.New(
	rego.Query(`data.example.allow`),
	rego.Module("module.rego", module),
	rego.Capabilities(ast.CapabilitiesForThisVersion()),
	rego.Unknowns([]string{"data.resources"}),
	rego.Store(store),
	rego.EnablePrintStatements(true),
	rego.Input(input),
)

r.Partial(ctx)