Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

object literal defined using := panics with "corrupt object" #1125

Closed
srenatus opened this issue Dec 20, 2018 · 0 comments · Fixed by #1127
Closed

object literal defined using := panics with "corrupt object" #1125

srenatus opened this issue Dec 20, 2018 · 0 comments · Fixed by #1127
Labels

Comments

@srenatus
Copy link
Contributor

Expected Behavior

Running this package through opa run demo.rego,

package demo

foo[f] {
	k = "key"
	v := "val"
	f := { k: v }
}

I get this:

$ opa run demo.rego
OPA 0.10.3-dev (commit a9f819f0-dirty, built at 2018-12-20T13:03:31Z)

Run 'help' to see a list of commands.

> data.demo.foo
[
  {
    "key": "val"
  }
]
>

Now, I'd expect the same output from the package

package demo

foo[f] {
	k := "key" # <<< 
	v := "val"
	f := { k: v }
}

Actual Behavior

$ opa run demo.rego
panic: corrupt object [recovered]
        panic: corrupt object

goroutine 1 [running]:
github.com/open-policy-agent/opa/ast.(*Compiler).compile.func1()
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/compile.go:611 +0x6a
panic(0x4670f60, 0x482e9f0)
        /usr/local/Cellar/go/1.11.3/libexec/src/runtime/panic.go:513 +0x1b9
github.com/open-policy-agent/opa/ast.(*object).Iter(0xc0001cd1d0, 0xc000224d20, 0x20, 0x46d77c0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/term.go:1563 +0xf5
github.com/open-policy-agent/opa/ast.(*object).Foreach(0xc0001cd1d0, 0xc0001d0aa0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/term.go:1586 +0x53
github.com/open-policy-agent/opa/ast.WalkBeforeAndAfter(0x483d140, 0xc0001cef60, 0x4753660, 0xc0001cd1d0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:104 +0x887
github.com/open-policy-agent/opa/ast.Walk(0x4833420, 0xc0000c84d8, 0x4753660, 0xc0001cd1d0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:30 +0x7b
github.com/open-policy-agent/opa/ast.WalkBeforeAndAfter(0x483d140, 0xc0001cef50, 0x4735a60, 0xc0001d0440)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:98 +0xe64
github.com/open-policy-agent/opa/ast.Walk(0x4833420, 0xc0000c84d8, 0x4735a60, 0xc0001d0440)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:30 +0x7b
github.com/open-policy-agent/opa/ast.WalkBeforeAndAfter(0x483d140, 0xc0001ceee0, 0x47541c0, 0xc0002260a0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:86 +0x9d7
github.com/open-policy-agent/opa/ast.Walk(0x4833420, 0xc0000c84d8, 0x47541c0, 0xc0002260a0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:30 +0x7b
github.com/open-policy-agent/opa/ast.WalkBeforeAndAfter(0x483d140, 0xc0001cedb0, 0x47367e0, 0xc0001d0a80)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:76 +0xb8f
github.com/open-policy-agent/opa/ast.Walk(0x4833420, 0xc0000c84d8, 0x47367e0, 0xc0001d0a80)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:30 +0x7b
github.com/open-policy-agent/opa/ast.WalkBeforeAndAfter(0x483d140, 0xc0001cea80, 0x471f040, 0xc0001d55c0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:61 +0xeec
github.com/open-policy-agent/opa/ast.Walk(0x4833420, 0xc0000c84d8, 0x471f040, 0xc0001d55c0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:30 +0x7b
github.com/open-policy-agent/opa/ast.WalkBeforeAndAfter(0x483d140, 0xc0001cea10, 0x46f22a0, 0xc0000d9e50)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:49 +0x28d
github.com/open-policy-agent/opa/ast.Walk(0x4833420, 0xc0000c84d8, 0x46f22a0, 0xc0000d9e50)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:30 +0x7b
github.com/open-policy-agent/opa/ast.WalkRules(0x46f22a0, 0xc0000d9e50, 0xc0001d05c0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/visit.go:227 +0x9d
github.com/open-policy-agent/opa/ast.(*Compiler).rewriteLocalAssignments(0xc0000e1dd0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/compile.go:798 +0x10d
github.com/open-policy-agent/opa/ast.(*Compiler).rewriteLocalAssignments-fm()
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/compile.go:205 +0x2a
github.com/open-policy-agent/opa/ast.(*Compiler).compile(0xc0000e1dd0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/compile.go:616 +0x81
github.com/open-policy-agent/opa/ast.(*Compiler).Compile(0xc0000e1dd0, 0xc000225838)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/ast/compile.go:249 +0x274
github.com/open-policy-agent/opa/runtime.compileAndStoreInputs(0x483da40, 0xc0000ba008, 0x48428e0, 0xc0000d9e00, 0x48336c0, 0xc0001c2c90, 0xc000184480, 0xa, 0x0, 0x46a0440)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/runtime/runtime.go:411 +0x1b7
github.com/open-policy-agent/opa/runtime.NewRuntime(0x483da40, 0xc0000ba008, 0xc0000d6240, 0x24, 0xc00017a220, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/runtime/runtime.go:168 +0x2ff
github.com/open-policy-agent/opa/cmd.init.7.func1(0xc000164a00, 0xc00016c2d0, 0x1, 0x1)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/cmd/run.go:113 +0x42b
github.com/open-policy-agent/opa/vendor/github.com/spf13/cobra.(*Command).execute(0xc000164a00, 0xc00016c2a0, 0x1, 0x1, 0xc000164a00, 0xc00016c2a0)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/vendor/github.com/spf13/cobra/command.go:766 +0x2cc
github.com/open-policy-agent/opa/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x4d06fe0, 0xc0000fdf78, 0x45e69bd, 0x477c3cf)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/vendor/github.com/spf13/cobra/command.go:852 +0x2fd
github.com/open-policy-agent/opa/vendor/github.com/spf13/cobra.(*Command).Execute(0x4d06fe0, 0x0, 0xc00013de60)
        /Users/stephan/go/src/github.com/open-policy-agent/opa/vendor/github.com/spf13/cobra/command.go:800 +0x2b
main.main()
        /Users/stephan/go/src/github.com/open-policy-agent/opa/main.go:12 +0x2d
@srenatus srenatus changed the title object literal defined using := panics object literal defined using := panics with "corrupt object" Dec 20, 2018
@tsandall tsandall added the bug label Dec 20, 2018
tsandall added a commit to tsandall/opa that referenced this issue Dec 21, 2018
During the assignment rewriting stages, variables that have been
assigned locally are rewritten, e.g., k := "foo"; {k: 1} becomes
__local0__ = "foo"; {__local0__: 1}. Previously, the rewriting would
simply mutate all of the term values without considering where the term
was defined. The problem with this was that it would corrupt the
object's hashtable if the rewritten term was an object key.

This commit resolves the issue by making a copy of the object key before
mutating the term value. This approach requires the compiler to copy the
object as well. If this proves to be a performance bottleneck, we can do
something more clever.

Note, it's arguable that the Object struct should not allow keys to be
mutated at all. This would be tricky given the current Object interface.
Perhaps we can improve this in the future to avoid similar issues.

These changes also resolve an issue with the rewritten var set returned
by the query compiler. Previously, all seen vars were returned in the
set (as opposed to only those that had been redeclared.) When the query
compiler inverts the map before returning it could accidentally elide
vars, e.g., given {a: b, b: b} if it inverted this to {b: a} then the
output would be correct but if it inverted this to {b: b} (which it
could depending on the iteration order) the output would be incorrect.

Fixes open-policy-agent#1125

Signed-off-by: Torin Sandall <[email protected]>
tsandall added a commit that referenced this issue Dec 23, 2018
During the assignment rewriting stages, variables that have been
assigned locally are rewritten, e.g., k := "foo"; {k: 1} becomes
__local0__ = "foo"; {__local0__: 1}. Previously, the rewriting would
simply mutate all of the term values without considering where the term
was defined. The problem with this was that it would corrupt the
object's hashtable if the rewritten term was an object key.

This commit resolves the issue by making a copy of the object key before
mutating the term value. This approach requires the compiler to copy the
object as well. If this proves to be a performance bottleneck, we can do
something more clever.

Note, it's arguable that the Object struct should not allow keys to be
mutated at all. This would be tricky given the current Object interface.
Perhaps we can improve this in the future to avoid similar issues.

These changes also resolve an issue with the rewritten var set returned
by the query compiler. Previously, all seen vars were returned in the
set (as opposed to only those that had been redeclared.) When the query
compiler inverts the map before returning it could accidentally elide
vars, e.g., given {a: b, b: b} if it inverted this to {b: a} then the
output would be correct but if it inverted this to {b: b} (which it
could depending on the iteration order) the output would be incorrect.

Fixes #1125

Signed-off-by: Torin Sandall <[email protected]>
tsandall added a commit to tsandall/opa that referenced this issue Jun 24, 2019
Mutating term values used as object keys (or set elements) is not safe
because the underlying hashtables get corrupted. This a known issue
that was previously fixed in the compiler (open-policy-agent#1125). The simplest
solution in this case is to use the ast.Transform helper instead of
ast.Walk so that a copy of object and set terms is made.

Fixes open-policy-agent#1177

Signed-off-by: Torin Sandall <[email protected]>
tsandall added a commit that referenced this issue Jun 26, 2019
Mutating term values used as object keys (or set elements) is not safe
because the underlying hashtables get corrupted. This a known issue
that was previously fixed in the compiler (#1125). The simplest
solution in this case is to use the ast.Transform helper instead of
ast.Walk so that a copy of object and set terms is made.

Fixes #1177

Signed-off-by: Torin Sandall <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants