Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

evalv3: panic: unexpected uninitialized node #3330

Closed
uhthomas opened this issue Jul 26, 2024 · 10 comments
Closed

evalv3: panic: unexpected uninitialized node #3330

uhthomas opened this issue Jul 26, 2024 · 10 comments

Comments

@uhthomas
Copy link
Contributor

uhthomas commented Jul 26, 2024

What version of CUE are you using (cue version)?

$ go run cuelang.org/go/cmd/[email protected] version
cue version v0.10.0-alpha.2.0.20240726102249-5de5b42c834f

go version go1.22.5
      -buildmode exe
       -compiler gc
     CGO_ENABLED 1
          GOARCH amd64
            GOOS linux
         GOAMD64 v1
cue.lang.version v0.10.0

Does this issue reproduce with the latest stable release?

What did you do?

Clone https://github.com/uhthomas/automata, and run

❯ CUE_EXPERIMENT=evalv3 go run cuelang.org/go/cmd/[email protected] export ./k8s/amour/list.cue

What did you expect to see?

Successful export.

What did you see instead?

❯ CUE_EXPERIMENT=evalv3 go run cuelang.org/go/cmd/[email protected] export ./k8s/amour/list.cue
panic: unexpected uninitialized node [recovered]
        panic: unexpected uninitialized node

goroutine 1 [running]:
cuelang.org/go/internal/core/adt.runTask.func1()
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/sched.go:692 +0x76
panic({0xca7f20?, 0xf2d960?})
        /usr/lib/go/src/runtime/panic.go:770 +0x132
cuelang.org/go/internal/core/adt.(*overlayContext).cloneNodeContext(0xc0c212a020?, 0x785e93273388?)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/overlay.go:197 +0x625
cuelang.org/go/internal/core/adt.(*overlayContext).cloneVertex(0xc01d39caa0, 0xc0c20efb80)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/overlay.go:186 +0x4f7
cuelang.org/go/internal/core/adt.(*overlayContext).cloneVertex(0xc01d39caa0, 0xc0c1f5d400)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/overlay.go:164 +0x2b5
cuelang.org/go/internal/core/adt.(*overlayContext).cloneRoot(0xc01d39caa0, 0xc0c1fb0690?)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/overlay.go:81 +0x25
cuelang.org/go/internal/core/adt.(*nodeContext).doDisjunct(0xc0c1f93208, {0xc0c1f38dc0, {0xf35200, 0xc003c9ee80}, {0x0, 0xc0c1fb0690, 0x0, 0x0, 0x0, 0x0, ...}}, ...)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/disjunct2.go:416 +0x26b
cuelang.org/go/internal/core/adt.(*nodeContext).crossProduct(0xc0c1f93208, {0xc01d39cd98?, 0x0?, 0xc01d39cea8?}, {0xc01d39ce58, 0x1, 0xf338e0?}, 0xc0c1d01d50, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/disjunct2.go:345 +0x3e5
cuelang.org/go/internal/core/adt.(*nodeContext).processDisjunctions(0xc0c1f93208)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/disjunct2.go:279 +0x176
cuelang.org/go/internal/core/adt.processDisjunctions(0xdcf6ba?, 0xc0c20efcc0, 0x73?)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/tasks.go:189 +0x1c
cuelang.org/go/internal/core/adt.runTask(0xc0c20efcc0, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/sched.go:712 +0x525
cuelang.org/go/internal/core/adt.(*scheduler).process(0xc0c1f933d8, 0x7eff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/sched.go:411 +0x23d
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1f5d400, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:159 +0x305
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0c1f8a008, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1f5ce60, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0c1f34608, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1f13360, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0c1f04008, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1edf900, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0c1ee3808, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1ede5a0, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0c1edb208, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1ede000, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0c1d67808, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0c1d5d180, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc0bfb2ec08, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0bfb20fa0, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc005523b80, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:283 +0xc27
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc004f1d208, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc004738aa0, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*nodeContext).completeAllArcs(0xc004f1c008, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:503 +0x22c
cuelang.org/go/internal/core/adt.(*Vertex).unify(0xc0047388c0, 0xc00032dc80, 0x7fff, 0x4)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/unify.go:226 +0x8e8
cuelang.org/go/internal/core/adt.(*OpContext).unify(0xc004f10da0?, 0xf33700?, 0x4f14b40?)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/eval.go:158 +0x386
cuelang.org/go/internal/core/adt.(*Vertex).Finalize(0x41383b?, 0xc00032dc80)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/internal/core/adt/composite.go:816 +0x59
cuelang.org/go/cue.newVertexRoot(0xc0002e5680, 0x0?, 0xc0047388c0)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cue/types.go:602 +0x25
cuelang.org/go/cue.newValueRoot(0x0?, 0xc003f65558?, {0xf3ac00?, 0xc0047388c0?})
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cue/types.go:611 +0x3e
cuelang.org/go/cue.(*Context).make(0xc0002e5680, 0xc0047388c0)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cue/context.go:253 +0x8c
cuelang.org/go/cue.(*Context).BuildInstances(0xc0002e5680, {0xc01d39f948, 0x1, 0xc000341360?})
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cue/context.go:148 +0x1e5
cuelang.org/go/cmd/cue/cmd.buildInstances(0xc0002f79e0, {0xc01d39f948, 0x1, 0x0?}, 0x1)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/cmd/common.go:710 +0x46
cuelang.org/go/cmd/cue/cmd.parseArgs(0xc0002f79e0, {0xc0002fb660, 0x1, 0x1}, 0xc0002e3090)
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/cmd/common.go:599 +0xc8e
cuelang.org/go/cmd/cue/cmd.runExport(0xc0002f79e0, {0xc0002fb660, 0x1, 0x1})
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/cmd/export.go:113 +0x8a
cuelang.org/go/cmd/cue/cmd.newExportCmd.mkRunE.func1(0xc0001bce00?, {0xc0002fb660, 0x1, 0x1})
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/cmd/root.go:126 +0x2ac
github.com/spf13/cobra.(*Command).execute(0xc0002f5508, {0xc0002fb620, 0x1, 0x1})
        /home/thomas/go/pkg/mod/github.com/spf13/[email protected]/command.go:985 +0xaca
github.com/spf13/cobra.(*Command).ExecuteC(0xc0002f4608)
        /home/thomas/go/pkg/mod/github.com/spf13/[email protected]/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        /home/thomas/go/pkg/mod/github.com/spf13/[email protected]/command.go:1041
cuelang.org/go/cmd/cue/cmd.(*Command).Run(0xc0002f79e0, {0x1531b40?, 0xcedcc0?})
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/cmd/root.go:391 +0x1f
cuelang.org/go/cmd/cue/cmd.Main()
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/cmd/root.go:299 +0x95
main.main()
        /home/thomas/go/pkg/mod/cuelang.org/[email protected]/cmd/cue/main.go:24 +0x13
exit status 2
@uhthomas uhthomas added NeedsInvestigation Triage Requires triage/attention labels Jul 26, 2024
@uhthomas
Copy link
Contributor Author

looks like ./k8s/amour/grafana is unhappy specifically.

@uhthomas
Copy link
Contributor Author

also

❯ CUE_EXPERIMENT=evalv3 go run cuelang.org/go/cmd/cue@master export ./amour/minecraft
#List.items: error in call to list.Concat: unexpected uninitialized node:
    ./amour/minecraft/list.cue:26:15
exit status 1

@uhthomas
Copy link
Contributor Author

uhthomas commented Jul 26, 2024

So it works if I comment these lines out:

https://github.com/uhthomas/automata/blob/8bad5f9e827a7161ccb7c8e2ee115efb796105c9/k8s/amour/minecraft/cf_atm9/stateful_set_list.cue#L142-L145

Unfortunately I tried making a minimal reproduction from this, but it works fine?

import appsv1 "k8s.io/api/apps/v1"

appsv1.#Deployment & {
	spec: template: spec: containers: [{
		name: "test"

		let probe = {exec: command: ["mc-health"]}

		livenessProbe:  probe
		readinessProbe: probe
		startupProbe: probe & {
			initialDelaySeconds: 240
			failureThreshold:    30
		}
	}]
}

@uhthomas
Copy link
Contributor Author

Okay, got it.

❯ CUE_EXPERIMENT=evalv3 go run cuelang.org/go/cmd/cue@master export ./a.cue
panic: unexpected uninitialized node [recovered]
        panic: unexpected uninitialized node [recovered]
        panic: unexpected uninitialized node
import (
	"list"

	appsv1 "k8s.io/api/apps/v1"
)

#Deployment: appsv1.#Deployment & {
	spec: template: spec: containers: [{
		name: "test"

		let probe = {exec: command: ["mc-health"]}

		livenessProbe:  probe
		readinessProbe: probe
		startupProbe: probe & {
			initialDelaySeconds: 240
			failureThreshold:    30
		}
	}]
}

deploymentList: list.Concat([[#Deployment]])

@uhthomas
Copy link
Contributor Author

I found #3331 whilst trying to make a really minimal reproduction.

@uhthomas
Copy link
Contributor Author

Funnily enough, the reproduction is also fixed if I change

let probe = {exec: command: ["mc-health"]}

to

let probe = v1.#Probe & {exec: command: ["mc-health"]}

@mvdan
Copy link
Member

mvdan commented Jul 30, 2024

I managed to reduce this further to under ten standalone lines:

exec cue export in.cue
cmp stdout export.stdout

env CUE_EXPERIMENT=evalv3
exec cue export in.cue
cmp stdout export.stdout

-- in.cue --
import "list"

#struct: {
	let empty = {}

	field: null | { n: int }
	field: empty & { n: 3 }
}

out: list.Concat([[#struct]])
-- export.stdout --
{
    "out": [
        {
            "field": {
                "n": 3
            }
        }
    ]
}

The test succeeds for the non-evalv3 part, but the evalv3 part panics as you show.

@mvdan mvdan added panic and removed Triage Requires triage/attention labels Jul 30, 2024
@cuematthew
Copy link
Contributor

I've spent some time refining this further. I suspect the panic is the consequence of things going off track a fair bit earlier on. I think there's something not quite right with definitions in v3, because:

exec cue export in.cue
cmp stdout export.stdout

env CUE_EXPERIMENT=evalv3
exec cue export in.cue
cmp stdout export.stdout

-- in.cue --
#struct: {
	field: { n: 3 } & g
	g: {}
}

out: #struct & {}
-- export.stdout --
{
    "out": {
        "field": {
            "n": 3
        },
        "g": {}
    }
}

This gives an error in v3 (not a panic), but in the old evaluator it works. However, if you change #struct into struct then it works in both. Similarly, @mvdan , in your simplification, changing the #struct into struct avoids the error (and panic).

@mpvl
Copy link
Member

mpvl commented Aug 5, 2024

A bit shorter reproducer yet:

#empty: {}
x: null | { n: 3 }
x: #empty & { n: 3 }
out: len(x)

@cuematthew : your case is interesting. g is within the definition is defined within a definition and itself closed. So one could argue the new evaluator is correct. However, since the reference to g is within the definition, the according to the spec the old evaluator is correct. But quite subtle.

@cuematthew
Copy link
Contributor

@mpvl Yes I see your point, but I thought that a definition was only closed when it is referenced. I think I would find it a little surprising to need to consider g: {} to be closed. Maybe my thinking is erroneous, but I definitely think it is the resulting value of the declaration of #struct that gets closed on use.

Also, in v3, export of:

#struct: {
	field: { n: 3 } & g
	g: {}
}

out: #struct
out2: out & {}

does not error, and gives:

{
    "out": {
        "field": {
            "n": 3
        },
        "g": {}
    },
    "out2": {
        "field": {
            "n": 3
        },
        "g": {}
    }
}

I don't know if you'd expect that or not.

@cueckoo cueckoo closed this as completed in cd689ef Aug 6, 2024
cueckoo pushed a commit that referenced this issue Aug 12, 2024
Issue #3325
Issue #3330
Issue #3331

Signed-off-by: Marcel van Lohuizen <[email protected]>
Change-Id: I16c3639fb016020ea58b75e7ecefa0fc1d1afe84
Reviewed-on: https://cue.gerrithub.io/c/cue-lang/cue/+/1199271
Reviewed-by: Matthew Sackman <[email protected]>
Reviewed-by: Daniel Martí <[email protected]>
Unity-Result: CUE porcuepine <[email protected]>
TryBot-Result: CUEcueckoo <[email protected]>
cueckoo pushed a commit that referenced this issue Aug 12, 2024
Before this change, any conjunct referred to was inserted
at the insertion point in full. Top-level conjuncts have
a tree representing the origin of their definition and
embedding. This caused the closedness rules to be repeated
when they should not, resulting in spurious "field not
allowed" errors.

The new algorithm splits a conjunct into branches with
roots that are shared with the insertion point removed.
This ensures that references that are in scope of a
definition do not get spuriously inserted.

Fixes #3330
Fixes #3331

Signed-off-by: Marcel van Lohuizen <[email protected]>
Change-Id: I7a573f9576d3c46374647045d8ec226a1cd543b3
Reviewed-on: https://review.gerrithub.io/c/cue-lang/cue/+/1199272
Reviewed-by: Matthew Sackman <[email protected]>
TryBot-Result: CUEcueckoo <[email protected]>
Reviewed-by: Daniel Martí <[email protected]>
Unity-Result: CUE porcuepine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants