references inside and into nested struct-templates ("graph-templates")

[georgfaust]

EDIT: I understood CUE some more and rephrased the question in the answer below.

I have nested structs where nodes may reference any other node in that struct (the level of nesting is arbitrary and unknown)
Note: with reference I do not (only) mean a CUE-reference, but an ID I can use to find a node in the exported json later.

#item: {data: *42 | int, ...}
#foo: {
	items: {
		i1: {id: uuid.FromInt(1)} & #item
		i2: {id: uuid.FromInt(1)} & #item
	}
	refs: {ref_baz: items.i2.id, ref_foobar: items.i1.id}
	cross_refs: {...}
}

Here for example ref_baz references item 2.
This is easy, when there is only one foo. But what if I want to use #foo as a template?

foos: {
	foo_1: #foo 
	foo_2: #foo
}

Now the IDs are obviously not unique anymore.

also I want to grab into foo from outside, or from one foo to another.

foos: {
	foo_1: #foo & {cross_refs: ref_foo: foo_2.items.i1.id}
	foo_2: #foo
	deep_refs: ref_bar: foo_1.items.i1.id
}

• All I need is a way to generate a random unique uuid, how is that done?
• maybe I'm completely wrong and this is better done in another way?

CUE is amazing! Thank you for it!

[SandyWalsh]

I think we're struggling with a similar problem #1723

[georgfaust]

I found this discussion about uuidgen() here: #192
I understand now, that CUE is a hermetic language and my naive approach may not work.

But the generated ID here is only needed to model the reference (in JSON). If you look at it in a more abstract way, the references itself do not break the hermetic sealing.

So my question should be: is there a way to model such "graph-templates"?

#template: n1-->n2

// use this template
user_of_template: {
  u1: #template // n1a-->n2a
  u2: #template // n1b-->n2b
  // can also create an edge reaching into the template
  deepref: n-->u1.n2 // n-->n2a
  // and 
  crossref: u1.n1 --> u2.n2 // n1a --> n2b
}

// this user_of_template could be used as a template as well (keeping its internal refs)

As seen in the OP you can already model that in CUE, but the reference information is lost, because the target of the ref is just copied.
So if there would be a way, to keep a "pointer" or sth to the target until export, only then the reference could be encoded as a random number.

The config files I am using (Building models / BIM / ICF) heavily use such internal references and are also very repetitive. I think I could reduce the size and complexity by magnitudes with CUE. This could be really a game changer. I'd even accept a non hermetic solution.

[myitcv]

Can you not leverage the uniqueness you already have in your structure?

i.e. your foos struct defines fields for each #foo instance, and within each #foo, the items each have a unique key. So then identify each #item by that unique path:

https://cuelang.org/play/?id=v24ulXI8rrE#cue@export@cue

As you say, this "uniqueness" only needs to exist within the model itself, hence you don't need true UUIDs.

If you wanted to define the references to actual #item values and have the export (i.e. manifestation) of the configuration reduce that to the id then you could do something like:

https://cuelang.org/play/?id=2TE1qNGGsST#cue@export@cue

Note that whilst this syntax looks a bit clunky today, it is the basis of a proposal for a "function" syntax. See the first heading "Functions" in #943.

[georgfaust]

Have you seen this pattern a lot in real-world examples?

I think I'm in another "part of the real-world" as other users of CUE. I assume that most are using CUE to configure Kubernetes or something similar.

I'm using CUE to describe the real real-world, that is: buildings.
That starts with the individual stones, pipes, cables, ..., the building is made of and ends with logical connections of light switches and lights (for example). These configurations can be very large and very repetitive. Imagine a hotel with 1000 identical rooms. While the hardware (stones, ...) is mostly hierarchical, the logical part is a graph, eg:

example of a graph that needs to be repeatable:

floor_a:
  room_aa:
    lights: 
      a: {}
      b: {}
    switches:
      x: {ref: a}
      y: {ref, [a,b]}
  cabinet:
      fuses:
        f27: room_aa.lights.a
        ...

I do not know how often these kind of graphs appear in other configurations.
path() may be too magic (funny you say that, you know, that CUE is pure magic for anyone not familiar with math?) but I think it would be great to somehow hold on to a reference before exporting. This should be useful somewhere else.

[myitcv]

Just to confirm my understanding: CUE is working from the perspective of representing these relationships. What you're looking to solve here is the problem of how such references are represented on export, correct?

[georgfaust]

Kind of. As long as I look at the .cue , eg:

foo: bar: {data: 42}
ref: foo.bar

the reference here is semantically correct.

But after cue eval the reference will be replaced by the referenced value.
This is not what I want. The node is not just qualified by data but also by the location in the hierarchy.

here cue will eval to sth with ref1==ref2, which is not what I want.

foo: bar: {data: 42}
foo: baz: {data: 42}
ref1: foo.bar
ref2: foo.baz

Therefore the effort to somehow reflect the hierarchy in a field (id).
Which is OK, but, as I said, it would be very nice, to have the ability to somehow not resolve a reference, but to keep it as an identifier of the referenced struct.

An example in python to clarify this (hopefully)

>>> a = {1: 1} 
>>> b = {1: 1}
>>> a == b
True
>>> id(a) == id(b)
False
>>> id(a)
140695700039488
>>> id(b)
140695700039232

the path() proposal would be one way to do that. But it could really just be an internal id like what python returns for id().
I have near zero understanding of the inner workings of CUE, so no idea if that is feasible.

I just think, that it would be useful to have structs with some inner relations, that can be moved around.

[myitcv]

Please can you confirm, how will use use this CUE? Export to JSON? Consume in a Go program? Other?

[georgfaust]

I'm exporting to JSON (and process in Elixir).