Authoring experience of injecting values into a hierarchy of values

[davidreuss]

has anyone worked on such a problem? Or does anyone have ideas how to go about this with CUE?

This is inspired by a similar theme brought up by Sam Boyer on slack in a thread.

Are there any examples on how to handle a hierarchy of values, where you essentially have a huge nested hash, and you in certain cases need to override the default, either per environment/zone/location … or similar.

I’m particularly interested in the “authoring” experience — and understanding what kind of values are allowed in a given file/location. The integration between schemas, and understandability of the effect of changing a piece of configuration.

The canonical example is maintaining a large configuration, where you might need to override many values (think resource / environment variable changes across a large variety of kubernetes deployments/objects for instance)

Thanks in advance.

[myitcv]

Pasting the Slack thread here for posterity (copy-pasted via StackEdit)

---

sam boyer 6 May at 19:40

Hi folks - a couple weeks ago, i announced that we’re putting a CUE-based schema system into the core of Grafana. Downstream of that, we’ve got a group working on a successor to the jsonnet-based prometheus monitoring mixins, which i intend to also be rooted in CUE. We’re still bootstrapping (and likely gonna pick a new name), but this seems likely to be an interesting space if you’re interested in the observability problem space, or its intersection with gitops.…and, maybe for purely CUE reasons, too: I just sent around this response i wrote to a commonly cited concern, and one i once had myself, about how CUE’s monotonic march to concreteness might collide with the need to just fix a damn thing in an emergency.

1

5 replies

---

myitcv 13 days ago

That link "CUE non-overridability, config trees" is an excellent read, thank you.

Zellyn Hunter 13 days ago

Good discussion, although I’d note that CSS is not the best example, because you can always override something in a panic by using more specific selectors. Of course, if you keep doing that without cleanup, you’re going to hate life

sam boyer 12 days ago

CSS is not the best example, because you can always override something in a panic by using more specific selectors.

It actually is the example i want - this is pretty much the property of CSS that i was actually trying to point to (albeit a possibly painful emergent property).I definitely agree that it’s possible to be sloppy af with this, as you describe, and get yourself into a nasty place. i don’t think it’s literally true that you can always add more selectors, though - at some point, you i think you’d run out of properties on the DOM node you’re trying to target that differentiates it from other nodes, and would have to add something to the DOM itself to make targeting possible.I think this may actually matter for the case our group is thinking about - if you’re trying to target just one instance of thing x in production for a fast fix, but there’s nothing to hang a unique selector on, then maybe the system is giving you feedback that you haven’t adequately represented the necessary dimensions of your prod infra?idk. Either way, the things people do during incident response are almost necessarily smashy and gross, but it’s definitely the wrong tradeoff to pick a system design that prevents such action out of concern for long-term, slow-burn harms.Now, people would also almost assuredly use those same complexity-accreting mechanisms when it’s not an emergency. But, given CUE’s amenability to analysis and introspection, it’s not clear to me that those effects are anything worse than the sloppy choices they’d make in any other system. (edited)

David Reuss 8 days ago

this thread, the linked blog post and the references talked about succinctly expresses my own personal views on the shortcomings in the configuration mangement systems of yesteryear …

i saw cue as a potential solution to the problem early on, and after having been flailing with other tools/languages and with cues api/language stability in sight i’m now ready to give cue a serious second look/attempt at easing some of the pain associated with managing our configuration — which are … a hot mess of a tirefire to put it mildly

i’ll be following this discussion closely …

David Reuss 8 days ago

the long term maintainability aspect has been the achilles heel in every system i’ve worked with — and you end up with walls and walls of text files that nobody can actually validate or reason about anything as it relates to the actual system

the hieara points are spot on

1

[myitcv]

FWIW, in my limited experience with these things, as part of Play with Go we use a tool called preguide. preguide is used to "compile" documentation, documentation that is generally based on scripts, a number of steps you follow as part of the guide.

The schema for a #Guide looks roughly like this, with each step keyed by a name:

#Guide: {
	#Step: {
		// ...
	}

	Steps: [name=_#StepName]: #Step & {
		Name: name
	}

        // ...
}

We will soon support multiple translations of a guide, and indeed multiple scenarios (e.g. one scenario for Go 1.15, another for Go 1.16 etc, because the output might be subtly different between the too). Therefore, we might need to capture variations on a per-step basis, either at the translation language level, and/or the scenario. Hence we will likely end up with a schema that looks like this (note this assumes some features described in #943 and other places):

	#Step: {
		// ...
	}

	#Scenario: {
		Name:  string
		Image: string
	}

	Scenarios: [not(_#Language)]: #Scenario & {
		Name: name
	}

	_#ScenarioName: or([ for name, _ in Scenarios {name}])

	Languages: [...#Language]
	_#Language: or(Languages)

	_#StepName: not(_#Language) & not(_#ScenarioName) & string

	// Now we know that _#Language, _#ScenarioName and _#StepName
	// are disjoint sets
	Steps: [name=_#StepName]: #Step & {
		Name: name
	}
	Steps: [_#Language]: [name=_#StepName]: #Step & {
		Name: name
	}
	Steps: [_#Language]: [_#ScenarioName]: [name=_#StepName]: #Step & {
		Name: name
	}
	Steps: [_#ScenarioName]: [name=_#StepName]: #Step & {
		Name: name
	}
	Steps: [_#ScenarioName]: [_#Language]: [name=_#StepName]: #Step & {
		Name: name
	}

Note: we are not in this position today, so this is somewhat blue-skies thinking.

What this would allow, however, is for the guide author to declare where and how the defaulting happens. i.e. the guide author could define that if a specific scenario + language step is not explicitly defined it should default from either a language-specific or scenario-specific step, with a further fallback to the non-specific step.

Perhaps this provides some insight?

[cueckoo]

This discussion has been migrated to cue-lang/cue#978.

For more details about CUE's migration to a new home, please see cue-lang/cue#1078.