server side package layering/derivation

[cgwalters]

Now that we support client-side composes, there's no reason not to take the next step and do server-side composes.

I would sketch out this architecture:

• Add from to treefile JSON; this could specify either a branch name or a SHA256
• Teach compose tree to check out the dependencies and layer on top, generating a new commit
• Inject the from commit as an ostree.from metadata key, that gives at least a SHA256 and optionally a branch name
• Teach checkout to read this and do a union checkout

[cgwalters]

A downside of this model (shared with docker) is that it duplicates all "cache files" like ldconfig, the RPM database etc. I suspect it's not going to matter too much. We could consider going for a "recommit" model with informational layering, i.e. we don't actually pull down the base commit.

[jberkus]

Well, for users who are concerned about efficiency, they can build a real custom tree. Server-side layering is just a simpler on-ramp to having your own trees, right?

[cgwalters]

There's two aspects to this - technological and organizational. For a case where one organization produces an OS that is consumed by multiple others (e.g. Fedora/CentOS), I think from a support perspective the layering model rather than "re-aggregation" is a lot better - one can see from the output of atomic host status what the base version is, etc.

There's also a lot of things around configuration details that start to matter - for example right now we carry some bits to tweak systemd to fix its interaction with rpm-ostree. Ideally, not everyone has to carry this stuff around. (Yes, ideally we fix the upstream RPM packages. I do push hard on that but...)

[cgwalters]

OK this now intersects with ostreedev/ostree-rs-ext#12