Scoping in more things?

[cgwalters]

Hi, I'm the maintainer of http://github.com/coreos/rpm-ostree/ which you already know about; it's somewhat related to this project but...does way more things today, in different ways.

One big difference I'm curious for your thoughts around is what I call "reproducible split images" - a lot more in ostreedev/ostree-rs-ext#69 and especially I found the (linked from there) https://grahamc.com/blog/nix-and-layered-docker-images/ blog very inspirational. Today, this project just generates a single tarball - fine for micro services, but doesn't scale as well to larger images, which we are targeting for a variety of use cases (especially bootc containers where we may e.g. have linux-firmware in there; don't want to redownload that each time you need a glibc security update).

Second, I'm coming at this from the perspective of https://gitlab.com/fedora/bootc/tracker/-/issues/32 - and specifically what I strongly want is support for non-RPM content; the baseline here is to support stitching that content on after a full RPM install. (It obviously gets a lot more complex to have that override/replace RPM content, or participate in e.g. %posttrans if you want to inject non-RPM content into the initramfs as generated by kernel.posttrans for example).

Do you consider either of these potentially in scope?

[tofay]

1 could be in scope. I've read into the nix layering before, including that excellent blog, and previously considered how to do something similar in a closed-source predecessor of rpmoci. That tool (mudlark) is a constraining wrapper around a dockerfile and uses docker, which makes it difficult to have fine grained control of the layering on a package basis. But rpmoci wouldn't have the same issue.
A couple of years ago I had considered doing a graph approach similar to the blog for mudlark, but with batching - we have ~20 images with 100s of packages and I envisaged having n packages per layer depending on package size. I didn't get so far as prototyping there due to time constraints.

I've been looking a bit at the zstd:chunked work in podman etc - does that have the same advantage re duplicate file downloads as reproducible split images? (I guess some filesystems don't the deduplication)

On 2, happy to consider so long as it doesn't add a dependency on docker/podman for sourcing the files.
The lack of support had been an annoyance for internal users - there's been cases where people want to override a config file in /etc for software that doesn't support dropin configuration, and they have to create an RPM with an install script to add their config for them if the config file is owned by an existing package.

Thanks for the links, they are interesting! I'm particularly curious what replaces rpm-ostree compose image in the fedora ecosystem. (I did consider making a dnf plugin for container image generation instead of rpmoci, but the lack of lockfile/reproducibility steered me away from that rpm-software-management/dnf5#833.)

[cgwalters]

I've been looking a bit at the zstd:chunked work in podman etc - does that have the same advantage re duplicate file downloads as reproducible split images? (I guess some filesystems don't the deduplication)

containers/image#902 (comment)

Or TL;DC: Orthogonal and complementary

(I guess some filesystems don't the deduplication)

Not sure what you mean (are you referring to reflinks?) That isn't required for any of this.

On 2, happy to consider so long as it doesn't add a dependency on docker/podman for sourcing the files.

Yes, today I maintain https://github.com/containers/containers-image-proxy-rs/ which forks off skopeo for this stuff (more lightweight than docker/podman). There's also https://github.com/oras-project/rust-oci-client