Please usrmerge Bullseye and above

[geofft]

For consistency, it seems like it would be a good idea to make it so that debian:testing (and soon debian:11) is usrmerge'd - debian-installer defaults to usrmerge and doesn't give you an option for an unmerged usr. Right now it looks like debian:testing is unmerged.

[geofft]

Linking to #60 - I think it makes sense that it was disabled for Buster (and should remain disabled), but I think it should be enabled for Bullseye if I understand the situation right.

[tianon]

For installed systems, I totally agree -- however, the place this gets complicated for the Docker images is that they're very, very often used as both "installed system" and "buildd", and I do not believe it's safe in Bullseye to build packages for general consumption on a usrmerge'd chroot, right?
(https://lists.debian.org/debian-devel-announce/2019/03/msg00001.html - Issues from "merged `/usr`" heading)

That being said, I think the conclusion in https://bugs.debian.org/978636#178 is that bookworm+ is totally safe to go full-usrmerge? 🙏 😅

[geofft]

Hmm, that's a reasonably compelling argument, for bullseye at least (and I agree that on bookworm, if merged usr is mandatory, then these problems don't arise).

My motivation is that that the interactive Docker image doesn't act like an interactive installed system, but I admit that's less compelling of an argument. Thanks for thinking about it though!

[geofft]

I'm not sure why I didn't think of this earlier, but apt update; apt install usrmerge works. So that's another argument in favor of keeping the image unmerged; it's easy to run that if you want to see what the merged layout looks like, but there's no conversion in the other direction (unless you count dpkg-fsys-usrunmess, I guess).

I think a working Dockerfile to poke at a usrmerged system is

FROM debian:testing # or 11, or unstable, or whatever
RUN echo debconf usrmerge/autoconvert select true | debconf-set-selections && apt-get update && apt-get -y install usrmerge

[zeha]

From my experience it's easy enough to build packages on a non-usr-merged system and still produce packages that will not work on them. Its certainly easier (and required by some tools) to build on non-usr-merged to support non-usr-merged, but its not a definite solution. If Docker images are supposed to be usable as build platforms ("buildd"), then they have to stay non-merged for now, even though this will have problems for non-build usages.

[tianon]

https://lists.debian.org/debian-devel/2021/08/msg00188.html is a very interesting detailed description of the discussion at hand.

IMO, the reasons stated there to wait for bookworm+1 don't apply quite as strongly to the Docker images since things built in them are more likely to be ancillary packages and thus less likely to break, but it is certainly an interesting conundrum and a heavily gray area.

Perhaps we could somehow bribe @smcv to opine on this specific facet of the problem (tldr; "Docker images: to merged-/usr or not to merged-/usr" or rather, to merged-/usr bookworm or wait until bookworm+1)? 🙏 😇

(I won't personally be offended at all if you simply hit unsubscribe instead, @smcv, but if you've got opinions you're willing to share they're certainly appreciated because you clearly have a very good grasp of the extent of the problem we're looking at! 😅)

[smcv]

As a first go at this, I think wait until we have a better picture of how many packages are still misbuilt with merged-/usr, and err on the side of unmerged because apt install usrmerge is easy to do.

[tianon]

Makes sense; your input (both here and the larger project) is very much appreciated! ❤️

[bobrik]

Doesn't look like usrmerge works on overlayfs:

• https://www.mail-archive.com/[email protected]/msg1848199.html

[tianon]

More "canonical" link for the above discussion: https://bugs.debian.org/1008202#10

The discussion there is that the usrmerge conversion doesn't work on overlayfs (but a pre-usrmerged-image itself does work just fine -- they're just totally normal vanilla symlinks after all).

[smcv]

We talked about this more recently and my suggestion is now this: https://lists.debian.org/debian-ctte/2022/07/msg00034.html

[tianon]

As always, really appreciate you sharing your thoughts and perspective! 🙇

I'll reply here since it doesn't seem like anyone in the CTTE objects or has input on your proposed plan so this feels like a more appropriate place to continue that discussion. 👍

I think your plan sounds great -- the only place I'm going to have a minor deviation (that shouldn't affect the end result) is that I plan to duplicate the logic for when/where to apply --merged-usr to avoid a versioned dependency on debootstrap (and so we can more easily apply a bit more nuance such as leaving bookworm unmerged for now). I plan to have a PR up ASAP so this can get into the rebuild I'll likely do next week. 👀

Edit: PR up at debuerreotype/debuerreotype#137

[tianon]

The (hopefully-final) PR on this subject is up at debuerreotype/debuerreotype#143 now 👀

(I also included some extra context over there about why I think our next round of image builds is the time to finally apply this to bookworm 👍)

[MichaIng]

Is there a workaround for applying usrmerge on a Debian Bullseye Docker container?

We just face the issue on MariaDB Salsa CI pipelines that testing distro upgrades Bullseye => Bookworm does not work. This is since usrmerge is pulled as dependency on Bookworm (usr-is-merged cannot be installed as long as the system is not actually usr-merged), but fails/denies to apply/install so that the whole distro-upgrade fails.

The ability for Debian package maintainers to test the upcoming distro upgrade would be quite helpful, and the official Debian Docker containers are a popular platform for CI/CD pipelines. Probably a set of dedicated usr-merged Bullseye containers could be released?

---

EDIT: Based on postinst script of usrmerge package, the following should work:

echo 'this system will not be supported in the future' > /etc/unsupported-skip-usrmerge-conversion