Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compose: allow composing --unified-core into an archive repo #1490

Closed
dustymabe opened this issue Aug 3, 2018 · 4 comments
Closed

compose: allow composing --unified-core into an archive repo #1490

dustymabe opened this issue Aug 3, 2018 · 4 comments

Comments

@dustymabe
Copy link
Member

AIUI today for unified core composing you need to compose into a bare-user repo and then pull-local into an archive repo before clients can pull from the remote. So:

  • mkdir bare; ostree --repo=./bare/ init --mode bare-user
  • mkdir archive; ostree --repo=./archive/ init --mode archive
  • rpm-ostree compose tree --ex-unified-core --repo=./bare host.json
  • ostree pull-local --repo=./archive ./bare/ refname
  • client: rpm-ostree upgrade

It would be nice if we could actually compose a unified core in an archive repo transparently. i.e. store the bare-user repo inside the archive repo in a cache directory or something so that we only have to do this:

  • mkdir archive; ostree --repo=./archive/ init --mode archive
  • rpm-ostree compose tree --ex-unified-core --repo=./bare host.json
  • client: rpm-ostree upgrade

The only potential problem I see here is the embedded bare-user repo being neglected and possibly never getting garbage collected at all.

@cgwalters
Copy link
Member

I think the simplest is to require --cachedir with this. If you don't do that today, most of the benefit of --unified-core is lost anyways since the packages aren't cached.

@cgwalters
Copy link
Member

In the big picture, while we won't break anything related to rpm-ostree compose tree here, the new focus of development is https://github.com/coreos/coreos-assembler/

And that has an opinionated model for this out of the box; it makes two repos and handles pruning them both, etc.

@dustymabe
Copy link
Member Author

And that has an opinionated model for this out of the box; it makes two repos and handles pruning them both, etc.

my $.02

coreos-assembler handles this for us, but my vote would be to support this in rpm-ostree itself so anyone not using coreos-assembler would get the benefits.

@jlebon
Copy link
Member

jlebon commented Nov 5, 2018

Yeah, this is fixed by #1657.

The only thing I'd add is that convenience is just one factor. By letting rpm-ostree take care of the bare-user dance, we're guaranteed that we'll always get hardlinks and devino caching during the compose, which previously was up to the user to set up right.

@dustymabe dustymabe reopened this Nov 5, 2018
jlebon added a commit to jlebon/rpm-ostree that referenced this issue Nov 5, 2018
Previously, we were limiting the target repo in unified mode to be a
bare-user repo located on the same filesystem (see message of previous
commit). This patch lifts this restriction by making a distinction
between the *build repo* and the *final* target repo.

To do this, we create a bare-user repo located near the pkgcache to
take advantage of hardlinks and devino caching at commit time. And only
after committing do we essentially `pull-local` into the final target
repo. This of course allows us to avoid potentially pulling across the
two filesystems file objects that are already present in the target
repo.

This will be used by coreos-assembler:
coreos/coreos-assembler#190

Closes: coreos#1490
rh-atomic-bot pushed a commit that referenced this issue Nov 5, 2018
Previously, we were limiting the target repo in unified mode to be a
bare-user repo located on the same filesystem (see message of previous
commit). This patch lifts this restriction by making a distinction
between the *build repo* and the *final* target repo.

To do this, we create a bare-user repo located near the pkgcache to
take advantage of hardlinks and devino caching at commit time. And only
after committing do we essentially `pull-local` into the final target
repo. This of course allows us to avoid potentially pulling across the
two filesystems file objects that are already present in the target
repo.

This will be used by coreos-assembler:
coreos/coreos-assembler#190

Closes: #1490

Closes: #1657
Approved by: cgwalters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants