DONOTMERGE: EXAMPLE: COSA without rhcos breaking patches

[ashcrow]

This is an example of master minus some commits which builds RHCOS. There is one issue that does pop up which is systemd-random-seed.service and systemd-journal-catalog-update.service fail to start, but work fine when restarted.

[ashcrow]

/hold

[cgwalters]

See coreos/ignition-dracut#79 for the units.

[jlebon]

Re. moving OSTree commit into build dir, I don't think it actually conflicts with RHCOS, but if it's deemed redundant because of the oscontainer, there's various options. See thread at #515 (comment).

[cgwalters]

Re. moving OSTree commit into build dir, I don't think it actually conflicts with RHCOS, but if it's deemed redundant because of the oscontainer, there's various options.

It is kind of redundant but...eh, on the other hand we have deep issues in "lifecycle binding" the cosa builds and the oscontainers, having the ostree commit always easily accessible with a build is useful.
Storage space is also pretty cheap, though we haven't started doing GC yet either...

[cgwalters]

I'm able to get something to build by just readding the inject-core-user bits. See https://github.com/cgwalters/coreos-assembler/tree/rhcos-4.2 - what do you think about replacing this PR with that?

[cgwalters]

Or hm, how did you test this PR? Did you go all the way to running a test pipeline with it, and it broke on the ostree-builddir stuff?

[ashcrow]

To test this I:

• rebuilt cosa
• overrode cosa on the commandline to use my local build
• Updated image.yaml to no longer require the inject-core-user
• Built and ran as normal within the cosa shell

[cgwalters]

Wait, hm, did you have a core user still?

[ashcrow]

I'm able to get something to build by just readding the inject-core-user bits. See https://github.com/cgwalters/coreos-assembler/tree/rhcos-4.2 - what do you think about replacing this PR with that?

I like it! I'm going to give it a shot next and see the difference. I like the idea of converting the Ignition config rather than trying to have two versions throughout 👍

[ashcrow]

Wait, hm, did you have a core user still?

Yup! core was still there.

Failed Units: 2
  systemd-journal-catalog-update.service
  systemd-random-seed.service
[core@coreos ~]$ 

[cgwalters]

Unit failures are fixed with #540

[ashcrow]

#540 merged this morning, I'm rebuilding this branch with that patch cherry-picked in.

[ashcrow]

@cgwalters the merged PR does indeed fix the issue 👍

[cgwalters]

OK, next step here I think is to debate: Do we create a rhcos-4.2 branch with the ostree-in-builddir changes reverted so the pipeline works unchanged, or do we go for adapting the pipeline to git master?

I vote for the latter - the pipeline changes should be straightforward.

[ashcrow]

I also vote for the later. I believe that, until master becomes the head for both RHCOS and FCOS our branches will continue to be true forks at worst, ❄️s at best.

/cc @miabbott @darkmuggle @yuqi-zhang @bgilbert @dustymabe @ajeddeloh

[ashcrow]

While discussion occurs I'm trying out cgwalters@6bc779f on top of this.

[jlebon]

I vote for the latter as well. Short of adding proper CI to this repo, having RHCOS also consume cosa straight from git master like FCOS is the best way to get fast feedback and subsequent fixes in. Any other strategy would just delay the same pain we're facing now.

[ashcrow]

I get

[coreos-assembler]$ cosa run
info: Missing CAP_SYS_ADMIN; using virt
Unsupported version 2.2.0

which makes sense since in my branch I reverted the Ignition 3.x stuff. I hacked around it and it worked with exception to there being no home directory for the core user. I think a combination of @cgwalters's Ignition 2 to 3 patch plus a few follow on patches to make a few items configurable/optional will make the cosa project workable for both RHCOS and FCOS.

[cgwalters]

I am confused as to where the core user is coming from...must be Anaconda but where are we adding it?

The lack of /home/core is because the cleanup is deleting it.

I think let's try to backport the bits to add core in Ignition to the spec2x branch?

[darkmuggle]

LGTM, I don't see any thing obvious. As long as it been tested 👍

[ashcrow]

LGTM, I don't see any thing obvious. As long as it been tested +1

@darkmuggle Which path are you 👍 for?

[ashcrow]

Testing #537 in it's current state rebased on master now (IE: without inject-core-user).

[cgwalters]

I am confused as to where the core user is coming from...must be Anaconda but where are we adding it?

Nevermind, i was looking at the wrong image.

[ashcrow]

Testing #537 in it's current state rebased on master now (IE: without inject-core-user).

The above built RHCOS (as long as I removed inject-core-user and let Ignition make the user in my config). I believe once #537 lands we should be at (or VERY near) to a point we can start trying out master for building.

[dustymabe]

I vote for the latter - the pipeline changes should be straightforward.

👍

[ashcrow]

Closing in favor of #537 rebased on master.