Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge mantle into coreos-assembler #1152

Merged
merged 2,944 commits into from
Feb 27, 2020
Merged

Merge mantle into coreos-assembler #1152

merged 2,944 commits into from
Feb 27, 2020

Conversation

cgwalters
Copy link
Member

Basically it doesn't make sense to separate building, testing,
and uploads. There are too many entangled problem domains,
among them:

  • How to run qemu
  • Parsing build schemas
  • Uploading to one AWS region, running tests, then replicating

etc.

Closes: #163

Stephen Lowrie and others added 30 commits October 4, 2019 14:15
podman: Only run podman network tests on fcos, use --net=none some
A minor fix for non x86 platforms to consider qemu-unpriv when getting the architecture in order
to retrieve the correct kolet binary

Closes: coreos#1074
Check for qemu-unpriv platform when converting board to arch
When using `-k` on `kola spawn`, it would seed the userdata with a v2
Ignition config. This of course doesn't work on FCOS. Detect the version
instead and seed according to that.
kola/spawn: fix using v2 for FCOS when using -k
When getting the qemu port, the list of connections in /proc/net/tcp is checked and the localhost address
is compares against. Ensure that the correct value is used for big and little endian arches
To ensure that we don't leak the `qemu` process by accident, just always
set the `PDEATHSIG` attribute on it before spawning. That way, even if
`kola` is abruptly killed, `qemu` will also be killed.

The only exception is `kola spawn --remove=false`, where do *do* want to
leak the `qemu` process. Pass that down as a flag.

This also essentially enables support for the `--qemu-options` flag on
the `unprivqemu` platform.
Allow passing an fd to `kola spawn` which scripts can use to easily
retrieve information about the spawned hosts. I'm planning to make use
of this in rpm-ostree's CI for example.
This switch will basically tell `kola spawn` that we're not interested
in a shell, but we want it to hang around. In combination with
`--json-info-fd`, this should allow scripts to leverage the CLI for
easily standing up temporary VMs.
Check endianness when getting the qemu port
gcp: Add UEFI_COMPATIBLE and SECURE_BOOT
platform/unprivqemu: Drop restrict=yes
I plan to use this for cosa to make it a bit easier to log in
manually to instances created by `cosa kola`.  It will be set
to a predictable path rather than a dir in `/tmp` that one may
have to hunt down.
Adds the aliyun platform API; initially only supports a select set of
ECS & OSS calls relating to importing & copying images.
I thought this passed at one time, but it seems to just hang
here.  I tried to debug it a little, but the expected flow here
is opaque to me, and it just doesn't seem worth it.

Bigger picture anyways, rather than inventing our own tests for most
upstream projects, we should run *their* tests that have
already been run in *their* CI too.  This is what I plan to
do for ostree/rpm-ostree at least with FCOS.
Now that CL is branched, let's improve ergonomics here and
default to `fcos` for distro.  This has bit me a few times already
where I ran `kola` directly and forgot the `-b` argument.
kola: Change distro default to fcos, drop cl
kola/podman: Bump memory tests to 128m for runc
Adds a new aliyun command and 3 new subcommands underneath it all
relating to image creation, copying, and deletion.
Adds a test that verifies pods continue when the crio service
is restarted.

See: https://access.redhat.com/solutions/3825511

Signed-off-by: Steve Milner <[email protected]>
We just lifted the `restrict=yes` from the qemu-unpriv path
which fixed the podman tests, but didn't remove it from the
`PlatformsNoInternet` list.  This meant that we were still
skipping tests like `crio.base` that now *should* run with
qemu-unpriv.

(In the PR we discussed reworking all of this so that
 "no networking" is an opt-in thing for qemu-unpriv,
 which I'll do once we've drained the queue of other mantle PRs)
kola: Note qemu-unpriv now has networking
These only apply to `cl` which is on a branch now.
@cgwalters cgwalters force-pushed the merge-mantle branch 2 times, most recently from b7b7ae5 to 26956cc Compare February 26, 2020 18:14
@cgwalters cgwalters changed the title WIP: Merge mantle into coreos-assembler Merge mantle into coreos-assembler Feb 26, 2020
Regression from 7383128 which
broke `coreos.ignition.mount.disks` because we were overriding
the provided serial.
Prashanth684 and others added 3 commits February 26, 2020 16:35
Changed from by-partlabel to by-partuuid by assigning a random uuid. On s390x, the by-partlabel string
is incorrect because GPT partitioning uses little endian by default and s390x is big endian. Refer
to: coreos/ignition-dracut#100.
qemu: Only set serial= if not already provided
kola: Use uuid instead of partlabel for ignition.mount disk test
@cgwalters
Copy link
Member Author

coreos/mantle#1195

cgwalters and others added 7 commits February 27, 2020 18:37
Regression from d4b252b
We need to keep parsing the cosa build, just not actually use it
to sync the target options in order to match how the upgrade
test works.
Basically it doesn't make sense to separate building, testing,
and uploads. There are too many entangled problem domains,
among them:

- How to run qemu
- Parsing build schemas
- Uploading to one AWS region, running tests, then replicating

etc.  This prepares the mantle project for merge into coreos-assembler
by moving it to a subdirectory.
We are going to merge mantle into the project.
Basically it doesn't make sense to separate building, testing,
and uploads. There are too many entangled problem domains,
among them:

    How to run qemu
    Parsing build schemas
    Uploading to one AWS region, running tests, then replicating

etc.

This merges the https://github.com/coreos/mantle project into
coreos-assembler.

Closes: coreos#163
Fixup for removing mantle git submodule.
@cgwalters
Copy link
Member Author

OK, this should be good to go now! I had to enable merge commits on this repository rather than "rebase and merge". And I'm not sure Prow can merge it because of that, I think we have Prow set up to rebase. So this one will have to be someone clicking the button manually.

@darkmuggle
Copy link
Contributor

/approve

@cgwalters WELL DONE!

@jlebon
Copy link
Member

jlebon commented Feb 27, 2020

+1,020,323... wonderful world of golang vendoring, here we come!
/lgtm

@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cgwalters, darkmuggle, jlebon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [cgwalters,darkmuggle,jlebon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jlebon
Copy link
Member

jlebon commented Feb 27, 2020

Note you will likely have to rm -rf mantle manually before doing git pull.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cosa/mantle integration