Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add ostree-container upload support #359

Closed
cgwalters opened this issue Jul 22, 2021 · 8 comments
Closed

add ostree-container upload support #359

cgwalters opened this issue Jul 22, 2021 · 8 comments
Assignees
@cgwalters
Labels
jira For syncing to JIRA

Comments

@cgwalters
Copy link
Member

Part of coreos/fedora-coreos-tracker#812

Specifically this should build on coreos/fedora-coreos-config#1097

  • Detect if ostree-format: oci
  • Upload the resulting container image to a configurable place (config where?) to e.g. quay.io/coreos/fedora-coreos:rawhide

Concerns:

  • do we have quay creds set up here?
  • How do we handle synchronization? Do we e.g. upload the container image after the s3 sync?
@cgwalters cgwalters added the jira For syncing to JIRA label Jul 22, 2021
@cgwalters cgwalters self-assigned this Jul 22, 2021
@jlebon
Copy link
Member

jlebon commented Jul 23, 2021 

* Upload the resulting container image to a configurable place (config where?) to e.g. `quay.io/coreos/fedora-coreos:rawhide`

Meh, we can probably start with hardcoding it in the Groovy for now?

Concerns:

* do we have quay creds set up here?

Yeah, we don't have any yet. We'll have to set it up.

* How do we handle synchronization?  Do we e.g. upload the container image _after_ the s3 sync?

I think conceptually this should be part of the release job. I guess right after/in parallel with the OSTree import request (that code only runs for production builds of course, but the image push would just always run since there's no compose/prod distinction there). For mechanical streams like rawhide, the release job is automatically triggered by the build pipeline.

@cgwalters
Copy link
Member Author

I don't think I can hack on this due to https://pagure.io/centos-infra/issue/386 - anyone who has access who can take this issue?

@jlebon
Copy link
Member

jlebon commented Jul 23, 2021

I pinged a few folks there. They're pretty responsive usually so let's give it a bit more time?

cgwalters added a commit to cgwalters/os that referenced this issue Jul 26, 2021
@cgwalters
image: Switch to ostree-format: oci
8e0a9d2 
This is the RHCOS version of coreos/fedora-coreos-config#1097

I was hoping to land this in FCOS first but I'm kind of blocked
on that in coreos/fedora-coreos-pipeline#359, so
let's settle for parallel.

NOTE!  This **does not change** the effect of `cosa upload-oscontainer` etc.
Some discussion on that here:
ostreedev/ostree-rs-ext#23
and I will file more issues related to that.

However, what this *will* do is allow us to push this *additional* image
arounnd and make `podman run registry.ci.openshift.org/rhcos/rhcos:4.8` e.g.
work.  In other words to start this will just be something *we* use
to quickly inspect rhcos-as-container, not something we actually ship.
(OK that's kind of a lie, it will end up in a place like e.g.
 http://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/4.9.0-0.nightly-2021-07-20-014024/
 instead of the existing `ostree.tar` so in theory other people outside
 of our group could run it too, but 🤫)
@cgwalters cgwalters mentioned this issue Jul 26, 2021
Merged
@cgwalters
Copy link
Member Author

OK, I have cluster/jenkins access now.

Does anyone know who owns https://quay.io/organization/coreos ? What do we need to create quay.io/coreos/fedora-coreos ?

@jlebon
Copy link
Member

jlebon commented Aug 19, 2021

@bgilbert may have the necessary permissions, or otherwise definitely know who to contact (likely James).

cgwalters added a commit to cgwalters/fedora-coreos-pipeline that referenced this issue Aug 24, 2021
@cgwalters
release: Copy oscontainer to quay.io
5a91197 
This is a bit hacky but should work.  My initial goal here
is just automated uploads of our builds, so that we can
start getting a better feel for managing "ostree-in-container"
releases.

We should sync to `quay.io/coreos` but that needs someone to set that up.

Also, I'd like to make the destination configurable in the same
way as the S3 bucket, proposal PR in coreos/fedora-coreos-config#1175

Closes: coreos#359
cgwalters added a commit to cgwalters/fedora-coreos-pipeline that referenced this issue Aug 24, 2021
@cgwalters
release: Copy oscontainer to quay.io
eebcf7b 
This is a bit hacky but should work.  My initial goal here
is just automated uploads of our builds, so that we can
start getting a better feel for managing "ostree-in-container"
releases.

We should sync to `quay.io/coreos` but that needs someone to set that up.

Also, I'd like to make the destination configurable in the same
way as the S3 bucket, proposal PR in coreos/fedora-coreos-config#1175

Closes: coreos#359
@cgwalters cgwalters mentioned this issue Aug 24, 2021
Closed
@coverprice
Copy link

I have admin access to the org. Connect to VPN and file a ticket with my team and I'll sort it out. I guess I'll need everyone's Quay logins, and what level of access they should have.

cgwalters added a commit to cgwalters/os that referenced this issue Aug 30, 2021
@cgwalters
image: Switch to ostree-format: oci
7a4d1bc 
This is the RHCOS version of coreos/fedora-coreos-config#1097

I was hoping to land this in FCOS first but I'm kind of blocked
on that in coreos/fedora-coreos-pipeline#359, so
let's settle for parallel.

NOTE!  This **does not change** the effect of `cosa upload-oscontainer` etc.
Some discussion on that here:
ostreedev/ostree-rs-ext#23
and I will file more issues related to that.

However, what this *will* do is allow us to push this *additional* image
arounnd and make `podman run registry.ci.openshift.org/rhcos/rhcos:4.8` e.g.
work.  In other words to start this will just be something *we* use
to quickly inspect rhcos-as-container, not something we actually ship.
(OK that's kind of a lie, it will end up in a place like e.g.
 http://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/4.9.0-0.nightly-2021-07-20-014024/
 instead of the existing `ostree.tar` so in theory other people outside
 of our group could run it too, but 🤫)
@cgwalters cgwalters mentioned this issue Aug 30, 2021
Merged
cgwalters added a commit to cgwalters/os that referenced this issue Sep 8, 2021
@cgwalters
image: Switch to ostree-format: oci
4d62fea 
This is the RHCOS version of coreos/fedora-coreos-config#1097

I was hoping to land this in FCOS first but I'm kind of blocked
on that in coreos/fedora-coreos-pipeline#359, so
let's settle for parallel.

NOTE!  This **does not change** the effect of `cosa upload-oscontainer` etc.
Some discussion on that here:
ostreedev/ostree-rs-ext#23
and I will file more issues related to that.

However, what this *will* do is allow us to push this *additional* image
arounnd and make `podman run registry.ci.openshift.org/rhcos/rhcos:4.8` e.g.
work.  In other words to start this will just be something *we* use
to quickly inspect rhcos-as-container, not something we actually ship.
(OK that's kind of a lie, it will end up in a place like e.g.
 http://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/4.9.0-0.nightly-2021-07-20-014024/
 instead of the existing `ostree.tar` so in theory other people outside
 of our group could run it too, but 🤫)
@cgwalters cgwalters mentioned this issue Sep 8, 2021
Merged
ravanelli pushed a commit to ravanelli/os that referenced this issue Oct 5, 2021
@cgwalters @ravanelli
image: Switch to ostree-format: oci
05f137a 
This is the RHCOS version of coreos/fedora-coreos-config#1097

I was hoping to land this in FCOS first but I'm kind of blocked
on that in coreos/fedora-coreos-pipeline#359, so
let's settle for parallel.

NOTE!  This **does not change** the effect of `cosa upload-oscontainer` etc.
Some discussion on that here:
ostreedev/ostree-rs-ext#23
and I will file more issues related to that.

However, what this *will* do is allow us to push this *additional* image
arounnd and make `podman run registry.ci.openshift.org/rhcos/rhcos:4.8` e.g.
work.  In other words to start this will just be something *we* use
to quickly inspect rhcos-as-container, not something we actually ship.
(OK that's kind of a lie, it will end up in a place like e.g.
 http://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/pre-release/4.9.0-0.nightly-2021-07-20-014024/
 instead of the existing `ostree.tar` so in theory other people outside
 of our group could run it too, but 🤫)
@cgwalters cgwalters mentioned this issue Oct 25, 2021
Merged
@cgwalters
Copy link
Member Author

I've been thinking about this and I think it should go to quay.io/fedora/coreos (i.e. not quay.io/coreos/fcos) - in keeping with this thread.

@cgwalters
Copy link
Member Author

WIP PR in #383

@cgwalters cgwalters mentioned this issue Dec 16, 2021
Closed
cheesesashimi pushed a commit to cheesesashimi/fedora-coreos-pipeline that referenced this issue Jan 14, 2022
@cgwalters @cheesesashimi
release: Copy oscontainer to quay.io
ee665e6 
This is a bit hacky but should work.  My initial goal here
is just automated uploads of our builds, so that we can
start getting a better feel for managing "ostree-in-container"
releases.

We should sync to `quay.io/coreos` but that needs someone to set that up.

Also, I'd like to make the destination configurable in the same
way as the S3 bucket, proposal PR in coreos/fedora-coreos-config#1175

Closes: coreos#359
cheesesashimi pushed a commit to cheesesashimi/fedora-coreos-pipeline that referenced this issue Jan 25, 2022
@cgwalters @cheesesashimi
release: Copy oscontainer to quay.io
ffa2b9b 
This is a bit hacky but should work.  My initial goal here
is just automated uploads of our builds, so that we can
start getting a better feel for managing "ostree-in-container"
releases.

We should sync to `quay.io/coreos` but that needs someone to set that up.

Also, I'd like to make the destination configurable in the same
way as the S3 bucket, proposal PR in coreos/fedora-coreos-config#1175

Closes: coreos#359
@jlebon jlebon closed this as completed in 2eb8af0 Jan 25, 2022
@cgwalters cgwalters mentioned this issue Apr 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cgwalters cgwalters

Labels
jira For syncing to JIRA
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

release: Copy oscontainer to quay.io cgwalters/fedora-coreos-pipeline
3 participants
@cgwalters @coverprice @jlebon