Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype for multi-arch manifest #1585

Closed
wants to merge 1 commit into from
Closed

Conversation

romainx
Copy link
Collaborator

@romainx romainx commented Jan 27, 2022

Hello,

This PR is a proposal to fix #1401.
It has two main points described below.

1. Make the manifest tool able to handle different os/architectures

I've made a first beta prototype to make it work on different architecture--there is still a lot of work to do. So the following call will generate a manifest from the image docker pull --platform linux/arm64 jupyter/base-notebook:b020a0cf3b96

python3 -m tagging.create_manifests --short-image-name base-notebook --platform linux/arm64 --owner jupyter --tag b020a0cf3b96 --wiki-path ../wiki

grep "arm" ../wiki/manifests/base-notebook-b020a0cf3b96.md | head
# * OS / Architecture: linux/arm64
# apt/now 2.0.6 arm64 [installed,local]
# base-files/now 11ubuntu5.4 arm64 [installed,local]
# base-passwd/now 3.5.47 arm64 [installed,local]
# bash/now 5.0-6ubuntu1.1 arm64 [installed,local]
# bsdutils/now 1:2.34-0.1ubuntu9.1 arm64 [installed,local]

TODO (at least):

  • put architecture in filename (or find another way of doing it)
  • manage the index

2. Create the manifest after the push to the registry

By creating the manifest after the push to the registry we would be able to access easily (without having to export them) the images of the various architectures.
I guess we will benefit from the buildx cache (or at least I'm pretty sure it's possible to do so) so the pull should not be a penalty.

@consideRatio and @mathbunnyru what do you think about this solution?
Once again is only a early prototype only modified to illustrate the proposal.
Thanks.

@romainx romainx marked this pull request as draft January 27, 2022 21:14
@mathbunnyru
Copy link
Member

@romainx I will take a few days to think if this is a good direction to go :)
I haven't forgotten about this PR :)

@mathbunnyru
Copy link
Member

In general, I think we can try this, I'm not sure how the buildx cache work, but if it's easier to implement the feature this way and it doesn't slow down our already slow CI, then we should try this out.

@romainx
Copy link
Collaborator Author

romainx commented Feb 4, 2022

@mathbunnyru 👍 thanks for the feedback.
I will go further on this and give it a try. It will not be easy to test so maybe I will have to perform a test outside of the project to validate our hypothesis regarding the impact on performances.
I will keep you posted here.

@romainx
Copy link
Collaborator Author

romainx commented Feb 6, 2022

Hello @mathbunnyru I will not go further on this according to what you intend to do in #1407 (comment) and our discussion in #1604.
My fear is to loose time doing something that will not be in line with what you want to do. If worflows are split by platform in different runner doing this change is useless.
So, I'm also closing this one.
Best.

@romainx romainx closed this Feb 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tags and manifests are created based on amd64 images, but assumed for the arm64 images
2 participants