Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSS-35 Feat/push base images with hash in tag #30

Merged
merged 12 commits into from
Jun 26, 2024

Conversation

badGarnet
Copy link
Contributor

@badGarnet badGarnet commented Jun 21, 2024

Summary

This PR partially resolves https://unstructured-ai.atlassian.net/browse/OSS-35 by:

  • add short hash to the base image tag so that new base images doesn't overwrite existing ones
  • add new tags wolfi-base-latest, rocky9.2-cpu-latest, rocky9.2-gpu-latest, rocky9.2-slim-latest that are the latest builds from ci; this is intended to facilitate smoother downstream development experience

Test instructions

During test two tags are pushed to quay.io:

  • wolfi-base-latest
  • wolfi-base-c62024e

Try pull those locally with arm64 platform they both should give the same image that has the hash 66f84447138b893b272e75044f9967379de3e874849cb2b9cb7aa2b3e9dea4e6
if pull with platform being amd64 they should both have hash 7c3af225a39f730f4feee705df6cd8d1570739dc130456cf589ac53347da0f1d

Next steps

  • add steps in ci to delete out of date images (either based on date or number of images in the repo with the same base)
  • add steps in ci to automatically generate pull request to affect downstream repos to update image reference

- tag wolfi image with its short sha as part of the tag
- create a new wolfi-base-latest tag
@badGarnet badGarnet changed the title Feat/push wolfie image with hash in tag OSS-35 Feat/push wolfie image with hash in tag Jun 21, 2024
@badGarnet badGarnet marked this pull request as ready for review June 25, 2024 20:39
Comment on lines 108 to 109
env:
IMAGE_TAG: wolfi-base-latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's easier to read when env is before run

Comment on lines 96 to 107
if [[ ${{ matrix.image }} == 'wolfi-base' ]]; then
export IMAGE_TAG=${{ matrix.image }}-$SHORT_SHA
else
export IMAGE_TAG=${{ matrix.image }}
fi
docker manifest create $DOCKER_REPOSITORY/$DOCKER_IMAGE:${IMAGE_TAG} $DOCKER_BUILD_REPOSITORY:${{ matrix.image }}-amd64 $DOCKER_BUILD_REPOSITORY:${{ matrix.image }}-arm64
docker manifest push $DOCKER_REPOSITORY/$DOCKER_IMAGE:${IMAGE_TAG}
- name: Push latest manifest for wolfi base
if: ${{ matrix.image }} == 'wolfi-base'
run: |
docker manifest create $DOCKER_REPOSITORY/$DOCKER_IMAGE:${IMAGE_TAG} $DOCKER_BUILD_REPOSITORY:${{ matrix.image }}-amd64 $DOCKER_BUILD_REPOSITORY:${{ matrix.image }}-arm64
docker manifest push $DOCKER_REPOSITORY/$DOCKER_IMAGE:${IMAGE_TAG}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do that in 1 step if you use if-else in bash anyway

Copy link
Contributor

@MthwRobinson MthwRobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pulled the latest and c62024e images from quay and verified the hashes match. Small nit: wolfi rather than wolfie in the PR title.

As a broad comment, for the rockylinux images we prevent the image from being overwritten by incrementing the number after -, so rocky9.2-10-cpu would become rocky9.2-11-cpu the next time it changes. I like this set up better because it doesn't rely on someone knowing/remembering that. And currently if you don't increment that number it overwrites the image and breaks downstream builds. As a follow on, I'd propose adopting this workflow for the rockylinux images as well.

@badGarnet badGarnet changed the title OSS-35 Feat/push wolfie image with hash in tag OSS-35 Feat/push wolfi image with hash in tag Jun 26, 2024
@badGarnet badGarnet changed the title OSS-35 Feat/push wolfi image with hash in tag OSS-35 Feat/push base images with hash in tag Jun 26, 2024
Copy link
Contributor

@MthwRobinson MthwRobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@micmarty-deepsense micmarty-deepsense left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, this PR will change things for the better 😌

@badGarnet badGarnet merged commit 7e30303 into main Jun 26, 2024
11 checks passed
@badGarnet badGarnet deleted the feat/push-wolfie-image-with-hash-in-tag branch June 26, 2024 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants