-
Notifications
You must be signed in to change notification settings - Fork 139
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build multiarch bootstrap image #1149
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change | ||||||
---|---|---|---|---|---|---|---|---|
|
@@ -85,10 +85,13 @@ presubmits: | |||||||
memory: "1Gi" | ||||||||
limits: | ||||||||
memory: "1Gi" | ||||||||
- name: build-bootstrap-image | ||||||||
- name: build-multiarch-bootstrap-image | ||||||||
always_run: false | ||||||||
run_if_changed: "images/bootstrap/.*|images/golang/.*" | ||||||||
decorate: true | ||||||||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As we are doubling the number of images that we are building - we may need to update the default 2 hours timeout for this job. I tried running this locally and it timed out - I updated the timeout to 3 hours and the job passed. Here is an example of how to configure the timeout: project-infra/github/ci/prow-deploy/files/jobs/kubevirt/kubevirt/kubevirt-periodics.yaml Lines 110 to 112 in 9ce9f79
Once we have the timeout increased for this and the publish job - I think this looks good. |
||||||||
decoration_config: | ||||||||
grace_period: 5m0s | ||||||||
timeout: 4h0m0s | ||||||||
labels: | ||||||||
preset-dind-enabled: "true" | ||||||||
preset-docker-mirror-proxy: "true" | ||||||||
|
@@ -103,16 +106,16 @@ presubmits: | |||||||
- "-ce" | ||||||||
- | | ||||||||
cd images | ||||||||
./publish_image.sh -b bootstrap quay.io kubevirtci | ||||||||
./publish_image.sh -b golang quay.io kubevirtci | ||||||||
./publish_multiarch_image.sh -b bootstrap quay.io kubevirtci | ||||||||
./publish_multiarch_image.sh -b -l golang quay.io kubevirtci | ||||||||
# docker-in-docker needs privileged mode | ||||||||
securityContext: | ||||||||
privileged: true | ||||||||
resources: | ||||||||
requests: | ||||||||
memory: "1Gi" | ||||||||
memory: "8Gi" | ||||||||
limits: | ||||||||
memory: "1Gi" | ||||||||
memory: "8Gi" | ||||||||
- name: build-kubekins-e2e-image | ||||||||
always_run: false | ||||||||
run_if_changed: "images/kubekins-e2e/.*" | ||||||||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
FROM bootstrap | ||
ARG ARCH | ||
FROM bootstrap-$ARCH | ||
|
||
ENV GIMME_GO_VERSION=1.17.8 | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,121 @@ | ||
#!/bin/bash -xe | ||
archs=(amd64 arm64) | ||
|
||
main() { | ||
local build_only local_base_image | ||
local_base_image=false | ||
while getopts "blh" opt; do | ||
case "$opt" in | ||
b) | ||
build_only=true | ||
;; | ||
l) | ||
local_base_image=true | ||
;; | ||
h) | ||
help | ||
exit 0 | ||
;; | ||
*) | ||
echo "Invalid argument: $opt" | ||
help | ||
exit 1 | ||
esac | ||
done | ||
shift $((OPTIND-1)) | ||
local build_target="${1:?}" | ||
local registry="${2:?}" | ||
local registry_org="${3:?}" | ||
local full_image_name image_tag base_image | ||
|
||
image_tag="$(get_image_tag)" | ||
full_image_name="$( | ||
get_full_image_name \ | ||
"$registry" \ | ||
"$registry_org" \ | ||
"${build_target##*/}" \ | ||
"$image_tag" | ||
)" | ||
|
||
( | ||
cd "$build_target" | ||
base_image="$(get_base_image)" | ||
|
||
build_image $local_base_image "$build_target" "$full_image_name" "$base_image" | ||
) | ||
[[ $build_only ]] && return | ||
publish_image "$full_image_name" | ||
publish_manifest "$full_image_name" | ||
} | ||
|
||
help() { | ||
cat <<EOF | ||
Usage: | ||
./publish_multiarch_image.sh [OPTIONS] BUILD_TARGET REGISTRY REGISTRY_ORG | ||
|
||
Build and publish multiarch infra images. | ||
|
||
OPTIONS | ||
-h Show this help message and exit. | ||
-b Only build the image and exit. Do not publish the built image. | ||
-l Use local base image | ||
EOF | ||
} | ||
|
||
get_base_image() { | ||
name="$(cat Dockerfile |grep FROM|awk '{print $2}')" | ||
echo "${name}" | ||
} | ||
|
||
get_image_tag() { | ||
local current_commit today | ||
current_commit="$(git rev-parse HEAD)" | ||
today="$(date +%Y%m%d)" | ||
echo "v${today}-${current_commit:0:7}" | ||
} | ||
|
||
build_image() { | ||
local local_base_image=${1:?} | ||
local build_target="${2:?}" | ||
local image_name="${3:?}" | ||
local base_image="${4:?}" | ||
# add qemu-user-static | ||
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes | ||
# build multi-arch images | ||
for arch in ${archs[*]};do | ||
if [[ $local_base_image == false ]]; then | ||
brianmcarey marked this conversation as resolved.
Show resolved
Hide resolved
|
||
docker pull --platform="linux/${arch}" ${base_image} | ||
fi | ||
docker build --platform="linux/${arch}" --build-arg ARCH=${arch} --build-arg IMAGE_ARG=${build_target} . -t "${image_name}-${arch}" -t "${build_target}-${arch}" | ||
done | ||
} | ||
|
||
publish_image() { | ||
local full_image_name="${1:?}" | ||
for arch in ${archs[*]};do | ||
docker push "${full_image_name}-${arch}" | ||
done | ||
} | ||
|
||
publish_manifest() { | ||
export DOCKER_CLI_EXPERIMENTAL="enabled" | ||
local amend | ||
local full_image_name="${1:?}" | ||
amend="" | ||
for arch in ${archs[*]};do | ||
amend+=" --amend ${full_image_name}-${arch}" | ||
done | ||
docker manifest create ${full_image_name} ${amend} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It would be cool to try this with a tool like buildah but we can try doing that later. Other kubevirt projects are starting to adopt buildah for multi-arch builds - kubevirt/hostpath-provisioner#115 |
||
docker manifest push --purge ${full_image_name} | ||
} | ||
|
||
get_full_image_name() { | ||
local registry="${1:?}" | ||
local registry_org="${2:?}" | ||
local image_name="${3:?}" | ||
local tag="${4:?}" | ||
|
||
echo "${registry}/${registry_org}/${image_name}:${tag}" | ||
} | ||
|
||
main "$@" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you need this option?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point - we probably don't need this specific local image flag.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need this flag. For Dockerfile whose baseimage is from remote repository, we need manually pull correct CPU arch base image, otherwise it would use local image even if the local image has different CPU arch with
--platform
.For these Dockerfile we use local built image, like the
golang
image, we do not need to pull base image.