-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore(build): add support for multiarch build #71
Changes from 4 commits
6ff47b5
abf49b0
050f650
51ca422
cb2661c
15ac321
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
# Copyright 2018-2020 The OpenEBS Authors. All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
name: build | ||
|
||
on: ['push'] | ||
|
||
jobs: | ||
lint: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Shellcheck | ||
uses: reviewdog/action-shellcheck@v1 | ||
with: | ||
github_token: ${{ secrets.github_token }} | ||
reporter: github-pr-review | ||
path: '.' | ||
pattern: '*.sh' | ||
|
||
cstor-base: | ||
runs-on: ubuntu-latest | ||
needs: ['lint'] | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set tag | ||
run: | | ||
BRANCH="${GITHUB_REF##*/}" | ||
CI_TAG=${BRANCH#v}-ci | ||
if [ ${BRANCH} = "master" ]; then | ||
CI_TAG="ci" | ||
fi | ||
echo "::set-env name=TAG::${CI_TAG}" | ||
echo "::set-env name=BRANCH::${BRANCH}" | ||
echo "BRANCH: ${BRANCH}" | ||
echo "TAG: ${CI_TAG}" | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@v1 | ||
with: | ||
platforms: all | ||
|
||
- name: Set up Docker Buildx | ||
id: buildx | ||
uses: docker/setup-buildx-action@v1 | ||
with: | ||
version: latest | ||
|
||
- name: Login to Docker Hub | ||
uses: docker/login-action@v1 | ||
with: | ||
username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
||
- name: Build & Push Image | ||
env: | ||
IMAGE_ORG: ${{ secrets.IMAGE_ORG}} | ||
run: | | ||
make -f Makefile.buildx.mk docker.buildx.cstor-base | ||
make -f Makefile.buildx.mk buildx.push.cstor-base | ||
|
||
cstor: | ||
runs-on: ubuntu-latest | ||
needs: ['lint', 'cstor-base'] | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set tag | ||
run: | | ||
BRANCH="${GITHUB_REF##*/}" | ||
CI_TAG=${BRANCH#v}-ci | ||
if [ ${BRANCH} = "master" ]; then | ||
CI_TAG="ci" | ||
fi | ||
echo "::set-env name=TAG::${CI_TAG}" | ||
echo "::set-env name=BRANCH::${BRANCH}" | ||
echo "BRANCH: ${BRANCH}" | ||
echo "TAG: ${CI_TAG}" | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@v1 | ||
with: | ||
platforms: all | ||
|
||
- name: Set up Docker Buildx | ||
id: buildx | ||
uses: docker/setup-buildx-action@v1 | ||
with: | ||
version: latest | ||
|
||
- name: Login to Docker Hub | ||
uses: docker/login-action@v1 | ||
with: | ||
username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
||
- name: Build & Push Image | ||
env: | ||
IMAGE_ORG: ${{ secrets.IMAGE_ORG}} | ||
run: | | ||
make -f Makefile.buildx.mk docker.buildx.cstor | ||
make -f Makefile.buildx.mk buildx.push.cstor |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
# Copyright 2018-2020 The OpenEBS Authors. All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
name: ci | ||
|
||
on: | ||
pull_request: | ||
branches: | ||
# on pull requests to master and release branches | ||
- master | ||
- 'v*' | ||
|
||
jobs: | ||
lint: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Shellcheck | ||
uses: reviewdog/action-shellcheck@v1 | ||
with: | ||
github_token: ${{ secrets.github_token }} | ||
reporter: github-pr-review | ||
path: '.' | ||
pattern: '*.sh' | ||
|
||
cstor-base: | ||
runs-on: ubuntu-latest | ||
needs: ['lint'] | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v2 | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@v1 | ||
with: | ||
platforms: all | ||
|
||
- name: Set up Docker Buildx | ||
id: buildx | ||
uses: docker/setup-buildx-action@v1 | ||
with: | ||
version: latest | ||
|
||
- name: Build Image | ||
env: | ||
IMG_RESULT: cache | ||
run: make -f Makefile.buildx.mk docker.buildx.cstor-base | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,95 @@ | ||
# Copyright 2018-2020 The OpenEBS Authors. All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
name: release | ||
|
||
on: | ||
create: | ||
tags: | ||
- 'v*' | ||
|
||
jobs: | ||
cstor-base: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v1 | ||
|
||
- name: Set Tag | ||
run: | | ||
TAG="${GITHUB_REF#refs/*/v}" | ||
echo "::set-env name=TAG::${TAG}" | ||
echo "::set-env name=RELEASE_TAG::${TAG}" | ||
echo "RELEASE_TAG ${TAG}" | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@v1 | ||
with: | ||
platforms: all | ||
|
||
- name: Set up Docker Buildx | ||
id: buildx | ||
uses: docker/setup-buildx-action@v1 | ||
with: | ||
version: latest | ||
|
||
- name: Login to Docker Hub | ||
uses: docker/login-action@v1 | ||
with: | ||
username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
||
- name: Build & Push Image | ||
env: | ||
IMAGE_ORG: ${{ secrets.IMAGE_ORG}} | ||
run: | | ||
make -f Makefile.buildx.mk docker.buildx.cstor-base | ||
make -f Makefile.buildx.mk buildx.push.cstor-base | ||
|
||
cstor: | ||
runs-on: ubuntu-latest | ||
needs: ['cstor-base'] | ||
steps: | ||
- name: Checkout | ||
uses: actions/checkout@v1 | ||
|
||
- name: Set Tag | ||
run: | | ||
TAG="${GITHUB_REF#refs/*/v}" | ||
echo "::set-env name=TAG::${TAG}" | ||
echo "::set-env name=RELEASE_TAG::${TAG}" | ||
echo "RELEASE_TAG ${TAG}" | ||
|
||
- name: Set up QEMU | ||
uses: docker/setup-qemu-action@v1 | ||
with: | ||
platforms: all | ||
|
||
- name: Set up Docker Buildx | ||
id: buildx | ||
uses: docker/setup-buildx-action@v1 | ||
with: | ||
version: latest | ||
|
||
- name: Login to Docker Hub | ||
uses: docker/login-action@v1 | ||
with: | ||
username: ${{ secrets.DOCKER_HUB_USERNAME }} | ||
password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }} | ||
|
||
- name: Build & Push Image | ||
env: | ||
IMAGE_ORG: ${{ secrets.IMAGE_ORG}} | ||
run: | | ||
make -f Makefile.buildx.mk docker.buildx.cstor | ||
make -f Makefile.buildx.mk buildx.push.cstor |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
# Copyright 2020 The OpenEBS Authors. All rights reserved. | ||
# | ||
# Licensed under the Apache License, Version 2.0 (the "License"); | ||
# you may not use this file except in compliance with the License. | ||
# You may obtain a copy of the License at | ||
# | ||
# http://www.apache.org/licenses/LICENSE-2.0 | ||
# | ||
# Unless required by applicable law or agreed to in writing, software | ||
# distributed under the License is distributed on an "AS IS" BASIS, | ||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
# See the License for the specific language governing permissions and | ||
# limitations under the License. | ||
|
||
# IMAGE_ORG can be used to customize the organization | ||
# under which images should be pushed. | ||
# By default the organization name is `openebs`. | ||
|
||
ifeq (${IMAGE_ORG}, ) | ||
IMAGE_ORG = openebs | ||
export IMAGE_ORG | ||
endif | ||
|
||
# Specify the date of build | ||
DBUILD_DATE=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ') | ||
|
||
# Specify the docker arg for repository url | ||
ifeq (${DBUILD_REPO_URL}, ) | ||
DBUILD_REPO_URL="https://github.com/openebs/libcstor" | ||
export DBUILD_REPO_URL | ||
endif | ||
|
||
# Specify the docker arg for website url | ||
ifeq (${DBUILD_SITE_URL}, ) | ||
DBUILD_SITE_URL="https://openebs.io" | ||
export DBUILD_SITE_URL | ||
endif | ||
|
||
# ============================================================================== | ||
# Build Options | ||
|
||
export DBUILD_ARGS=--build-arg DBUILD_DATE=${DBUILD_DATE} --build-arg DBUILD_REPO_URL=${DBUILD_REPO_URL} --build-arg DBUILD_SITE_URL=${DBUILD_SITE_URL} | ||
|
||
ifeq (${TAG}, ) | ||
export TAG=ci | ||
endif | ||
|
||
|
||
# Build cstor-base & cstor docker image with buildx | ||
# Experimental docker feature to build cross platform multi-architecture docker images | ||
# https://docs.docker.com/buildx/working-with-buildx/ | ||
|
||
# default list of platforms for which multiarch image is built | ||
ifeq (${PLATFORMS}, ) | ||
export PLATFORMS="linux/amd64,linux/arm64,linux/ppc64le" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. armv7 support? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. armv7 will require some code changes as the build is failing. One of the users who has a arm/v7 machine is trying that out https://kubernetes.slack.com/archives/CUAKPFU78/p1604006389024000 |
||
endif | ||
|
||
# if IMG_RESULT is unspecified, by default the image will be pushed to registry | ||
ifeq (${IMG_RESULT}, load) | ||
export PUSH_ARG="--load" | ||
# if load is specified, image will be built only for the build machine architecture. | ||
export PLATFORMS="local" | ||
else ifeq (${IMG_RESULT}, cache) | ||
# if cache is specified, image will only be available in the build cache, it won't be pushed or loaded | ||
# therefore no PUSH_ARG will be specified | ||
else | ||
export PUSH_ARG="--push" | ||
endif | ||
|
||
# Name of the multiarch image for cstor-base | ||
DOCKERX_IMAGE_CSTOR_BASE:=${IMAGE_ORG}/cstor-base:${TAG} | ||
|
||
# Name of the multiarch image for cstor | ||
DOCKERX_IMAGE_CSTOR:=${IMAGE_ORG}/cstor-pool:${TAG} | ||
|
||
# COMPONENT names for image builds | ||
CSTOR_BASE:=cstor-base | ||
CSTOR:=cstor | ||
|
||
.PHONY: docker.buildx | ||
docker.buildx: | ||
export DOCKER_CLI_EXPERIMENTAL=enabled | ||
@if ! docker buildx ls | grep -q container-builder; then\ | ||
docker buildx create --platform ${PLATFORMS} --name container-builder --use;\ | ||
fi | ||
@docker buildx build --platform ${PLATFORMS} \ | ||
-t "$(DOCKERX_IMAGE_NAME)" ${BUILD_ARGS} -f $(PWD)/docker/$(COMPONENT).Dockerfile \ | ||
. ${PUSH_ARG} | ||
@echo "--> Build docker image: $(DOCKERX_IMAGE_NAME)" | ||
@echo | ||
|
||
.PHONY: docker.buildx.cstor-base | ||
docker.buildx.cstor-base: DOCKERX_IMAGE_NAME=$(DOCKERX_IMAGE_CSTOR_BASE) | ||
docker.buildx.cstor-base: COMPONENT=$(CSTOR_BASE) | ||
docker.buildx.cstor-base: BUILD_ARGS=${DBUILD_ARGS} | ||
docker.buildx.cstor-base: docker.buildx | ||
|
||
.PHONY: docker.buildx.cstor | ||
docker.buildx.cstor: DOCKERX_IMAGE_NAME=$(DOCKERX_IMAGE_CSTOR) | ||
docker.buildx.cstor: COMPONENT=$(CSTOR) | ||
docker.buildx.cstor: BUILD_ARGS=--build-arg BASE_IMAGE=$(DOCKERX_IMAGE_CSTOR_BASE) ${DBUILD_ARGS} | ||
docker.buildx.cstor: docker.buildx | ||
|
||
.PHONY: buildx.push.cstor-base | ||
buildx.push.cstor-base: | ||
BUILDX=true DIMAGE=${IMAGE_ORG}/cstor-base ./docker/buildxpush.sh | ||
|
||
.PHONY: buildx.push.cstor | ||
buildx.push.cstor: | ||
BUILDX=true DIMAGE=${IMAGE_ORG}/cstor-pool ./docker/buildxpush.sh |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you cache the image, you wont be able to use the image. after exiting buildix. It exists only in the buildkit cache.
For pull requests, this line will always pull image from the docker hub. It wont use the locally built cstor-base image.
We may need to push the image to an intermediate repo to use that image.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO we can just remove the cstor build and have only the cstor-base build in pull request as the cstor image only copies a entrypoint script to the cstor-base image.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, Then I think for now, that would work. In the long term, will need to identify a way to push intermediate images and use them.