arm64 support #90
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,3 +1,5 @@ | ||
| build | ||
| .kvm-images | ||
| .installed-requirements | ||
| .installed-qemu | ||
| namibase/nami-linux-x64.tar.gz |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,22 +1,94 @@ | ||
| language: bash | ||
| dist: focal | ||
| virt: vm | ||
| group: edge | ||
| os: linux | ||
| services: | ||
| - docker | ||
|
|
||
| env: | ||
| global: | ||
| - BASENAME=bitnami/minideb | ||
| - LATEST=buster | ||
| - DISTS_WITH_SNAPSHOT="$LATEST" | ||
|
|
||
| .build_job: &build_job | ||
| stage: build | ||
| before_install: | ||
| - docker version | ||
| # Fix for Ubuntu Xenial apt-daily.service triggering | ||
| # https://unix.stackexchange.com/questions/315502/how-to-disable-apt-daily-service-on-ubuntu-cloud-vm-image | ||
| - | | ||
| while sudo fuser /var/{lib/{dpkg,apt/lists},cache/apt/archives}/lock >/dev/null 2>&1; do | ||
| sleep 1 | ||
| done | ||
| - sudo rm -f /usr/local/bin/jq | ||
| install: | ||
| - sudo make .installed-requirements | ||
| script: | ||
| - sudo bash buildone $DIST $PLATFORM | ||
| - 'if [[ "$TRAVIS_BRANCH" == "master" && "$DISTS_WITH_SNAPSHOT" =~ (^|[[:space:]])"$DIST"($|[[:space:]]) ]] ; then sudo bash buildone_snapshot $DIST "$(./snapshot_id)" $PLATFORM ; fi' | ||
| after_success: | ||
| - 'if [[ "$TRAVIS_BRANCH" == "master" && "$LATEST" == "$DIST" ]] ; then sudo docker tag "$BASENAME:$DIST-$PLATFORM" "$BASENAME:latest-$PLATFORM" ; fi' | ||
| - 'if [[ "$TRAVIS_BRANCH" == "master" ]] ; then sudo bash pushone $DIST $PLATFORM ; fi' | ||
| - 'if [[ "$TRAVIS_BRANCH" == "master" && "$DISTS_WITH_SNAPSHOT" =~ (^|[[:space:]])"$DIST"($|[[:space:]]) ]] ; then sudo bash pushone "$DIST-snapshot-$(./snapshot_id)" $PLATFORM ; fi' | ||
| - 'if [[ "$TRAVIS_BRANCH" == "master" && "$LATEST" == "$DIST" ]] ; then sudo bash pushone latest $PLATFORM ; fi' | ||
|
|
||
| jobs: | ||
| include: | ||
| - stage: shellcheck | ||
| install: | ||
| - sudo apt-get -qq update | ||
| - sudo apt-get install -y shellcheck | ||
| script: bash shellcheck | ||
| - <<: *build_job | ||
| arch: amd64 | ||
| env: | ||
| - DIST=jessie PLATFORM=amd64 | ||
| - <<: *build_job | ||
| arch: amd64 | ||
| env: | ||
| - DIST=stretch PLATFORM=amd64 | ||
| - <<: *build_job | ||
| arch: amd64 | ||
| env: | ||
| - DIST=buster PLATFORM=amd64 | ||
| - <<: *build_job | ||
| arch: arm64-graviton2 | ||
| env: | ||
| - DIST=stretch PLATFORM=arm64 | ||
|
There was a problem hiding this comment. Unfortunately, I think we need to use There was a problem hiding this comment. There was a problem hiding this comment. Didn't notice that build was running on There was a problem hiding this comment. Anyway on There was a problem hiding this comment. There could be a problem with Stating that There was a problem hiding this comment. The third option is to try to build the images on GitHub Actions. There was a problem hiding this comment.
where travis runs stages sequentially on the same machine (multiple machines also doable), github runs jobs in parallel on multiple machines by default. this should solve any speed issues probably? docker provides a qemu setup action for multiarch GH workflows. Im I'm not mistaken, GH by default runs up to four jobs on four runners (each runner 2CPU/7GB) concurrently (might be four not sure anymore) which can be decreased as well. all you'd need to do is define a job with a step that runs the qemu_build script with args for example, this CI runs in parallel on new commits to master, and all commits to all PRs (and emulates Travis's auto-cancel previous runs feature) , plus CD runs once upon a GitHub Release or Prerelease. as alternative to split files, add a push step to the job that only runs on a release/prerelease event: on:
pull_request:
push:
branches: master
release:
types: [released, prereleased]
workflow_dispatch: # allows triggering manually from the Actions tab
...
- name: Push image
if: github.event_name == 'release'
run: docker pushThere was a problem hiding this comment. For the records: I just tried to build the images on GH Actions on my fork ( There was a problem hiding this comment. 14 mins for buster arm64.. +1 for the effort though! plz dont delete the branch :) |
||
| - <<: *build_job | ||
| arch: arm64-graviton2 | ||
| env: | ||
| - DIST=buster PLATFORM=arm64 | ||
| - stage: deploy | ||
| if: branch = master AND type = push | ||
| env: | ||
| - DISTS="stretch buster latest" | ||
dani8art marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. As jessie does not get a multiplatform manifest and we are tagging it as There was a problem hiding this comment. Right, I've added a build job for jessie image push. |
||
| before_install: mkdir $HOME/.docker | ||
| install: 'echo "{ \"experimental\": \"enabled\" }" > $HOME/.docker/config.json' | ||
| script: | ||
| - | | ||
| if [ -n "${DOCKER_PASSWORD:-}" ]; then | ||
| docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD" | ||
| fi | ||
|
|
||
| # Create and merge a PR to update minideb-extras | ||
| CIRCLE_CI_FUNCTIONS_URL=${CIRCLE_CI_FUNCTIONS_URL:-https://raw.githubusercontent.com/bitnami/test-infra/master/circle/functions} | ||
| source <(curl -sSL "$CIRCLE_CI_FUNCTIONS_URL") | ||
| for DIST in $DISTS ; do | ||
| sudo docker manifest create $BASENAME:$DIST $BASENAME:$DIST-amd64 $BASENAME:$DIST-arm64 | ||
| sudo docker manifest push $BASENAME:$DIST | ||
| sudo docker pull $BASENAME:$DIST | ||
|
|
||
| if [[ "$DISTS_WITH_SNAPSHOT" =~ (^|[[:space:]])"$DIST"($|[[:space:]]) ]] ; then | ||
| SNAPSHOT_NAME="$DIST-snapshot-$(./snapshot_id)" | ||
| sudo docker manifest create $SNAPSHOT_NAME:$DIST $SNAPSHOT_NAME:$DIST-amd64 $SNAPSHOT_NAME:$DIST-arm64 | ||
|
There was a problem hiding this comment. I think you meant |
||
| sudo docker manifest push $SNAPSHOT_NAME:$DIST | ||
| sudo docker pull $SNAPSHOT_NAME:$DIST | ||
| fi | ||
|
|
||
| # Use '.RepoDigests 0' for getting Dockerhub repo digest as it was the first pushed | ||
| DIST_REPO_DIGEST=$(docker image inspect --format '{{index .RepoDigests 0}}' "$BASENAME:${DIST}") | ||
juamedgod marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| update_minideb_derived "https://github.com/$BASENAME-runtimes" "$DIST" "$DIST_REPO_DIGEST" | ||
| done | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -eu | ||
|
|
||
| do_sudo() { | ||
| if [[ "0" == "$(id --user)" ]]; then | ||
| "$@" | ||
| else | ||
| sudo "$@" | ||
| fi | ||
| } | ||
|
|
||
| while do_sudo fuser /var/{lib/{dpkg,apt/lists},cache/apt/archives}/lock >/dev/null 2>&1; do | ||
| sleep 1 | ||
| done | ||
|
|
||
| do_sudo apt-get update | ||
| do_sudo apt-get install -y qemu-kvm libvirt-bin qemu-utils genisoimage virtinst curl rsync qemu-system-x86 qemu-system-arm cloud-image-utils | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,46 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -e | ||
| set -u | ||
| set -o pipefail | ||
|
|
||
| DIST=${1:?Specify the distrubution name} | ||
| PLATFORM=${2:-amd64} | ||
|
|
||
| BASENAME=bitnami/minideb | ||
| GCR_BASENAME=gcr.io/bitnami-containers/minideb | ||
| QUAY_BASENAME=quay.io/bitnami/minideb | ||
|
|
||
| if [ -n "${DOCKER_PASSWORD:-}" ]; then | ||
| docker login -u "$DOCKER_USERNAME" -p "$DOCKER_PASSWORD" | ||
| fi | ||
|
|
||
| if [ -n "${QUAY_PASSWORD:-}" ]; then | ||
| docker login -u "$QUAY_USERNAME" -p "$QUAY_PASSWORD" quay.io | ||
| fi | ||
|
|
||
| if [ -n "${GCR_KEY:-}" ]; then | ||
| gcloud auth activate-service-account "$GCR_EMAIL" --key-file <(echo "$GCR_KEY") | ||
| fi | ||
|
|
||
| ENABLE_DOCKER_CONTENT_TRUST=0 | ||
| if [ -n "${DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE:-}" ] && [ -n "${DOCKER_CONTENT_TRUST_REPOSITORY_KEY:-}" ]; then | ||
| tmpdir=$(mktemp -d) | ||
| (cd "${tmpdir}" && bash -c 'echo -n "${DOCKER_CONTENT_TRUST_REPOSITORY_KEY}" | base64 -d > key') | ||
| chmod 400 "${tmpdir}/key" | ||
| docker trust key load "${tmpdir}/key" | ||
| rm -rf "${tmpdir}" | ||
| export ENABLE_DOCKER_CONTENT_TRUST=1 | ||
| fi | ||
|
|
||
| push() { | ||
| local dist="$1" | ||
| DOCKER_CONTENT_TRUST=${ENABLE_DOCKER_CONTENT_TRUST} docker push "${BASENAME}:${dist}" | ||
| docker push "${QUAY_BASENAME}:${dist}" | ||
| gcloud docker -- push "${GCR_BASENAME}:${dist}" | ||
| } | ||
|
|
||
| docker tag "${BASENAME}:${DIST}" "${QUAY_BASENAME}:${DIST}-${PLATFORM}" | ||
| docker tag "${BASENAME}:${DIST}" "${GCR_BASENAME}:${DIST}-${PLATFORM}" | ||
| push "$DIST-${PLATFORM}" | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is this needed? (deleting jq)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
.installed-requirementsmake target fails without removing the travis-included jq executableThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uhm, which error was it throwing? If the package was already installed it should simply ignore it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't remember the exact error message, but the
jqexecutable is not installed in travis machine via apt.Apt simply refuses to overwrite an existing file.