Skip to content

Commit

Permalink
Fixes to CI build workflows for dev, staging and production pipelines…
Browse files Browse the repository at this point in the history
… into feature/evm (#1906)

* Enable aarch64-apple-darwin builds (#1893) (#1894)

* Build pipeline for building on arm64 macOS host architecture. Included CI build workflow for building the arm64 binaries

* Rename to aarch64-apple-darwin to resolve build dependency bug

* Amended markdown documentation

(cherry picked from commit 29e10be)

* CI workflow fixes for dev, staging and production pipelines

* Fix naming bug

* Removed publishing to dockerhub pipeline in build staging workflow, and renamed "mac" to "osx"

* Added workflow to push arm build image into Dockerhub. Rename docker tag convention to differentiate arm and x64 build images.

* Fix docker tags

* Reverting changes to pushing only linux-x64 into dockerhub registry.

* Include image layer to install rust dependencies

* Install protobuf-compiler pkg dep

* bug fixes

* Fix bug in pkg_local_ensure_osx_sysroot function that does not exit out of entered dir if the package exists. Include new gnu-tar support for macos in platform_init function to support tar --transform on macos

* Install target toolchains in dockerfiles

* Fix target toolchain for linux-arm docker build

* Resolve rustc toolchain bug

* Removed optional label in proto3 syntax to fix build error

* Fix _tar func in make.sh to pass all args to gnu-tar, and fix docker release build workflow to only push x64 linux to docker registry.

* Re-order build targets to select major dev envs first.

* Fix merge errors

* Revert to feature/evm branch version

* Reverting eth.proto version to feature/evm version
t Please enter the commit message for your changes. Lines starting
  • Loading branch information
sieniven authored Apr 21, 2023
1 parent 57b0565 commit a36f2a2
Show file tree
Hide file tree
Showing 15 changed files with 517 additions and 215 deletions.
19 changes: 1 addition & 18 deletions .github/workflows/build-dev.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,23 +45,6 @@ jobs:
docker tag defichain-x86_64-pc-linux-gnu:${ver} defi/defichain:${ver}
docker push defi/defichain:${ver}
linux-armhf:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="arm-linux-gnueabihf" ./make.sh docker-release

- name: Publish artifact - arm-linux-gnueabihf
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf
path: ./build/defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz

win-x64:
runs-on: ubuntu-latest
steps:
Expand All @@ -79,7 +62,7 @@ jobs:
name: defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32
path: ./build/defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32.tar.gz

mac-x64:
osx-x64:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
Expand Down
152 changes: 139 additions & 13 deletions .github/workflows/build-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ env:
DOCKER_HUB_USER: defi

jobs:
linux:
linux-x64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

Expand Down Expand Up @@ -43,7 +43,45 @@ jobs:
done
docker push defi/defichain:${ver}
windows:
linux-armhf:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="arm-linux-gnueabihf" ./make.sh docker-release

- name: Publish artifact - arm-linux-gnueabihf
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf
path: ./build/defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz

linux-aarch64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="aarch64-linux-gnu" ./make.sh docker-release

- name: Publish artifact - aarch64-linux-gnu
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu
path: ./build/defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu.tar.gz

win-x64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

Expand All @@ -62,7 +100,7 @@ jobs:
name: defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32
path: ./build/defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32.tar.gz

macos:
osx-x64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

Expand All @@ -81,11 +119,33 @@ jobs:
name: defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin
path: ./build/defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin.tar.gz

osx-aarch64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="aarch64-apple-darwin" ./make.sh docker-release

- name: Publish artifact - aarch64-apple-darwin
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin
path: ./build/defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin.tar.gz

create-release:
needs:
- linux
- windows
- macos
- linux-x64
- linux-armhf
- linux-aarch64
- win-x64
- osx-x64
- osx-aarch64
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

Expand All @@ -101,7 +161,7 @@ jobs:
- name: Get artifacts
uses: actions/download-artifact@v3

- name: zip package for windows
- name: zip package for win-x64
run: |
set -e; ver=${{ env.BUILD_VERSION }}
cd defichain-${ver}-x86_64-w64-mingw32
Expand All @@ -121,12 +181,18 @@ jobs:
set -e; ver=${{ env.BUILD_VERSION }}
cd ./defichain-${ver}-x86_64-pc-linux-gnu
sha256sum ./defichain-${ver}-x86_64-pc-linux-gnu.tar.gz > ./defichain-${ver}-x86_64-pc-linux-gnu.tar.gz.SHA256
cd ./defichain-${ver}-arm-linux-gnueabihf
sha256sum ./defichain-${ver}-arm-linux-gnueabihf.tar.gz > ./defichain-${ver}-arm-linux-gnueabihf.tar.gz.SHA256
cd ./defichain-${ver}-aarch64-linux-gnu
sha256sum ./defichain-${ver}-aarch64-linux-gnu.tar.gz > ./defichain-${ver}-aarch64-linux-gnu.tar.gz.SHA256
cd .. && cd ./defichain-${ver}-x86_64-w64-mingw32
sha256sum ./defichain-${ver}-x86_64-w64-mingw32.zip > ./defichain-${ver}-x86_64-w64-mingw32.zip.SHA256
cd .. && cd ./defichain-${ver}-x86_64-apple-darwin
sha256sum ./defichain-${ver}-x86_64-apple-darwin.tar.gz > ././defichain-${ver}-x86_64-apple-darwin.tar.gz.SHA256
cd .. && cd ./defichain-${ver}-aarch64-apple-darwin
sha256sum ./defichain-${ver}-aarch64-apple-darwin.tar.gz > ././defichain-${ver}-aarch64-apple-darwin.tar.gz.SHA256
- name: Upload release asset - linux
- name: Upload release asset - linux-x64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -136,7 +202,7 @@ jobs:
asset_name: defichain-${{ env.BUILD_VERSION }}-x86_64-pc-linux-gnu.tar.gz
asset_content_type: application/gzip

- name: Upload checksum - linux
- name: Upload checksum - linux-x64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -145,8 +211,48 @@ jobs:
asset_path: ./defichain-${{ env.BUILD_VERSION }}-x86_64-pc-linux-gnu/defichain-${{ env.BUILD_VERSION }}-x86_64-pc-linux-gnu.tar.gz.SHA256
asset_name: defichain-${{ env.BUILD_VERSION }}-x86_64-pc-linux-gnu.tar.gz.SHA256
asset_content_type: text/plain

- name: Upload release asset - linux-armhf
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_by_tag.outputs.upload_url }}
asset_path: ./defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf/defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz
asset_name: defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz
asset_content_type: application/gzip

- name: Upload checksum - linux-armhf
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_by_tag.outputs.upload_url }}
asset_path: ./defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf/defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz.SHA256
asset_name: defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz.SHA256
asset_content_type: text/plain

- name: Upload release asset - linux-aarch64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_by_tag.outputs.upload_url }}
asset_path: ./defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu/defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu.tar.gz
asset_name: defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu.tar.gz
asset_content_type: application/gzip

- name: Upload checksum - linux-aarch64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_by_tag.outputs.upload_url }}
asset_path: ./defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu/defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu.tar.gz.SHA256
asset_name: defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu.tar.gz.SHA256
asset_content_type: text/plain

- name: Upload release asset - windows
- name: Upload release asset - win-x64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -156,7 +262,7 @@ jobs:
asset_name: defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32.zip
asset_content_type: application/zip

- name: Upload checksum asset - windows
- name: Upload checksum asset - win-x64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -166,7 +272,7 @@ jobs:
asset_name: defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32.zip.SHA256
asset_content_type: text/plain

- name: Upload release asset - macos
- name: Upload release asset - osx-x64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -176,7 +282,7 @@ jobs:
asset_name: defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin.tar.gz
asset_content_type: application/gzip

- name: Upload checksum asset - macos
- name: Upload checksum asset - osx-x64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -185,3 +291,23 @@ jobs:
asset_path: ./defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin/defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin.tar.gz.SHA256
asset_name: defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin.tar.gz.SHA256
asset_content_type: text/plain

- name: Upload release asset - osx-aarch64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_by_tag.outputs.upload_url }}
asset_path: ./defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin/defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin.tar.gz
asset_name: defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin.tar.gz
asset_content_type: application/gzip

- name: Upload checksum asset - osx-aarch64
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ steps.get_release_by_tag.outputs.upload_url }}
asset_path: ./defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin/defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin.tar.gz.SHA256
asset_name: defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin.tar.gz.SHA256
asset_content_type: text/plain
121 changes: 121 additions & 0 deletions .github/workflows/build-staging.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
name: Build - Staging

on:
workflow_dispatch:
release:
types: [created]

jobs:
linux-x64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 DOCKERFILE="x86_64-pc-linux-gnu-clang" TARGET="x86_64-pc-linux-gnu" ./make.sh docker-release

- name: Publish artifacts
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-x86_64-pc-linux-gnu
path: ./build/defichain-${{ env.BUILD_VERSION }}-x86_64-pc-linux-gnu.tar.gz

linux-armhf:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="arm-linux-gnueabihf" ./make.sh docker-release

- name: Publish artifact - arm-linux-gnueabihf
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf
path: ./build/defichain-${{ env.BUILD_VERSION }}-arm-linux-gnueabihf.tar.gz

linux-aarch64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="aarch64-linux-gnu" ./make.sh docker-release

- name: Publish artifact - aarch64-linux-gnu
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu
path: ./build/defichain-${{ env.BUILD_VERSION }}-aarch64-linux-gnu.tar.gz

win-x64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="x86_64-w64-mingw32" ./make.sh docker-release

- name: Publish artifact - x86_64-w64-mingw32
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32
path: ./build/defichain-${{ env.BUILD_VERSION }}-x86_64-w64-mingw32.tar.gz

osx-x64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="x86_64-apple-darwin" ./make.sh docker-release

- name: Publish artifact - x86_64-apple-darwin
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin
path: ./build/defichain-${{ env.BUILD_VERSION }}-x86_64-apple-darwin.tar.gz

osx-aarch64:
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')

steps:
- uses: actions/checkout@v3

- name: Populate environment
run: GIT_VERSION=1 ./make.sh ci-export-vars

- name: Build and package
run: GIT_VERSION=1 TARGET="aarch64-apple-darwin" ./make.sh docker-release

- name: Publish artifact - aarch64-apple-darwin
uses: actions/upload-artifact@v3
with:
name: defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin
path: ./build/defichain-${{ env.BUILD_VERSION }}-aarch64-apple-darwin.tar.gz
Loading

0 comments on commit a36f2a2

Please sign in to comment.