Skip to content

Gitlab CI

Jump to bottom
Andrey Penechko edited this page Sep 10, 2019 · 3 revisions

This guide currently covers Continuous Integration with Gitlab. Feel free to add your own methods or improvements as you see fit.

  • gitlab-runner

Gitlab

It is possible to achieve continous integration using gitlab-runner. It is recommended to install gitlab-runner yourself so that you may directly install compilers and other libraries to the system and not have to include installing these dependencies as part of the CI process.

Local Dependencies

If your project relies on any local dependencies such as dplug-aax or your own libraries, then you will need to add them to the project as git submodules (this requires that they are git repositories on gitlab). Add each git module using

git submodule add [email protected]:yourusername/yourproject.git

You should have a file in your repo called .gitmodules. Edit this file and change the url property to use a path instead of a url.

[submodule "dplug-aax"]
	path = dplug-aax
	url = ../../yourusername/dplug-aax.git

This will result in having the submodules cloned in the root directory of your project so you will want to add the dependency as such.

"dependencies":
{
    "dplug-aax": { "path": "./dplug-aax" }
}

To tell gitlab-runner to clone the submodules you will need to add the following to your .gitlab-ci.yml file.

variables:
  GIT_SUBMODULE_STRATEGY: recursive

gitlab-runner setup

Follow the guide for installing gitlab-runner on each platform that you wish to automate builds for. During the setup process, choose shell for the executor. While it is not recommended on the gitlab wiki, for Windows it may be necessary to install the gitlab-runner service in user mode rather than system mode. In system mode, it appears that the repos are cloned into C:\Windows\System32 and dub throws an error due to not having permissions to read dub.json in the directory.

You will need to install compilers and other dependencies as you normally would for your development machine.

Here is an example gitlab-ci.yml script that uses tags to determine which runner to use for each build. It assumes that dplug-build is installed on each runner as well.

variables:
  GIT_SUBMODULE_STRATEGY: recursive

stages:
  - build

build windows:
  stage: build
  tags:
    - win
  script:
    - dplug-build -c VST -c VST3 -b release-nobounds --compiler ldc2

build linux:
  stage: build
  tags:
    - linux
  script:
    - dplug-build -c VST -c VST3 -c LV2 -b release-nobounds --compiler dmd

build MacOS:
  stage: build
  tags:
    - macOS
  script:
    - dplug-build -c VST -c VST3 -c AU -b release-nobounds --compiler ldc2

Cross-platform using KVM on linux

Requirements

  • A linux distribution capable of running KVM
  • VT-x or amd-v enabled bios
  • (optional) VT-d is required for building AAX on MacOS
  • (optional) Atleast 2 USB pci busses for AAX
  • (optional) 2 ILok keys for simultaneous building of Windows and MacOS AAX plugins

MacOS

The legality of this is method is a little fuzzy since, running MacOS on non-apple hardware violates their licenses. If you decide to use this method please be aware of this.

Once you have a linux distribution setup with KVM installed, you will be able to set up a MacOS virtual machine. Luckily there is a great open source project availabe to make this process extremely simple: OSX-KVM.

AAX Requirements

AAX complicates things more since it relies on having an ILok device on the machine. Luckily, it can be done using USB passthrough. After following the guide for installing OSX-KVM, see their notes about achieving USB passthrough. You will need to use vfio-pci to pass through the PCI bus that contains your ILok device. After you have managed to pass through the PCI bus to your MacOS, you can install ILok License Manager, and use your ILok as you normally would.

note that this will cause the entire PCI bus to be "owned" by the VM so you will no longer be able to use any of the USB devices attached to it on the host machine

Windows

Windows is fairly easy to set up. You will need to obtain a Windows installer ISO and use Virtual Machine Manager to create a new virtual machine.

There is an excellent guide here on how to install a Windows 10 VM on Ubuntu 18.04.

AAX Requirements

Fortunately USB passthrough for windows using Virtual Machine Manager is much simpler than the MacOS method. Open the settings for the Windows VM, and choose to add a USB Device, and select your ILok from the list. Not that you will want to make sure it is on a different PCI bus than the one being used for MacOS, otherwise you will not be able to build AAX on both VMs simultaneously.

Using Gitlab shared runner

Here is an example that uses ldc docker image and sets up dplug-build tool on gitlab shared linux runner.

variables:
  GIT_SUBMODULE_STRATEGY: recursive

stages:
  - build

build linux:
  image: dlang2/ldc-ubuntu
  stage: build
  tags:
    - linux
  script:
    - apt-get update && apt-get -y upgrade && apt-get install -y git libx11-dev
    - git clone --depth 1 https://github.com/AuburnSounds/Dplug ../dplug
    - cd ../dplug/tools/dplug-build
    - dub build
    - ln -sf $(pwd)/dplug-build /usr/bin/dplug-build 
    - cd $CI_PROJECT_DIR
    - export VST2_SDK="$CI_PROJECT_DIR/VST2_SDK"
    - dplug-build -c VST -c VST3 -c LV2 --installer --final
Clone this wiki locally