BuildBot

Using BuildBot can be useful to automate OpenXT builds.

Let's assume you're running a 32bits installation of Debian Squeeze, which is the recommended configuration for building OpenXT.
Installing BuildBot is as easy as "apt-get install buildbot". The Squeeze version is quite outdated, but it does the job.

BuildBot works in 2 parts, the slave(s) that build, and the master that control the slave(s).
In the following, I'm describing a setup with just one master and one slave, both on the same machine.
Disclaimer: everything here is done in a pretty dummy way, there's a lot of room for improvement.

Creating the master and the slave

The configuration happens in /var/lib/buildbot for the master/slave configuration, and in /etc/default/buildbot for the deamon configuration.
To get started, "cd /var/lib/buildbot" as root (or as buildbot, after enabling its shell), and create a master and a slave:

# buildbot create-master master
# cd master
# mv master.cfg.sample master.cfg
# cd ..
# buildbot create-slave slave 127.0.0.1:9989 slave password
# cd ../master

Configuring the master

master.cfg is the main BuildBot file. That's where you define projects, steps and schedulers. Here's some modifications I made to it:

• Renaming the "BuildSlave" from "but1name" to "slave", and changing the password from "bot1passwd" to "password".

• Disable the scheduler by commenting out the c['schedulers'].append line(s).

• Fix the factory steps (the scripts are detailed in the "Creating the build scripts" section):

    f1.addStep(Compile(command=["bash", "-e", "./fetch.sh"], name="Fetch", description="Fetch", descriptionDone="Fetched"))
    f1.addStep(Compile(command=["bash", "-e", "./clean.sh"], name="Clean", description="Clean", descriptionDone="Cleaned"))
    f1.addStep(Compile(command=["bash", "-e", "./build.sh"], name="Build", description="Build", descriptionDone="Built"))
    f1.addStep(Compile(command=["bash", "-e", "./copy.sh"],  name="Copy",  description="Copy",  descriptionDone="Copied"))
  

• In b1, the builder, fix the name ("openxt") the slavename ("slave"), and the builddir ("/home/buildbot")

• Set allowForce=true, as described in a comment

• Fix the project name and URL

Configuring the initscript

Below is the content of my /etc/default/buildbot

# buildbots to manage
# add a new set of variables for each buildbot to start

BB_NUMBER[0]=0			# index for the other values; negative disables the bot
BB_NAME[0]="master"		# short name printed on startup / stop
BB_USER[0]="buildbot"		# user to run as
BB_BASEDIR[0]="/var/lib/buildbot/master"		# basedir argument to buildbot (absolute path)
BB_OPTIONS[0]=""		# buildbot options
BB_PREFIXCMD[0]=""		# prefix command, i.e. nice, linux32, dchroot

BB_NUMBER[1]=1                  # index for the other values; negative disables the bot
BB_NAME[1]="slave"              # short name printed on startup / stop
BB_USER[1]="buildbot"           # user to run as
BB_BASEDIR[1]="/var/lib/buildbot/slave"                # basedir argument to buildbot (absolute path)
BB_OPTIONS[1]=""                # buildbot options
BB_PREFIXCMD[1]=""              # prefix command, i.e. nice, linux32, dchroot

Creating the build scripts

In /home/buildbot/build, I created the scripts corresponding to the steps added to the factory:
clean.sh:

#!/bin/bash

umask 0022
rm -rf build
mkdir build

That removes the whole build tree and create an empty folder for the new build.
If the previous build finished, or at least lasted for a while, this step could take more than 15 minutes...

fetch.sh:

#!/bin/bash

for i in git/*.git; do
    echo "Fetching `basename $i`..."
    cd $i
    git fetch --all
    cd - > /dev/null
done

This fetches all the repository for the local git mirror.
The fetch should take less than 10 seconds, hopefully not enough time for somebody to push half a feature!

build.sh:

#!/bin/bash -ex

umask 0022
cd build
git clone file:///home/buildbot/build/git/openxt.git
cd openxt
cp -r ../../certs .
cp example-config .config
cat <<EOF >> .config
OPENXT_GIT_MIRROR="file:///home/buildbot/build/git"
REPO_PROD_CACERT="/home/buildbot/build/certs/prod-cacert.pem"
REPO_DEV_CACERT="/home/buildbot/build/certs/dev-cacert.pem"
REPO_DEV_SIGNING_CERT="/home/buildbot/build/certs/dev-cacert.pem"
REPO_DEV_SIGNING_KEY="/home/buildbot/build/certs/dev-cakey.pem"
EOF
./do_build.sh | tee build.log
ret=${PIPESTATUS[0]}
cd -
cd -

exit $ret

This step clones the main openxt repo and overwrites some config bits to match the setup.
This assumes that a valid set of certificates has been created in /home/buildbot/build/certs

copy.sh:

#!/bin/bash

umask 0022
cp build/openxt/build-output/openxt-dev--master/iso/installer.iso /var/www/openxt/installer.iso

This steps copies the resulting OpenXT iso to the local web server, assuming that buildbot has write access to /var/www/openxt

Mirroring the repositories

# cd /home/buildbot/build
# mkdir git
# for i in blktap.git bootage.git dm-agent.git dm-wrapper.git fbtap.git gene3fs.git icbinn.git idl.git input.git installer.git ioemu.git ioemu-pq.git ipxe-pq.git libedid.git libpciemu.git libxcdbus.git libxenbackend.git linux-3.11-pq.git manager.git meta-selinux.git msi-installer.git network.git ocaml.git openxt.git polmod-example.git pv-linux-drivers.git qemu-dm.git qemu-dm-pq.git refpolicy-xt-pq.git resized.git sdk.git seabios.git seabios-pq.git selinux-policy.git surfman.git sync-client.git sync-cli.git sync-database.git sync-server.git sync-ui-helper.git sync-wui.git toolstack-data.git toolstack.git uid.git v4v.git win-tools.git xblanker.git xclibs.git xctools.git xc-vusb-daemon.git xc-vusb.git xc-windows.git xenaccess.git xenaccess-pq.git xenclient-oe-extra.git xenclient-oe.git xen-common-pq.git xenfb2.git xsm-policy.git; do git clone --mirror git://github.com/OpenXT/$i git/$i; done
# chown -R buildbot *

Starting and testing

# /etc/init.d/buildbot stop
# /etc/init.d/buildbot start

Then open http://127.0.0.1:8010 in a web browser.
As the scheduler is disabled, the only way to start a build is to force it, in the builder page.