[buildomat] Package all global zone tarballs into single tarball

[smklein]

My expectation is that we'll:

1. Download global-zone-packages.tar.gz in the helios repo
2. Unpack the contents into a staging directory
3. Use image builder to copy the contents of these services into the host OS image

Additionally, I made the list of artifacts a bit more explicit. Eventually we could parse the package manifest to access this info, but for now, I don't want to include any "intermediate" artifacts that shouldn't be installed on a system.

[iliana]

👀

[smklein]

See: https://github.com/oxidecomputer/buildomat#configuration , under publish specifically

I think this'll be necessary for the helios repo to be able to "pick up these output artifacts" by a particular commit of Omicron

[smklein]

I'm doing that work in oxidecomputer/helios#56

[jclulow]

I think the artefact we want to produce here actually looks a lot more like a layer archive for a zone image; i.e., it would contain an oxide.json metadata file and then a set of files and directories under root/ that should be unpacked as-is into the OS ramdisk image:

$ tar tfz omicron-gz.tar.gz
oxide.json
root/
root/opt/
root/opt/oxide/
root/opt/oxide/sled-agent/
root/opt/oxide/sled-agent/pkg/
root/opt/oxide/sled-agent/pkg/config-rss.toml
root/opt/oxide/sled-agent/pkg/config.toml
root/opt/oxide/sled-agent/sled-agent
root/opt/oxide/mg-ddm/
root/opt/oxide/mg-ddm/pkg/
root/opt/oxide/mg-ddm/pkg/ddm/
root/opt/oxide/mg-ddm/pkg/ddm/manifest.xml
root/opt/oxide/mg-ddm/pkg/ddm_method_script.sh
root/opt/oxide/mg-ddm/ddmd
root/opt/oxide/mg-ddm/ddmadm
root/lib/
root/lib/svc/
root/lib/svc/manifest/
root/lib/svc/manifest/site/
root/lib/svc/manifest/site/sled-agent.xml

I used this script to produce the archive above, based on CI bits from the latest omicron at time of writing:

#!/bin/bash

set -o errexit
set -o pipefail
set -o xtrace

out="$PWD/omicron-gz.tar.gz"

#
# Create a temporary directory in which to work:
#
if ! dir=$(mktemp -d); then
        exit 1
fi
trap 'cd /; rm -rf "$dir"' EXIT

#
# Grab package.tar.gz from a "helios / package" buildomat job in the omicron
# repository:
#
url='https://buildomat.eng.oxide.computer/wg/0/artefact'
url+='/01GTMSQ3YXVEERT7507JD5JHMR'
url+='/PC4GK7nwpbMpBW38aIYmpuxGZyGzLismlZVfoDYO5gt8FdTq'
url+='/01GTMSQFKEV7RCX1GJPR4TKFFW'
url+='/01GTMV6X38PK4DFWDKVMMAJCAS'
url+='/package.tar.gz'
curl -Lo "$dir/package.tar.gz" "$url"

#
# Unpack the archive so that we can pick out the bits we need:
#
mkdir -p "$dir/extract"
cd "$dir/extract"
tar xvfz "$dir/package.tar.gz"

mkdir -p "$dir/root/opt/oxide/sled-agent"
cd "$dir/root/opt/oxide/sled-agent"
tar xvfz "$dir/extract/out/omicron-sled-agent.tar"
mkdir -p "$dir/root/lib/svc/manifest/site"
mv pkg/manifest.xml "$dir/root/lib/svc/manifest/site/sled-agent.xml"

mkdir -p "$dir/root/opt/oxide/mg-ddm"
cd "$dir/root/opt/oxide/mg-ddm"
tar xvfz "$dir/extract/out/maghemite.tar"

echo '{"v":"1","t":"layer"}' > "$dir/oxide.json"
cd "$dir" && tar cvfz "$out" oxide.json root

Then helios.git can just unpack this one published artefact into a proto directory and we can layer it on as the final step in packing up the ramdisk image without needing to move things around, etc.

[smklein]

I think the artefact we want to produce here actually looks a lot more like a layer archive for a zone image; i.e., it would contain an oxide.json metadata file and then a set of files and directories under root/ that should be unpacked as-is into the OS ramdisk image:

Sure thing, updated package.sh to use this approach.