Skip to content

Commit

Permalink
patch: Support for building images on OpenStack
Browse files Browse the repository at this point in the history
* Support for building images directly in Openstack by utilising the Openstack packer builder.
* Supports Ubuntu and Flatcar (only tested on these two OS but could support others)
* New "page" in the book to reflect new process

Co-authored-by: yankcrime - Nick Jones
Co-authored-by: drew-viles - Drew Viles
Co-authored-by: okozachenko1203 - Oleksandr Kozachenko
Co-authored-by: mitch000001 - Michael Wagner
Co-authored-by: mboersma - Matt Boersma
  • Loading branch information
yankcrime authored and drew-viles committed Mar 22, 2023
1 parent 62265c5 commit eacb6bf
Show file tree
Hide file tree
Showing 13 changed files with 316 additions and 4 deletions.
3 changes: 3 additions & 0 deletions OWNERS_ALIASES
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ aliases:
- mboersma
image-builder-openstack-reviewers:
- hidekazuna
image-builder-openstack-maintainers
- drew-viles
- yankcrime
image-builder-cloudstack-reviewers:
- rohityadavcloud
- davidjumani
Expand Down
1 change: 1 addition & 0 deletions docs/book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
- [GCP](./capi/providers/gcp.md)
- [Nutanix](./capi/providers/nutanix.md)
- [OpenStack](./capi/providers/openstack.md)
- [OpenStack remote image building](./capi/providers/openstack-remote.md)
- [OCI](./capi/providers/oci.md)
- [raw](./capi/providers/raw.md)
- [vSphere](./capi/providers/vsphere.md)
Expand Down
1 change: 1 addition & 0 deletions docs/book/src/capi/capi.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ If any needed binaries are not present, they can be installed to `images/capi/.b
* [OCI](./providers/oci.md)
* [3DSOutscale](./providers/3dsoutscale.md)
* [OpenStack](./providers/openstack.md)
* [OpenStack remote image building](./providers/openstack-remote.md)
* [Raw](./providers/raw.md)
* [VirtualBox](./providers/virtualbox.md)
* [vSphere](./providers/vsphere.md)
Expand Down
2 changes: 1 addition & 1 deletion docs/book/src/capi/providers/oci.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,4 +129,4 @@ The following example JSON would use the [Windows Server 2019 Datacenter Edition
"subnet_ocid": "Fill Subnet OCID here",
"availability_domain": "Fill Availability Domain here"
}
```
```
75 changes: 75 additions & 0 deletions docs/book/src/capi/providers/openstack-remote.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Building Images on OpenStack

## Hypervisor

The image is built using OpenStack.

### Prerequisites for OpenStack builds

First, check for prerequisites at [Packer docs for the OpenStack builder](https://developer.hashicorp.com/packer/plugins/builders/openstack).

Also ensure that you have a [Ubuntu 20.04](https://cloud-images.ubuntu.com/focal/current/) or [Ubuntu 22.04](https://cloud-images.ubuntu.com/jammy/current/) cloud image available in your OpenStack instance before continuing as it will need to be referenced.
This build process also supports Flatcar Linux, but only Stable has been tested.

#### Note
> Other operating systems could be supported and additions are welcome.
## Setup Openstack authentication
Ensure you have set up your method of authentication. See the [examples here](https://docs.openstack.org/python-openstackclient/zed/cli/authentication.html).
You can also check out the [packer builder](https://developer.hashicorp.com/packer/plugins/builders/openstack#configuration-reference) for more information on authentication.

You should be able to run commands against OpenStack before running this builder, otherwise it will fail.
You can test with a simple command such as `openstack image list`.

## Building Images

The build [prerequisites](../capi.md#prerequisites) for using `image-builder` for
building OpenStack images are managed by running:

```bash
cd image-builder/images/capi
make deps-openstack
```

### Define variables for OpenStack build

Using the [Openstack packer provider](https://developer.hashicorp.com/packer/plugins/builders/openstack), an instance will be deployed and an image built from it.
A certain set of environment variables must be defined in a json file and referenced as shown below in the build example.

Replace UPPERCASE variables with your values.
```json
{
"source_image": "OPENSTACK_SOURCE_IMAGE_ID",
"networks": "OPENSTACK_NETWORK_ID",
"flavor": "OPENSTACK_INSTANCE_FLAVOR_NAME",
"floating_ip_network": "OPENSTACK_FLOATING_IP_NETWORK_NAME",
"image_name": "KUBE-UBUNTU",
"image_visibility": "public",
"image_disk_format": "raw",
"volume_type": "",
"ssh_username": "ubuntu"
}
```

Check out `images/capi/packer/openstack/packer.json` for more variables such as allowing the use of floating IPs and config drives.

### Building Image on OpenStack

From the `images/capi` directory, run `PACKER_VAR_FILES=var_file.json make build-openstack-<DISTRO>`.

An instance is built in OpenStack from the source image defined. Once completed, the instance is shut down and the image is created.
This image will default to private, however this can be controlled via `image_visibility`.

For building a ubuntu 22.04-based CAPI image with Kubernetes 1.23.15, run the following commands:

#### Example
```bash
$ git clone https://github.com/kubernetes-sigs/image-builder.git
$ cd image-builder/images/capi/
$ make deps-openstack
$ PACKER_VAR_FILES=var_file.json make build-openstack-ubuntu-2204
```

The resulting image will be named `ubuntu-2204-kube-v1.23.15` based on the following format: `<OS>-kube-<KUBERNETES_SEMVER>`.

This can be modified by overriding the `image_name` variable if required.
35 changes: 32 additions & 3 deletions images/capi/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ version: ## Display version of image-builder

.PHONY: deps
deps: ## Installs/checks all dependencies
deps: deps-ami deps-azure deps-do deps-gce deps-ova deps-qemu deps-raw deps-oci deps-osc deps-vbox deps-powervs deps-nutanix
deps: deps-ami deps-azure deps-do deps-gce deps-ova deps-openstack deps-qemu deps-raw deps-oci deps-osc deps-vbox deps-powervs deps-nutanix

.PHONY: deps-ami
deps-ami: ## Installs/checks dependencies for AMI builds
Expand Down Expand Up @@ -81,7 +81,7 @@ deps-osc:
hack/ensure-packer.sh
hack/ensure-goss.sh
packer plugins install github.com/outscale/outscale

.PHONY: deps-gce
deps-gce: ## Installs/checks dependencies for GCE builds
deps-gce:
Expand All @@ -98,6 +98,13 @@ deps-ova:
hack/ensure-goss.sh
hack/ensure-ovftool.sh

.PHONY: deps-openstack
deps-openstack: ## Installs/checks dependencies for OpenStack builds
deps-openstack:
hack/ensure-ansible.sh
hack/ensure-packer.sh
hack/ensure-goss.sh

.PHONY: deps-qemu
deps-qemu: ## Installs/checks dependencies for QEMU builds
deps-qemu:
Expand Down Expand Up @@ -327,6 +334,8 @@ OCI_BUILD_NAMES ?= oci-ubuntu-1804 oci-ubuntu-2004 oci-ubuntu-2204 oci-orac

DO_BUILD_NAMES ?= do-centos-7 do-ubuntu-1804 do-ubuntu-2004

OPENSTACK_BUILD_NAMES ?= openstack-ubuntu-2004 openstack-ubuntu-2204 openstack-flatcar

OSC_BUILD_NAMES ?= osc-ubuntu-2004

QEMU_BUILD_NAMES ?= qemu-ubuntu-1804 qemu-ubuntu-2004 qemu-ubuntu-2204 qemu-centos-7 qemu-ubuntu-2004-efi qemu-rhel-8 qemu-rockylinux-8 qemu-flatcar
Expand Down Expand Up @@ -361,6 +370,8 @@ AZURE_VALIDATE_SIG_TARGETS := $(addprefix validate-,$(AZURE_BUILD_SIG_NAMES))
AZURE_VALIDATE_SIG_GEN2_TARGETS := $(addprefix validate-,$(AZURE_BUILD_SIG_GEN2_NAMES))
DO_BUILD_TARGETS := $(addprefix build-,$(DO_BUILD_NAMES))
DO_VALIDATE_TARGETS := $(addprefix validate-,$(DO_BUILD_NAMES))
OPENSTACK_BUILD_TARGETS := $(addprefix build-,$(OPENSTACK_BUILD_NAMES))
OPENSTACK_VALIDATE_TARGETS := $(addprefix validate-,$(OPENSTACK_BUILD_NAMES))
QEMU_BUILD_TARGETS := $(addprefix build-,$(QEMU_BUILD_NAMES))
QEMU_VALIDATE_TARGETS := $(addprefix validate-,$(QEMU_BUILD_NAMES))
QEMU_KUBEVIRT_BUILD_TARGETS := $(addprefix build-,$(QEMU_KUBEVIRT_BUILD_NAMES))
Expand Down Expand Up @@ -460,6 +471,14 @@ $(DO_BUILD_TARGETS): deps-do
$(DO_VALIDATE_TARGETS): deps-do
packer validate $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/digitalocean/$(subst validate-do-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/digitalocean/packer.json

.PHONY: $(OPENSTACK_BUILD_TARGETS)
$(OPENSTACK_BUILD_TARGETS): deps-openstack
packer build $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/openstack/$(subst build-openstack-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/openstack/packer.json

.PHONY: $(OPENSTACK_VALIDATE_TARGETS)
$(OPENSTACK_VALIDATE_TARGETS): deps-openstack
packer validate $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/openstack/$(subst validate-openstack-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/openstack/packer.json

.PHONY: $(QEMU_BUILD_TARGETS)
$(QEMU_BUILD_TARGETS): deps-qemu
packer build $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/qemu/$(subst build-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/qemu/packer.json
Expand Down Expand Up @@ -495,7 +514,7 @@ $(OCI_VALIDATE_TARGETS): deps-oci
packer validate $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/oci/$(subst validate-oci-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/oci/packer.json

.PHONY: $(OSC_BUILD_TARGETS)
$(OSC_BUILD_TARGETS): deps-osc
$(OSC_BUILD_TARGETS): deps-osc
packer build $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/outscale/$(subst build-osc-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/outscale/packer.json

.PHONY: $(OSC_VALIDATE_TARGETS)
Expand Down Expand Up @@ -673,6 +692,11 @@ build-node-ova-local-base-rockylinux-8: ## Builds RockyLinux 8 Base Node OVA w l
build-node-ova-local-base-ubuntu-1804: ## Builds Ubuntu 18.04 Base Node OVA w local hypervisor
build-node-ova-local-base-ubuntu-2004: ## Builds Ubuntu 20.04 Base Node OVA w local hypervisor

build-openstack-ubuntu-2004: ## Builds Ubuntu 20.04 OpenStack image
build-openstack-ubuntu-2204: ## Builds Ubuntu 22.04 OpenStack image
build-openstack-flatcar: ## Builds Flatcar OpenStack image
build-openstack-all: $(OPENSTACK_BUILD_TARGETS)

build-qemu-flatcar: ## Builds Flatcar QEMU image
build-qemu-ubuntu-1804: ## Builds Ubuntu 18.04 QEMU image
build-qemu-ubuntu-2004: ## Builds Ubuntu 20.04 QEMU image
Expand Down Expand Up @@ -757,6 +781,11 @@ validate-do-ubuntu-2004: ## Validates Ubuntu 20.04 DigitalOcean Snapshot Packer
validate-do-centos-7: ## Validates Centos 7 DigitalOcean Snapshot Packer config
validate-do-all: $(DO_VALIDATE_TARGETS) ## Validates all DigitalOcean Snapshot Packer config

validate-openstack-ubuntu-2004: ## Validates Ubuntu 18.04 Openstack Image Packer config
validate-openstack-ubuntu-2204: ## Validates Ubuntu 20.04 Openstack Image Packer config
validate-openstack-flatcar: ## Validates Flatcar Openstack Image Packer config
validate-openstack-all: $(OPENSTACK_VALIDATE_TARGETS) ## Validates all Openstack Glance Image Packer config

validate-gce-ubuntu-1804: ## Validates Ubuntu 18.04 GCE Snapshot Packer config
validate-gce-ubuntu-2004: ## Validates Ubuntu 20.04 GCE Snapshot Packer config
validate-gce-ubuntu-2204: ## Validates Ubuntu 22.04 GCE Snapshot Packer config
Expand Down
3 changes: 3 additions & 0 deletions images/capi/ansible/roles/providers/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@
- include_tasks: googlecompute.yml
when: packer_builder_type.startswith('googlecompute')

- include_tasks: openstack.yml
when: packer_builder_type.startswith('openstack')

- include_tasks: oci.yml
when: packer_builder_type.startswith('oracle-oci')

Expand Down
33 changes: 33 additions & 0 deletions images/capi/ansible/roles/providers/tasks/openstack.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Copyright 2020 The Kubernetes Authors.

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at

# http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---
- name: Install cloud-init packages
apt:
name: "{{ packages }}"
state: present
force_apt_get: yes
vars:
packages:
- cloud-init
- cloud-guest-utils
- cloud-initramfs-copymods
- cloud-initramfs-dyn-netconf
when: ansible_os_family == "Debian"

- name: Disable Hyper-V KVP protocol daemon on Ubuntu
systemd:
name: hv-kvp-daemon
state: stopped
enabled: false
when: ansible_os_family == "Debian"
9 changes: 9 additions & 0 deletions images/capi/packer/openstack/OWNERS
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# See the OWNERS docs at https://go.k8s.io/owners

approvers:
- cluster-api-openstack-maintainers

reviewers:
- cluster-api-openstack-maintainers
- image-builder-openstack-reviewers
- image-builder-openstack-maintainers
12 changes: 12 additions & 0 deletions images/capi/packer/openstack/flatcar.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"ansible_extra_vars": "ansible_python_interpreter=/opt/bin/python",
"build_name": "flatcar",
"crictl_source_type": "http",
"distro_name": "flatcar",
"kubernetes_cni_source_type": "http",
"kubernetes_source_type": "http",
"source_image": "flatcar",
"systemd_prefix": "/etc/systemd",
"sysusr_prefix": "/opt",
"sysusrlocal_prefix": "/opt"
}
Loading

0 comments on commit eacb6bf

Please sign in to comment.