Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add external Power VS cloud provider support #614

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
- [Power VS Cluster](./topics/powervs/index.md)
- [Prerequisites](./topics/powervs/prerequisites.md)
- [Creating a cluster](./topics/powervs/creating-a-cluster.md)
- [Creating a cluster with External Cloud Provider](./topics/powervs/external-cloud-provider.md)
- [Developer Guide](./developer/index.md)
- [Rapid iterative development with Tilt](./developer/tilt.md)
- [Guide for API conversions](./developer/conversion.md)
Expand Down
2 changes: 2 additions & 0 deletions docs/book/src/topics/powervs/creating-a-cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ following the steps below.

2. Use clusterctl to render the yaml through templates and deploy the cluster

**Note:** To deploy workload cluster with Power VS cloud controller manager which is currently in experimental stage follow [these](/topics/powervs/external-cloud-provider.html) steps.

**Note:** the `IBMPOWERVS_IMAGE_ID` value below should reflect the ID of the custom qcow2 image, the `kubernetes-version` value below should reflect the kubernetes version of the custom qcow2 image.

```console
Expand Down
32 changes: 32 additions & 0 deletions docs/book/src/topics/powervs/external-cloud-provider.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# IBM Power VS External Cloud Provider
Karthik-K-N marked this conversation as resolved.
Show resolved Hide resolved
## This feature currently in experimental stage

## Steps

- To deploy a Power VS workload cluster with IBM Power VS external [cloud provider](https://kubernetes.io/docs/concepts/architecture/cloud-controller/), create a cluster configuration with the [external cloud provider template](https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/blob/main/templates/cluster-template-powervs-cloud-provider.yaml)
- The [external cloud provider template](https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/blob/main/templates/cluster-template-powervs-cloud-provider.yaml) will use [clusterresourceset](https://cluster-api.sigs.k8s.io/tasks/experimental-features/cluster-resource-set.html) and will create the necessary config map, secret and roles to run the cloud controller manager
- As a prerequisite set the `powervs-provider-id-fmt` [flag](https://github.com/kubernetes-sigs/cluster-api-provider-ibmcloud/blob/64c9e1d17f1733c721f45a559edba3f4b712bcb0/main.go#L220) with value v2

### Deploy Power VS cluster with IBM Power VS cloud provider

```
IBMPOWERVS_SSHKEY_NAME="my-pub-key" \
IBMPOWERVS_VIP="192.168.151.22" \
IBMPOWERVS_VIP_EXTERNAL="158.175.162.22" \
IBMPOWERVS_VIP_CIDR="29" \
IBMPOWERVS_IMAGE_NAME="capibm-powervs-centos-8-1-22-4" \
IBMPOWERVS_SERVICE_INSTANCE_ID="7845d372-d4e1-46b8-91fc-41051c984601" \
IBMPOWERVS_NETWORK_NAME="capi-test-3" \
IBMACCOUNT_ID="ibm-accountid" \
IBMPOWERVS_REGION="powervs-region" \
IBMPOWERVS_ZONE="powervs-zone" \
BASE64_API_KEY=$(echo -n $IBMCLOUD_API_KEY | base64) \
clusterctl generate cluster ibm-powervs-1 --kubernetes-version v1.22.4 \
--target-namespace default \
--control-plane-machine-count=3 \
--worker-machine-count=1 \
--from ./cluster-template-powervs-cloud-provider.yaml | kubectl apply -f -
```

When the cluster is created with above parameters, The IBM Power VS cloud provider will
1. Initialize the node by fetching appropriate VM information such as IP, zone, region from Power Cloud.
44 changes: 44 additions & 0 deletions hack/ccm/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# syntax=docker/dockerfile:1.1-experimental

# Copyright 2022 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

ARG GOLANG_IMAGE=golang:1.17
ARG TARGETPLATFORM=linux/amd64
ARG ARCH=amd64

# Build vpcctl binary
FROM ${GOLANG_IMAGE} as vpc-builder
ARG ARCH
ARG VPC_CONTROLLER_COMMIT
WORKDIR /build
RUN git clone https://github.com/openshift/cloud-provider-vpc-controller
Karthik-K-N marked this conversation as resolved.
Show resolved Hide resolved
RUN cd cloud-provider-vpc-controller/cmd && git checkout $VPC_CONTROLLER_COMMIT && CGO_ENABLED=0 GOARCH=$ARCH go build \
-ldflags "-s -w" -o /build/vpcctl .

# Build IBM cloud controller manager binary
FROM ${GOLANG_IMAGE} AS ccm-builder
ARG ARCH
ARG POWERVS_CLOUD_CONTROLLER_COMMIT
WORKDIR /build
RUN git clone https://github.com/openshift/cloud-provider-powervs
Karthik-K-N marked this conversation as resolved.
Show resolved Hide resolved
RUN cd cloud-provider-powervs && git checkout $POWERVS_CLOUD_CONTROLLER_COMMIT && CGO_ENABLED=0 GOARCH=$ARCH go build \
-ldflags "-s -w" -o /build/ibm-cloud-controller-manager .

# Assemble the final image
FROM --platform=$TARGETPLATFORM quay.io/centos/centos:stream8 AS centos-base
LABEL description="IBM PowerVS Cloud Controller Manager"
COPY --from=vpc-builder /build/vpcctl /bin/vpcctl
COPY --from=ccm-builder /build/ibm-cloud-controller-manager /bin/ibm-cloud-controller-manager
ENTRYPOINT [ "/bin/ibm-cloud-controller-manager" ]
54 changes: 54 additions & 0 deletions hack/ccm/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Copyright 2022 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

REGISTRY=gcr.io/k8s-staging-capi-ibmcloud
IMG=powervs-cloud-controller-manager

# VPC_CONTROLLER_COMMIT can be fetched from here https://github.com/openshift/cloud-provider-vpc-controller/commits/master
VPC_CONTROLLER_COMMIT?=9b99b4e
# POWERVS_CLOUD_CONTROLLER_COMMIT can be fetched from here https://github.com/openshift/cloud-provider-powervs/commits/main
POWERVS_CLOUD_CONTROLLER_COMMIT?=a6bfa07
TAG?=$(VPC_CONTROLLER_COMMIT)_$(POWERVS_CLOUD_CONTROLLER_COMMIT)

build-image-and-push-linux-amd64: init-buildx
{ \
set -e ; \
docker buildx build \
--build-arg TARGETPLATFORM=linux/amd64 --build-arg ARCH=amd64 \
--build-arg VPC_CONTROLLER_COMMIT=$(VPC_CONTROLLER_COMMIT) --build-arg POWERVS_CLOUD_CONTROLLER_COMMIT=$(POWERVS_CLOUD_CONTROLLER_COMMIT)\
-t $(REGISTRY)/$(IMG):$(TAG)_linux_amd64 . --push --target centos-base; \
}

build-image-and-push-linux-ppc64le: init-buildx
{ \
set -e ; \
docker buildx build \
--build-arg TARGETPLATFORM=linux/ppc64le --build-arg ARCH=ppc64le\
--build-arg VPC_CONTROLLER_COMMIT=$(VPC_CONTROLLER_COMMIT) --build-arg POWERVS_CLOUD_CONTROLLER_COMMIT=$(POWERVS_CLOUD_CONTROLLER_COMMIT)\
-t $(REGISTRY)/$(IMG):$(TAG)_linux_ppc64le . --push --target centos-base; \
}

init-buildx:
# Ensure we use a builder that can leverage it (the default on linux will not)
docker buildx rm multiarch-multiplatform-builder
docker buildx create --use --name=multiarch-multiplatform-builder
docker run --rm --privileged multiarch/qemu-user-static --reset --credential yes --persistent yes
# Register gcloud as a Docker credential helper.
# Required for "docker buildx build --push".
gcloud auth configure-docker --quiet


build-and-push-multi-arch: build-image-and-push-linux-amd64 build-image-and-push-linux-ppc64le
docker manifest create --amend $(REGISTRY)/$(IMG):$(TAG) $(REGISTRY)/$(IMG):$(TAG)_linux_amd64 $(REGISTRY)/$(IMG):$(TAG)_linux_ppc64le
docker manifest push -p $(REGISTRY)/$(IMG):$(TAG)
18 changes: 18 additions & 0 deletions hack/ccm/cloudbuild.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# See https://cloud.google.com/cloud-build/docs/build-config
timeout: 3000s
options:
substitution_option: ALLOW_LOOSE
steps:
- name: 'gcr.io/k8s-testimages/gcb-docker-gcloud:v20210722-085d930'
dir: 'hack/ccm'
entrypoint: make
env:
- PULL_BASE_REF=${_PULL_BASE_REF}
- HOME=/root
args:
- build-and-push-multi-arch
substitutions:
# _GIT_TAG will be filled with a git-based tag for the image, of the form vYYYYMMDD-hash, and
# can be used as a substitution
_GIT_TAG: '12345'
_PULL_BASE_REF: 'dev'
Loading