Skip to content

Commit

Permalink
feat/provider-oci: add full-stack creation
Browse files Browse the repository at this point in the history
  • Loading branch information
mtulio committed Feb 17, 2023
1 parent 837c59c commit b878cf5
Show file tree
Hide file tree
Showing 10 changed files with 698 additions and 170 deletions.
97 changes: 81 additions & 16 deletions docs/guides/installing-agnostic-oci.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
# Install OKD/OCP on OCI using agnostic method

> This document is under development.
Install OCP/OKD Cluster on Oracle Cloud Infrastructure using agnostic installation/UPI.

## Prerequisites


### Setup Ansible project

> This steps should be made only when OCI provider is under development - not merged to `main` branch. Then the normal install flow should be used.
- Setup your ansible workdir (optional, you can use the defaults)

```bash
Expand Down Expand Up @@ -41,7 +44,7 @@ wget https://raw.githubusercontent.com/mtulio/ansible-collection-okd-installer/m

- Update with OCI requirements

```
```bash
cat <<EOF >> requirements.txt
# Oracle Cloud Infrastructure
Expand All @@ -59,19 +62,21 @@ EOF

- Install ansible and dependencies

```
```bash
pip install -r requirements.txt
```

- Install the collections
- Install the Collections

```
```bash
ansible-galaxy collection install -r requirements.yml
```

- Get the latest (under development) okd-installer for OCI

```
> https://github.com/mtulio/ansible-collection-okd-installer/pull/26
```bash
git clone -b feat-add-provider-oci --recursive \
[email protected]:mtulio/ansible-collection-okd-installer.git \
collections/ansible_collections/mtulio/okd_installer
Expand All @@ -80,7 +85,7 @@ git clone -b feat-add-provider-oci --recursive \
- Check if the collection is present


```
```bash
$ ansible-galaxy collection list |egrep "(okd_installer|^oracle)"
mtulio.okd_installer 0.0.0-latest
oracle.oci 4.11.0
Expand Down Expand Up @@ -110,19 +115,19 @@ ansible localhost \

You must be able to collect the user information.

## okd-installer
## OCP Cluster Setup on OCI

### Generate the vars file

```bash
cat <<EOF > ~/.oci/env
OCI_COMPARTMENT_ID="<CHANGE_ME:ocid1.compartment.oc1.UUID>"
EOF

source ~/.oci/env

cat <<EOF > ./vars-oci-ha.yaml
provider: oci
cluster_name: ocp-oci
cluster_name: mrb
config_cluster_region: us-sanjose-1
oci_compartment_id: ${OCI_COMPARTMENT_ID}
Expand All @@ -144,7 +149,9 @@ compute_instance_spec:
cpu_count: 8
memory_gb: 16
#> TODO extract from stream file
# https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.12/builds/412.86.202212081411-0/aarch64/rhcos-412.86.202212081411-0-openstack.aarch64.qcow2.gz
# $ jq -r '.architectures["x86_64"].artifacts.openstack.formats["qcow2.gz"].disk.location' ~/.ansible/okd-installer/clusters/ocp-oci/coreos-stream.json`
custom_image_id: rhcos-412.86.202212081411-0-openstack.aarch64.qcow2.gz
EOF
```
Expand Down Expand Up @@ -192,14 +199,72 @@ ansible-playbook mtulio.okd_installer.stack_loadbalancer \
### Compute Stack
- Bootstrap
TODO
#### Bootstrap
- Mirror image (Ansible Role+Playbook Not implemented)
- Control Plane
> TODO: config to mirror from openstack image to OCI
TODO
> Currently the image is download manually, and added to the OCI Console as a image.
- Compute
TODO
Steps to mirror:
- Get the artifact URL from stream-json
- Create Bucket for images, if not exits
- Upload the image qcow2.gz
- Get the signed URL for the image object
- Create an image from signed URL
- Get the image ID, and set the global var `custom_image_id`
> `$ jq -r '.architectures["x86_64"].artifacts.openstack.formats["qcow2.gz"].disk.location' ~/.ansible/okd-installer/clusters/ocp-oci/coreos-stream.json`
```bash
os_mirror: yes
os_mirror_src: stream
os_mirror_stream:
architecture: x86_64
platform: openstack
format: qcow2.gz
os_mirror_dest_provider: oci
os_mirror_dest_oci:
compartment_id:
bucket:
```
- Upload the bootstrap ignition to blob and Create the Bootstrap Instance
```bash
ansible-playbook mtulio.okd_installer.create_node \
-e node_role=bootstrap \
-e @./vars-oci-ha.yaml
```
- Create the Control Plane nodes
```bash
ansible-playbook mtulio.okd_installer.create_node \
-e node_role=controlplane \
-e @./vars-oci-ha.yaml
```
- Create the Compute nodes
> TODO: create instance Pool
> TODO: Approve certificates (bash loop or use existing playbook)
```
oc adm certificate approve $(oc get csr -o json |jq -r '.items[] | select(.status.certificate == null).metadata.name')
```
## Review the cluster
```bash
export KUBECONFIG=${HOME}/.ansible/okd-installer/clusters/${cluster_name}/auth/kubeconfig
oc get nodes
oc get co
```
2 changes: 1 addition & 1 deletion playbooks/vars/oci/profiles/ha/dns.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ cloud_dns_zones:
# private
- name: "{{ cluster_state.dns.cluster_domain }}"
provider: oci
view_name: "{{ cluster_state.infra_id }}-vpc"
view_name: "{{ cluster_state.infra_id }}-vcn"
spec:
scope: PRIVATE
zone_type: PRIMARY
Expand Down
54 changes: 38 additions & 16 deletions playbooks/vars/oci/profiles/ha/loadbalancer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ cloud_load_balancer_provider: oci

# OCI NLB: https://docs.oracle.com/en-us/iaas/tools/oci-ansible-collection/4.12.0/collections/oracle/oci/oci_network_load_balancer_module.html#ansible-collections-oracle-oci-oci-network-load-balancer-module
cloud_loadbalancers:
- name: "{{ cluster_state.infra_id }}-ext"
- name: "{{ cluster_state.infra_id }}-nlb"
provider: oci
type: network

# Is it supported multi-subnets?
subnet_name: "{{ cluster_state.infra_id }}-net-public"
subnet_name: "{{ cluster_state.infra_id }}-net-public-lb"
spec:
compartment_id: "{{ oci_compartment_id }}"
display_name: "{{ cluster_state.infra_id }}-ext"
display_name: "{{ cluster_state.infra_id }}-nlb"
is_private: false
is_preserve_source_destination: true
nlb_ip_version: IPV4
Expand All @@ -44,10 +44,10 @@ cloud_loadbalancers:
backend_set:
- provider: oci
spec:
name: "{{ cluster_state.infra_id }}-aext"
name: "{{ cluster_state.infra_id }}-api"
is_preserve_source: no
ip_version: IPV4
#policy: TWO_TUPLE
policy: FIVE_TUPLE
#backends: []
health_checker:
port: 6443
Expand All @@ -59,10 +59,25 @@ cloud_loadbalancers:

- provider: oci
spec:
name: "{{ cluster_state.infra_id }}-ig-80"
name: "{{ cluster_state.infra_id }}-mcs"
is_preserve_source: no
ip_version: IPV4
#policy: TWO_TUPLE
policy: FIVE_TUPLE
#backends: []
health_checker:
port: 22623
protocol: HTTPS
return_code: 200
url_path: /healthz
interval_in_millis: 10000
timeout_in_millis: 3000

- provider: oci
spec:
name: "{{ cluster_state.infra_id }}-ing-http"
is_preserve_source: no
ip_version: IPV4
policy: FIVE_TUPLE
#backends: [] # TCP/31794
health_checker:
port: 31261
Expand All @@ -74,7 +89,7 @@ cloud_loadbalancers:

- provider: oci
spec:
name: "{{ cluster_state.infra_id }}-ig-443"
name: "{{ cluster_state.infra_id }}-ing-https"
is_preserve_source: no
ip_version: IPV4
#policy: TWO_TUPLE
Expand All @@ -90,22 +105,29 @@ cloud_loadbalancers:
# https://docs.oracle.com/en-us/iaas/tools/oci-ansible-collection/4.12.0/collections/oracle/oci/oci_network_load_balancer_listener_module.html#ansible-collections-oracle-oci-oci-network-load-balancer-listener-module
listeners:
- spec:
name: "{{ cluster_state.infra_id }}-aext"
default_backend_set_name: "{{ cluster_state.infra_id }}-aext"
name: "{{ cluster_state.infra_id }}-api"
default_backend_set_name: "{{ cluster_state.infra_id }}-api"
ip_version: IPV4
port: 6443
protocol: TCP

- spec:
name: "{{ cluster_state.infra_id }}-ig-80"
default_backend_set_name: "{{ cluster_state.infra_id }}-ig-80"
name: "{{ cluster_state.infra_id }}-mext"
default_backend_set_name: "{{ cluster_state.infra_id }}-mcs"
ip_version: IPV4
port: 22623
protocol: TCP

- spec:
name: "{{ cluster_state.infra_id }}-ing-http"
default_backend_set_name: "{{ cluster_state.infra_id }}-ing-http"
ip_version: IPV4
port: 80
protocol: TCP

- spec:
name: "{{ cluster_state.infra_id }}-ig-443"
default_backend_set_name: "{{ cluster_state.infra_id }}-ig-443"
name: "{{ cluster_state.infra_id }}-ing-https"
default_backend_set_name: "{{ cluster_state.infra_id }}-ing-https"
ip_version: IPV4
port: 443
protocol: TCP
Expand Down Expand Up @@ -138,7 +160,7 @@ cloud_loadbalancers:
# private address
- name: register_dns
rr_ip: private
view_name: "{{ cluster_state.infra_id }}-vpc"
view_name: "{{ cluster_state.infra_id }}-vcn"
spec:
zone_name_or_id: "{{ cluster_state.dns.cluster_domain }}"
compartment_id: "{{ oci_compartment_id }}"
Expand All @@ -150,7 +172,7 @@ cloud_loadbalancers:

- name: register_dns
rr_ip: public
view_name: "{{ cluster_state.infra_id }}-vpc"
view_name: "{{ cluster_state.infra_id }}-vcn"
spec:
zone_name_or_id: "{{ cluster_state.dns.cluster_domain }}"
compartment_id: "{{ oci_compartment_id }}"
Expand Down
Loading

0 comments on commit b878cf5

Please sign in to comment.