Update Ceph docs for OpenStackDataPlaneNodeSet

The OpenStackDataPlaneNodeSet CRD has been introduced to the data plane operator so the Ceph documentation needs to be updated to work with the new CRD. Also, ceph-hci-pre needs to come before configure-os, as the latter is what actually applies the firewall rules that the former generates. Signed-off-by: John Fulton <[email protected]>
stuggi · Sep 20, 2023 · fd40530 · fd40530
1 parent d93060b
commit fd40530
Show file tree

Hide file tree

Showing 3 changed files with 96 additions and 102 deletions.
diff --git a/ceph.md b/ceph.md
@@ -161,24 +161,23 @@ spec:
             mountPath: "/etc/ceph"
             readOnly: true
 ```
-The `OpenStackDataPlane` can also use `extraMounts`.
+The `OpenStackDataPlaneNodeSet` can also use `extraMounts`.
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
-  roles:
-    edpm-compute:
-      nodeTemplate:
-        extraMounts:
-        - extraVolType: Ceph
-          volumes:
-          - name: ceph
-            secret:
-              secretName: ceph-conf-files
-          mounts:
-          - name: ceph
-            mountPath: "/etc/ceph"
-            readOnly: true
+  ...
+  nodeTemplate:
+    extraMounts:
+    - extraVolType: Ceph
+      volumes:
+      - name: ceph
+        secret:
+          secretName: ceph-conf-files
+      mounts:
+      - name: ceph
+        mountPath: "/etc/ceph"
+        readOnly: true
 ```
 When a CR containing the above is created, an Ansible pod
 running on OpenShift mounts the files in the Ceph secret
@@ -307,14 +306,14 @@ overwrite any custom service with the same name during reconciliation.
 
 After the `ConfigMap` and `OpenStackDataPlaneService` services above
 have been created (e.g. `oc create -f ceph-nova.yaml`), update the
-`OpenStackDataPlane`
+`OpenStackDataPlaneNodeSet`
 [EDPM services list](https://openstack-k8s-operators.github.io/dataplane-operator/composable_services)
 to replace the `nova` service with `nova-custom-ceph` and add the
 `ceph-client` service.
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
   roles:
@@ -495,10 +494,10 @@ spec:
 ## Full Examples
 
 The examples above are focussed on showing how a
-single `OpenStackControlPlane` and `OpenStackDataPlane`
+single `OpenStackControlPlane` and `OpenStackDataPlaneNodeSet`
 CR can be modified to include Ceph configuration by adding
 `extraMounts` and `customServiceConfig`. Links to complete
 examples are below.
 
 - `OpenStackControlPlane`: [core_v1beta1_openstackcontrolplane_network_isolation_ceph.yaml](https://github.com/openstack-k8s-operators/openstack-operator/blob/main/config/samples/core_v1beta1_openstackcontrolplane_network_isolation_ceph.yaml)
-- `OpenStackDataPlane`: [dataplane_v1beta1_openstackdataplane_ceph.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplane_ceph.yaml)
+- `OpenStackDataPlaneNodeSet`: [dataplane_v1beta1_openstackdataplanenodeset_ceph.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplanenodeset_ceph.yaml)
diff --git a/edpm_service_overview.md b/edpm_service_overview.md
@@ -71,9 +71,9 @@ ports for the Ceph Monitoring service.
     dport: [6789, 3300]
 ```
 
-When the `run-os`
-[composable service](https://openstack-k8s-operators.github.io/dataplane-operator/composable_services/)
-runs, it executes the role
+When the `configure-os` and `run-os`
+[composable services](https://openstack-k8s-operators.github.io/dataplane-operator/composable_services/)
+run, they execute the role
 [edpm_nftables](https://github.com/openstack-k8s-operators/edpm-ansible/tree/main/roles/edpm_nftables)
 This role reads files in `/var/lib/edpm-config/firewall/`
 and creates a `edpm-rules.nft` file in `/etc/nftables/` and then
@@ -94,11 +94,11 @@ which results in the following output from the NFT command.
 		tcp dport { 3300, 6789 } ct state new counter packets 0 bytes 0 accept comment "110 allow ceph_mon"
 [root@edpm-compute-0 ~]# 
 ```
-If the service needs to be deployed after the `run-os` service has
-run, then the Ansible for that service can directly call the
-`edpm_nftables` role to update the files in `/etc/nftables` and
-reload the rules. An example of this from the `edpm_libvirt` role
-is below.
+If the service needs to be deployed after the `configure-os` and
+`run-os` services have run, then the Ansible for that service can
+directly call the `edpm_nftables` role to update the files in
+`/etc/nftables` and reload the rules. An example of this from the
+`edpm_libvirt` role is below.
 
 ```yaml
 - name: Copy qemu vnc firewall config

diff --git a/hci.md b/hci.md
@@ -11,34 +11,30 @@ deployments are also known as Hyperconverged Infrastructure (HCI).
 3. Configure OpenStack to use the collocated Ceph server
 
 In order to complete the above procedure, the `services` list of the
-`OpenStackDataPlane` CR needs to be edited.
+`OpenStackDataPlaneNodeSet` CR needs to be edited.
 
-## OpenStackDataPlane services list
+## OpenStackDataPlaneNodeSet services list
 
-EDPM nodes can be configured by creating an `OpenStackDataPlane` CRs
-(or `OpenStackDataPlaneRole` and `OpenStackDataPlaneNode` CRs)
-which the
+EDPM nodes can be configured by creating an
+`OpenStackDataPlaneNodeSet` CR which the
 [dataplane-operator](https://openstack-k8s-operators.github.io/dataplane-operator)
-will reconcile. These types of CRs have a `services` list like the
-following:
+will reconcile when an `OpenStackDataPlaneDeployment` CR is created.
+These types of CRs have a `services` list like the following:
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
-  roles:
-    edpm-compute:
-      ...
-      services:
-        - configure-network
-        - validate-network
-        - install-os
-        - configure-os
-        - run-os
-        - ovn
-        - libvirt
-        - nova
+  services:
+    - configure-network
+    - validate-network
+    - install-os
+    - configure-os
+    - run-os
+    - ovn
+    - libvirt
+    - nova
 ```
 Only the services which are on the list will be configured.
 
@@ -60,35 +56,33 @@ This example also assumes that the EDPM nodes:
 - Are at least three in number (Ceph clusters must have at least three
   nodes for redundancy)
 
-Create an `OpenStackDataPlane` CR file, e.g. `dataplane_cr.yaml` to
-represent the EDPM nodes. See
-[dataplane_v1beta1_openstackdataplane.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplane.yaml)
-for an example which you will modify as described in this document.
+Create an `OpenStackDataPlaneNodeSet` CR file,
+e.g. `dataplane_cr.yaml` to represent the EDPM nodes. See
+[dataplane_v1beta1_openstackdataplanenodeset.yaml](https://github.com/openstack-k8s-operators/dataplane-operator/blob/main/config/samples/dataplane_v1beta1_openstackdataplanenodeset.yaml)
+for an example to modify as described in this document.
 
 Do not yet create the CR in OpenShift as the edits described in the
-next sections will be required.
+next sections are required.
 
 ### Shorten the Service list
 
-Shorten the services list and add the `ceph-hci-pre` service so that
-it only has `configure-network`, `validate-network`, and
-`ceph-hci-pre`.
+Update the `services` list:
+
+- Add the `ceph-hci-pre` service before the `configure-os` service.
+- Remove any services after the `run-os` service for now.
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
-  roles:
-    edpm-compute:
-      ...
-      services:
-        - configure-network
-        - validate-network
-        - install-os
-        - configure-os
-        - ceph-hci-pre
-        - run-os
+  services:
+    - configure-network
+    - validate-network
+    - install-os
+    - ceph-hci-pre
+    - configure-os
+    - run-os
 ```
 In the example above the services for `ovn`, `libvirt`, and `nova`
 have been removed. If there are other services besides the one in
@@ -101,23 +95,28 @@ edpm-ansible role called `ceph-hci-pre`. This role injects a
 `ceph-networks.yaml` file into `/var/lib/edpm-config/firewall`
 so that when the `edpm_nftables` role runs, firewall ports are open
 for Ceph services. By default the `ceph-networks.yaml` file only
-contains directives to open the ports required by the Ceph RBD block
-storage service. This is because of the following default Ansible
-variable values:
+contains directives to open the ports required by the Ceph RBD
+(block), RGW (object) and NFS (files) services. This is because of the
+following default Ansible variable value:
 ```yaml
 edpm_ceph_hci_pre_enabled_services:
   - ceph_mon
   - ceph_mgr
   - ceph_osd
+  - ceph_rgw
+  - ceph_nfs
+  - ceph_rgw_frontend
+  - ceph_nfs_frontend
 ```
-If other Ceph services like RGW, CephFS, or Dashboard will be deployed
+If other Ceph services, like the Ceph Dashboard, will be deployed
 on HCI nodes, then add additional services to the enabled services
 list above. For more informatoin, see the `ceph-hci-pre` role in the
 [edpm-ansible role documentation](https://openstack-k8s-operators.github.io/edpm-ansible/roles.html).
 
-The `run-os` service is run after `ceph-hci-pre` because it enables
-the firewall rules which `ceph-hci-pre` put in place. The `run-os`
-service also configures NTP, which is requried by Ceph.
+The `configure-os` and `run-os` services are run after `ceph-hci-pre`
+because they enable the firewall rules which `ceph-hci-pre` put in
+place. The `run-os` service also configures NTP, which is requried by
+Ceph.
 
 ### Add a Ceph cluster network
 
@@ -441,21 +440,20 @@ and Ceph configuration file.
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
-  roles:
-    edpm-compute:
-      nodeTemplate:
-        extraMounts:
-        - extraVolType: Ceph
-          volumes:
-          - name: ceph
-            secret:
-              secretName: ceph-conf-files
-          mounts:
-          - name: ceph
-            mountPath: "/etc/ceph"
-            readOnly: true
+  ...
+  nodeTemplate:
+    extraMounts:
+    - extraVolType: Ceph
+      volumes:
+      - name: ceph
+        secret:
+          secretName: ceph-conf-files
+      mounts:
+      - name: ceph
+        mountPath: "/etc/ceph"
+        readOnly: true
 ```
 
 ### Restore the full services list
@@ -467,23 +465,20 @@ the full services list needs to be restored. For example:
 
 ```yaml
 apiVersion: dataplane.openstack.org/v1beta1
-kind: OpenStackDataPlane
+kind: OpenStackDataPlaneNodeSet
 spec:
   ...
-  roles:
-    edpm-compute:
-      ...
-      services:
-        - configure-network
-        - validate-network
-        - install-os
-        - configure-os
-        - ceph-hci-pre
-        - run-os
-        - ceph-client
-        - ovn
-        - libvirt
-        - nova-custom-ceph
+  services:
+    - configure-network
+    - validate-network
+    - install-os
+    - configure-os
+    - ceph-hci-pre
+    - run-os
+    - ceph-client
+    - ovn
+    - libvirt
+    - nova-custom-ceph
 ```
 In addition to restoring the default service list, the `ceph-client`
 service is added after `run-os`. The `ceph-client` service configures