Skip to content

Latest commit

 

History

History
125 lines (104 loc) · 5.1 KB

edpm_service_overview.md

File metadata and controls

125 lines (104 loc) · 5.1 KB
Raw

Overview of Configuring Services on EDPM Nodes

This document gives a high level overview of how to create a service which needs to be hosted on a RHEL-based External Dataplane Management (EDPM) node.

OpenStack Operator

Though the service may not run directly on OpenShift workers, the service definition must be made through the Custom Resources of the OpenStack Operator which can run Ansible playbooks to configure the service.

The service should fit the composable service model so that the deployer can orchestrate when and how the new service is deployed.

A PR sent to the openstack-operator repository for a new service should contain an OpenStackDataPlaneService definition (examples) and a sample of how to use the service in an OpenStackDataPlaneNodeSet (examples).

See the OpenStack Operator documentation for more information.

EDPM Ansible

The Ansible playbook(s) and role(s) which configure the new service should be contributed to the edpm-ansible repository. The Ansible Execution Environment (AEE) pod, which is created by the dataplane operator, contains a copy of this repository in its container image.

See the EDPM Ansible documentation for more information.

EDPM Service Patterns

Common usage patterns of Data Plane Operator custom resources and EDPM Ansible for services which run on EDPM nodes.

Configuration Files

As described in Services configuration / k8s operators, Secrets and ConfigMaps can store configuration information. If an extraMount is used in an OpenStackDataPlaneNodeSet, then the AEE pod will mount those Secrets and ConfigMaps. Because Ansible will then have direct access to that data as files, the ansible.builtin.copy module (or similar) may be used to put that configuration data on an EDPM node.

Firewall

If the new service requires firewall ports to be opened, then the service's EDPM Ansible role should create a YAML file in /var/lib/edpm-config/firewall/ on EDPM nodes with the appropriate syntax. For example, the following file in this directory opens the ports for the Ceph Monitoring service.

- rule_name: "110 allow ceph_mon"
  rule:
    proto: tcp
    dport: [6789, 3300]

When the configure-os and run-os composable services run, they execute the role edpm_nftables This role reads files in /var/lib/edpm-config/firewall/ and creates a edpm-rules.nft file in /etc/nftables/ and then configures the live firewall to use it.

For example, the rule above results in the following lines in /etc/nftables/edpm-rules.nft

[root@edpm-compute-0 ~]# grep ceph /etc/nftables/edpm-rules.nft
# 110 allow ceph_mon {'proto': 'tcp', 'dport': [6789, 3300]}
add rule inet filter EDPM_INPUT tcp dport { 6789,3300 } ct state new counter accept comment "110 allow ceph_mon"
[root@edpm-compute-0 ~]#

which results in the following output from the NFT command.

[root@edpm-compute-0 ~]# nft list ruleset | grep ceph_mon
		tcp dport { 3300, 6789 } ct state new counter packets 0 bytes 0 accept comment "110 allow ceph_mon"
[root@edpm-compute-0 ~]#

If the service needs to be deployed after the configure-os and run-os services have run, then the Ansible for that service can directly call the edpm_nftables role to update the files in /etc/nftables and reload the rules. An example of this from the edpm_libvirt role is below.

- name: Copy qemu vnc firewall config
  ansible.builtin.template:
    src: "firewall.yaml"
    dest: "/var/lib/edpm-config/firewall/vnc.yaml"
    mode: "640"
- name: Configure firewall for the vnc
  ansible.builtin.include_role:
      name: osp.edpm.edpm_nftables
      tasks_from: "configure.yml"
- name: Reload firewall for new vnc rule
  ansible.builtin.include_role:
      name: osp.edpm.edpm_nftables
      tasks_from: "run.yml"

NTP

The openstack-operator provided service run-os (described in composable services) calls an edpm-ansible role to configure the system clock to synchronize with NTP servers. If the new service requires NTP, then put it after run-os on the services list in the service's sample OpenStackDataPlaneNodeSet file.