From 4f1f78f423f4816aa8a354283c968e4a27f135d4 Mon Sep 17 00:00:00 2001 From: Maysa Macedo Date: Mon, 13 Mar 2023 14:12:33 +0100 Subject: [PATCH] [OCPBU-199] enhancement proposal for dual-stack support with openstack --- enhancements/network/openstack-dual-stack.md | 247 +++++++++++++++++++ 1 file changed, 247 insertions(+) create mode 100644 enhancements/network/openstack-dual-stack.md diff --git a/enhancements/network/openstack-dual-stack.md b/enhancements/network/openstack-dual-stack.md new file mode 100644 index 0000000000..36c845c1f9 --- /dev/null +++ b/enhancements/network/openstack-dual-stack.md @@ -0,0 +1,247 @@ +--- +title: openstack-dual-stack +authors: + - "@maysamacedo" +reviewers: + - "@mandre for SME review on OpenStack and general feedback" + - "@cybertron for SME review on on-premise networking" + - "@patrickdillon for SME review and approval on installer" +approvers: + - "@mandre" +api-approvers: + - "None" +creation-date: 2023-03-13 +last-updated: 2023-03-13 +tracking-link: + - https://issues.redhat.com/browse/OCPBU-199 + - https://issues.redhat.com/browse/OSASINFRA-1938 +--- + +# OpenStack dual-stack + +Covers adding support of OpenShift dual-stack networking on OpenStack. + +## Summary + +Customers need support for OpenShift clusters running on OpenStack with IPv4 and IPv6. + +## Motivation + +Customers need dual-stack support to allow migration of workloads from IPv4 to IPv6 +as they are running out of IPv4 addresses. This will allow OpenShift on OpenStack to +serve a wider variety of workloads, like the ones needed by telco. + +### User Stories + +* As an user of a IPI OpenShift cluster, I want to have ingress and egress working with dual-stack. +* As an user of a IPI OpenShift cluster, I want clusterIP services to be working with dual-stack. +* As an user of a IPI OpenShift cluster, I want LoadBalancer type services to be working with dual-stack. +* As an user of a IPI OpenShift cluster, I want to be able to convert a single stack cluster to dual-stack and the way back. + +### Goals + +- OpenStack VMs are configured with one dual-stack interface. +- The IPv6 Subnets can be configured with: Stateless, SLAAC or Stateful. +- The addresses on the VM's interface are configured with the same address +assigned by OpenStack to the port. +- Nodes are configured with both IPv4 and IPv6 addresses, without exposing +additional addresses, like storage interfaces. +- Pods are configured with both IPv4 and IPv6 addresses from the cidrs defined on the `clusterNetwork` or the Nodes Addresses. +- A pre-existing dual-stack network is used for the OpenShift cluster + +### Non-Goals + +- Support for OpenStack endpoints configured with dual-stack. +- Installer (IPI) handles creation of the dual-stack OpenStack Network and Subnets. + +## Proposal + +Customers will be able to specify the IPv4 and IPv6 Subnets +meant to be used in the `install-config.yaml`. + +### Workflow Description + +1. The customers will configure a network (tenant or provider) with IPv4 and +IPv6 subnets. The IPv6 address mode for the Subnet can be of any type: +SLAAC, Stateless or Stateful. Here is a sample of network and subnets creation: + +```bash +$ openstack network create dual-stack +$ openstack subnet create --network dual-stack --subnet-range 10.0.0.0/16 ipv4-subnet +$ openstack subnet create --network dual-stack --ipv6-address-mode slaac --ipv6-ra-mode slaac --subnet-range 2001:db8:2222:5555::/64 --ip-version 6 ipv6-subnet +``` + +2. The customer will create API and Ingress Ports using the dual-stack network created in the +previous step. Here is a sample of ports creation: + +```bash +$ openstack port create api --network dual-stack -f value -c fixed_ips +[{'subnet_id': 'eac94701-fd9c-4d7b-a0a6-292506e19ea9', 'ip_address': '10.0.0.239'}, {'subnet_id': '70728437-9708-42ec-bf4a-b844e3846190', 'ip_address': '2001:db8:2222:5555:f816:0000:0000:0000'}] +``` + +```bash +$ openstack port create ingress --network dual-stack -f value -c fixed_ips +[{'subnet_id': 'eac94701-fd9c-4d7b-a0a6-292506e19ea9', 'ip_address': '10.0.0.147'}, {'subnet_id': '70728437-9708-42ec-bf4a-b844e3846190', 'ip_address': '2001:db8:2222:5555:f816:1111:1111:1111'}] +``` + +3. The customer will write the `install-config.yaml` and add the details of the control plane port to the new field `controlPlanePort`, which contains Filters of ID and/or Name for the Network and Subnets. The `controlPlanePort` API definition is consolidated to look similar to `PortOpts` field in the Machine's `OpenstackProviderSpec`. + +```yaml +platform: + openstack: + ingressVIPs: ['10.0.0.147', '2001:db8:2222:5555:f816:1111:1111:1111'] + apiVIPs: ['10.0.0.239','2001:db8:2222:5555:f816:0000:0000:0000'] + controlPlanePort: + fixedIPs: + - subnet: + id: eac94701-fd9c-4d7b-a0a6-292506e19ea9 + - subnet: + id: 70728437-9708-42ec-bf4a-b844e3846190 + network: + name: hostonly +``` + +The generated Machine will look like the following: + +```yaml +apiVersion: machine.openshift.io/v1beta1 +kind: Machine +metadata: + creationTimestamp: null + labels: + machine.openshift.io/cluster-api-cluster: ostest-h8q6h + machine.openshift.io/cluster-api-machine-role: master + machine.openshift.io/cluster-api-machine-type: master + name: ostest-h8q6h-master-0 + namespace: openshift-machine-api +spec: + lifecycleHooks: {} + metadata: {} + providerSpec: + value: + apiVersion: machine.openshift.io/v1alpha1 + cloudName: openstack + cloudsSecret: + name: openstack-cloud-credentials + namespace: openshift-machine-api + flavor: m1.xlarge + image: ostest-h8q6h-rhcos + kind: OpenstackProviderSpec + metadata: + creationTimestamp: null + networks: + - filter: + name: hostonly + subnets: + - filter: + id: db333b56-1fb1-4fd6-a74c-65dfb9a49137 + - filter: + id: f4e1fe8c-2230-44dc-b57d-c126a857261c + primarySubnet: db333b56-1fb1-4fd6-a74c-65dfb9a49137 + securityGroups: + - filter: {} + name: ostest-h8q6h-master + - filter: {} + uuid: 32f60fe4-5260-4489-a36c-aaaa3fb24215 + serverGroupName: ostest-h8q6h-master + serverMetadata: + Name: ostest-h8q6h-master + openshiftClusterID: ostest-h8q6h + tags: + - openshiftClusterID=ostest-h8q6h + trunk: true + userDataSecret: + name: master-user-data +status: {} +``` +4. The installer will validate the network, subnets and VIPs, add the security group to the api and ingress +ports and link the floating IPs to those Ports, when necessary. Also, create the additional IPv6 security group rules +and configured the servers with dual-stack addresses, add the dual-stack allowed address pairs and +include in the bootstrap node the following Network Manager configuration: + +```ini +[connection] +type=ethernet +[ipv6] +method=auto +``` +5. The machine-config-operator will configure the ethernet connection following the OpenStack [guidelines](https://docs.openstack.org/neutron/wallaby/admin/config-ipv6.html#configuring-interfaces-of-the-guest) to make sure the generated IPv6 addresses match the ones configured in OpenStack. +6. The cluster-cloud-controller-manager will enable the `CloudDualStackNodeIPs` feature gate for the openstack-cloud-controller-manager. + +### API Extensions + +The `machinesSubnet` field, which is available in the `install-config.yaml` under +`platform` and `openstack` will be deprecated in favor of a new field named `controlPlanePort` +to persist both IPv4 and IPv6 provided Subnets. + +### Implementation Details/Notes/Constraints + +The idea behind using bring your own network instead of basic IPI installation is to allow support +for a wider variety of ways on how to configure the Network and Subnet. As an example, the Ports have +to be pre-created by the user because when using SLAAC or Stateless the specification of fixed-ips during +Port creation is not allowed. UPI can also be an option in future releases. + +### Risks and Mitigations + +The support for dual-stack in Kubelet's node-ip setting is a [feature in alpha state in Kubernetes v1.27](https://kubernetes.io/docs/concepts/services-networking/dual-stack/#configure-ipv4-ipv6-dual-stack) and requires a [fix to merge](https://github.com/kubernetes/kubernetes/pull/118662). + +### Drawbacks + +NA + +## Design Details + +### Open Questions + +### Test Plan + +- Develop CI for basic dual-stack installation with Provider Network. + +### Graduation Criteria + +The Tech-Preview target is OCP 4.14 and GA would be 4.15. + +#### Dev Preview -> Tech Preview + +- Succesfully bring up a day 1 dual-stack OpenStack cluster without manual intervation + of the user with exception of any documented requirements. +- End user documentation +- Sufficient test coverage +- Gather feedback from users rather than just developers + +#### Tech Preview -> GA + +- Sufficient time for feedback +- Migration from single-stack to dual-stack and the other way around +- Support IPv6 primary dual-stack deployments +- User facing documentation created in [openshift-docs](https://github.com/openshift/openshift-docs/) + +#### Removing a deprecated feature + +NA + +### Upgrade / Downgrade Strategy + +It's not possible to upgrade directly from one version using single-stack to another version with dual-stack. +The customer would have to first upgrade to a version that has support for dual-stack, but maintaining the single-stack. +After upgrade the customer could follow [the conversion procedure to dual-stack](https://docs.openshift.com/container-platform/4.12/networking/ovn_kubernetes_network_provider/converting-to-dual-stack.html). + +### Version Skew Strategy + +NA + +### Operational Aspects of API Extensions + +NA + +#### Failure Modes + +NA + +#### Support Procedures + +## Implementation History + +## Alternatives + +