From abd824c64c8fc7d83155cfa0b23ceb223e60cf59 Mon Sep 17 00:00:00 2001 From: Maysa Macedo Date: Mon, 13 Mar 2023 14:12:33 +0100 Subject: [PATCH] [OCPBU-199] enhancement proposal for dual-stack support with openstack --- enhancements/network/openstack-dual-stack.md | 173 +++++++++++++++++++ 1 file changed, 173 insertions(+) create mode 100644 enhancements/network/openstack-dual-stack.md diff --git a/enhancements/network/openstack-dual-stack.md b/enhancements/network/openstack-dual-stack.md new file mode 100644 index 00000000000..4cd1ab41c75 --- /dev/null +++ b/enhancements/network/openstack-dual-stack.md @@ -0,0 +1,173 @@ +--- +title: openstack-dual-stack +authors: + - @maysamacedo +reviewers: + - "@mandre for SME review on OpenStack and general feedback" + - "@cybertron for SME review on on-premise networking" + - "@patrickdillon for SME review and approval on installer" +approvers: + - "@mandre" +creation-date: 2023-03-13 +last-updated: 2023-03-13 +tracking-link: + - https://issues.redhat.com/browse/OCPBU-199 + - https://issues.redhat.com/browse/OSASINFRA-1938 +--- + +# OpenStack dual-stack + +Covers adding support of OpenShift dual-stack networking on OpenStack. + +## Summary + +Customers need support for OpenShift clusters running on OpenStack with IPv4 and IPv6. + +## Motivation + +Customers need dual-stack support to allow migration of workloads from IPv4 to IPv6 +as they are running out of IPv4 addresses. This will allow openshift on openstack to +serve a wider variety of workloads. + +### User Stories + +* As an user of a IPI OpenShift cluster, I want to have ingress and egress working with dual-stack. +* As an user of a IPI OpenShift cluster, I want clusterIP services to be working with dual-stack. + +### Goals + +- OpenStack VMs are configured with one dual-stack interface. +- The IPv6 Subnets can be configured with: Stateless, SLAAC or Stateful. +- The addresses on the VM's interface are configured with the same address +assigned by OpenStack to the Port. +- Nodes are configure with both IPv4 and IPv6 addresses, without exposing +additional addresses, like storage interfaces. +- Pods are configured with both IPv4 and IPv6 addresses. + +### Non-Goals + +- Support for OpenStack endpoints configured with dual-stack. +- Support for LoadBalancer type Services. +- Installer (IPI) handles creation of the Network and dual-stack Subnet. + +## Proposal + +Customers will be able to specify the IPv4 and IPv6 Subnets +meant to be used in the `install-config.yaml`. + +### Workflow Description + +1. The customers will configure a network (tenant or provider) with IPv4 and +IPv6 subnets. The IPv6 address mode for the Subnet can be of any type: +SLAAC, Stateless or Stateful. +2. The customer will create API and Ingress Ports using the Network created in the +previous step. +3. The customer will write the `install-config.yaml` and add the subnet IDs to +the new field `machinesSubnets`. +4. The customer will generate the manifests and configured MachineConfigs +for the master and workers depending on the IPv6 addresses mode chosen for the Subnets +in step 1. If SLAAC or Stateless was used the MachineConfigs has to contain `ip_addr_mode=eui64` +and `method=auto` in the connection for the ethernet interface. While when using stateful +it has to enable `ip=dhcp,dhcpv6` in the kernel arguments. +5. The `uuid` from the network has to be added in the worker MachineSet alongside with the +subnets that were pre-filled for the machine-api-provider-openstack to create one unique Port +with both addresses configured for the Server. +6. The CIDRs of the subnets have to added to the `cloud-provider-config.yaml` +in the `address-sort-order` field to make sure IPv4 address will have higher priority in the +Node addresses. +7. The installer will validate the subnets and VIPs, add the security group to the api and ingress +ports and link the floating IPs to those Ports. Also, create the additional IPv6 security group rules +and configured the servers with dual-stack addresses and add the dual-stack allowed address pairs. +8. In machine-config-operator, kubelet will be update to not be configured with node-ips. +9. cloud-provider-openStack will filter the node addresses that are meant to be specified in the Status of the Node +by looking for a tag that was applied in the main Ports of the Node. + +### API Extensions + +The `machinesSubnet` field, which is available in the `install-config.yaml` under +`platform` and `openstack` will be deprecated in favor of a new field named `machinesSubnets` +to persist both IPv4 and IPv6 provided Subnets. + +For installation on OpenStack with dual-stack kubelet will no longer rely on the node-ip to get the node +addresses, given that when using external cloud provider, the node-ip currently has no support for dual-stack, this [enhancement](https://github.com/kubernetes/enhancements/pull/3706) describes it. Instead, +the node addresses will be provided directly by the cloud provider openstack, which can rely on a tag to filter them. This [issue](https://github.com/kubernetes/cloud-provider-openstack/issues/2096) details the idea behind it. + +### Implementation Details/Notes/Constraints + +The idea behind using bring your own network instead of basic IPI installation is to allow support +for a wider variety of ways on how to configure the Network and Subnet. As an example, the Ports have +to be pre-created by the user because when using SLAAC or Stateless the specification of fixed-ips during +Port creation is not allowed. + +### Risks and Mitigations + +We need to validate if the proposal for not setting the node-ip will be a robust enough +solution that we want to maintain, specially around upgrades. If not robust, we can use the dual-stack support for +node-ip that is being [developed upstream](https://github.com/kubernetes/kubernetes/pull/116305). + +### Drawbacks + +NA + +## Design Details + +### Open Questions + +1. Do we want to let the user configure the `MachineConfig` prior installation according + to the type of IPv6 mode used or we want to teach the machine-config-operator how to + configure those? +2. How does the routing work when using tenant network and attempt to connect to the API + using IPv6 address in the GUA range? until now we tested with Provider network. +3. Can we update machine-api-provider-openstack to not require the network uuid defined +in the ProviderSpec? +4. Will we restrict the requirement for pre-creation of Port for API and Ingress only to dual-stack? +5. When to support migration from single stack to dual-stack? + +### Test Plan + +- Develop CI for basic dual-stack installation with Provider Network. + +### Graduation Criteria + +The Tech-Preview target is OCP 4.14 and GA would be 4.15. + +#### Dev Preview -> Tech Preview + +- Succesfully bring up a dual-stack OpenStack cluster without manual intervation + of the user with exception of the documented requirements. +- End user documentation +- Sufficient test coverage +- Gather feedback from users rather than just developers + +#### Tech Preview -> GA + +- Sufficient time for feedback +- User facing documentation created in [openshift-docs](https://github.com/openshift/openshift-docs/) + +#### Removing a deprecated feature + +NA + +### Upgrade / Downgrade Strategy + +TBD + +### Version Skew Strategy + +NA + +### Operational Aspects of API Extensions + +NA + +#### Failure Modes + +NA + +#### Support Procedures + +## Implementation History + +## Alternatives + +