-
Notifications
You must be signed in to change notification settings - Fork 476
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[OCPBU-199] enhancement proposal for dual-stack support with openstack
- Loading branch information
1 parent
19c6f76
commit abd824c
Showing
1 changed file
with
173 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,173 @@ | ||
| --- | ||
| title: openstack-dual-stack | ||
| authors: | ||
| - @maysamacedo | ||
| reviewers: | ||
| - "@mandre for SME review on OpenStack and general feedback" | ||
| - "@cybertron for SME review on on-premise networking" | ||
| - "@patrickdillon for SME review and approval on installer" | ||
| approvers: | ||
| - "@mandre" | ||
| creation-date: 2023-03-13 | ||
| last-updated: 2023-03-13 | ||
| tracking-link: | ||
| - https://issues.redhat.com/browse/OCPBU-199 | ||
| - https://issues.redhat.com/browse/OSASINFRA-1938 | ||
| --- | ||
|
|
||
| # OpenStack dual-stack | ||
|
|
||
| Covers adding support of OpenShift dual-stack networking on OpenStack. | ||
|
|
||
| ## Summary | ||
|
|
||
| Customers need support for OpenShift clusters running on OpenStack with IPv4 and IPv6. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Customers need dual-stack support to allow migration of workloads from IPv4 to IPv6 | ||
| as they are running out of IPv4 addresses. This will allow openshift on openstack to | ||
| serve a wider variety of workloads. | ||
|
|
||
| ### User Stories | ||
|
|
||
| * As an user of a IPI OpenShift cluster, I want to have ingress and egress working with dual-stack. | ||
| * As an user of a IPI OpenShift cluster, I want clusterIP services to be working with dual-stack. | ||
|
|
||
| ### Goals | ||
|
|
||
| - OpenStack VMs are configured with one dual-stack interface. | ||
| - The IPv6 Subnets can be configured with: Stateless, SLAAC or Stateful. | ||
| - The addresses on the VM's interface are configured with the same address | ||
| assigned by OpenStack to the Port. | ||
| - Nodes are configure with both IPv4 and IPv6 addresses, without exposing | ||
| additional addresses, like storage interfaces. | ||
| - Pods are configured with both IPv4 and IPv6 addresses. | ||
|
|
||
| ### Non-Goals | ||
|
|
||
| - Support for OpenStack endpoints configured with dual-stack. | ||
| - Support for LoadBalancer type Services. | ||
| - Installer (IPI) handles creation of the Network and dual-stack Subnet. | ||
|
|
||
| ## Proposal | ||
|
|
||
| Customers will be able to specify the IPv4 and IPv6 Subnets | ||
| meant to be used in the `install-config.yaml`. | ||
|
|
||
| ### Workflow Description | ||
|
|
||
| 1. The customers will configure a network (tenant or provider) with IPv4 and | ||
| IPv6 subnets. The IPv6 address mode for the Subnet can be of any type: | ||
| SLAAC, Stateless or Stateful. | ||
| 2. The customer will create API and Ingress Ports using the Network created in the | ||
| previous step. | ||
| 3. The customer will write the `install-config.yaml` and add the subnet IDs to | ||
| the new field `machinesSubnets`. | ||
| 4. The customer will generate the manifests and configured MachineConfigs | ||
| for the master and workers depending on the IPv6 addresses mode chosen for the Subnets | ||
| in step 1. If SLAAC or Stateless was used the MachineConfigs has to contain `ip_addr_mode=eui64` | ||
| and `method=auto` in the connection for the ethernet interface. While when using stateful | ||
| it has to enable `ip=dhcp,dhcpv6` in the kernel arguments. | ||
| 5. The `uuid` from the network has to be added in the worker MachineSet alongside with the | ||
| subnets that were pre-filled for the machine-api-provider-openstack to create one unique Port | ||
| with both addresses configured for the Server. | ||
| 6. The CIDRs of the subnets have to added to the `cloud-provider-config.yaml` | ||
| in the `address-sort-order` field to make sure IPv4 address will have higher priority in the | ||
| Node addresses. | ||
| 7. The installer will validate the subnets and VIPs, add the security group to the api and ingress | ||
| ports and link the floating IPs to those Ports. Also, create the additional IPv6 security group rules | ||
| and configured the servers with dual-stack addresses and add the dual-stack allowed address pairs. | ||
| 8. In machine-config-operator, kubelet will be update to not be configured with node-ips. | ||
| 9. cloud-provider-openStack will filter the node addresses that are meant to be specified in the Status of the Node | ||
| by looking for a tag that was applied in the main Ports of the Node. | ||
|
|
||
| ### API Extensions | ||
|
|
||
| The `machinesSubnet` field, which is available in the `install-config.yaml` under | ||
| `platform` and `openstack` will be deprecated in favor of a new field named `machinesSubnets` | ||
| to persist both IPv4 and IPv6 provided Subnets. | ||
|
|
||
| For installation on OpenStack with dual-stack kubelet will no longer rely on the node-ip to get the node | ||
| addresses, given that when using external cloud provider, the node-ip currently has no support for dual-stack, this [enhancement](https://github.com/kubernetes/enhancements/pull/3706) describes it. Instead, | ||
| the node addresses will be provided directly by the cloud provider openstack, which can rely on a tag to filter them. This [issue](https://github.com/kubernetes/cloud-provider-openstack/issues/2096) details the idea behind it. | ||
|
|
||
| ### Implementation Details/Notes/Constraints | ||
|
|
||
| The idea behind using bring your own network instead of basic IPI installation is to allow support | ||
| for a wider variety of ways on how to configure the Network and Subnet. As an example, the Ports have | ||
| to be pre-created by the user because when using SLAAC or Stateless the specification of fixed-ips during | ||
| Port creation is not allowed. | ||
|
|
||
| ### Risks and Mitigations | ||
|
|
||
| We need to validate if the proposal for not setting the node-ip will be a robust enough | ||
| solution that we want to maintain, specially around upgrades. If not robust, we can use the dual-stack support for | ||
| node-ip that is being [developed upstream](https://github.com/kubernetes/kubernetes/pull/116305). | ||
|
|
||
| ### Drawbacks | ||
|
|
||
| NA | ||
|
|
||
| ## Design Details | ||
|
|
||
| ### Open Questions | ||
|
|
||
| 1. Do we want to let the user configure the `MachineConfig` prior installation according | ||
| to the type of IPv6 mode used or we want to teach the machine-config-operator how to | ||
| configure those? | ||
| 2. How does the routing work when using tenant network and attempt to connect to the API | ||
| using IPv6 address in the GUA range? until now we tested with Provider network. | ||
| 3. Can we update machine-api-provider-openstack to not require the network uuid defined | ||
| in the ProviderSpec? | ||
| 4. Will we restrict the requirement for pre-creation of Port for API and Ingress only to dual-stack? | ||
| 5. When to support migration from single stack to dual-stack? | ||
|
|
||
| ### Test Plan | ||
|
|
||
| - Develop CI for basic dual-stack installation with Provider Network. | ||
|
|
||
| ### Graduation Criteria | ||
|
|
||
| The Tech-Preview target is OCP 4.14 and GA would be 4.15. | ||
|
|
||
| #### Dev Preview -> Tech Preview | ||
|
|
||
| - Succesfully bring up a dual-stack OpenStack cluster without manual intervation | ||
| of the user with exception of the documented requirements. | ||
| - End user documentation | ||
| - Sufficient test coverage | ||
| - Gather feedback from users rather than just developers | ||
|
|
||
| #### Tech Preview -> GA | ||
|
|
||
| - Sufficient time for feedback | ||
| - User facing documentation created in [openshift-docs](https://github.com/openshift/openshift-docs/) | ||
|
|
||
| #### Removing a deprecated feature | ||
|
|
||
| NA | ||
|
|
||
| ### Upgrade / Downgrade Strategy | ||
|
|
||
| TBD | ||
|
|
||
| ### Version Skew Strategy | ||
|
|
||
| NA | ||
|
|
||
| ### Operational Aspects of API Extensions | ||
|
|
||
| NA | ||
|
|
||
| #### Failure Modes | ||
|
|
||
| NA | ||
|
|
||
| #### Support Procedures | ||
|
|
||
| ## Implementation History | ||
|
|
||
| ## Alternatives | ||
|
|
||
|
|