-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add unmanaged infrastructure proposal
- Loading branch information
Showing
1 changed file
with
170 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,170 @@ | ||
| --- | ||
| title: Unmanaged cluster infastructure | ||
| authors: | ||
| - "@enxebre" | ||
| - "@joelspeed" | ||
| - "@alexander-demichev" | ||
| reviewers: | ||
| - "@vincepri" | ||
| - "@randomvariable" | ||
| - "@CecileRobertMichon" | ||
| - "@yastij" | ||
|
|
||
| creation-date: 2021-02-03 | ||
| last-updated: 2021-02-03 | ||
| status: implementable | ||
| see-also: | ||
| replaces: | ||
| superseded-by: | ||
| --- | ||
|
|
||
| # Unmanaged cluster infrastucture | ||
|
|
||
| ## Table of Contents | ||
|
|
||
| A table of contents is helpful for quickly jumping to sections of a proposal and for highlighting | ||
| any additional information provided beyond the standard proposal template. | ||
| [Tools for generating](https://github.com/ekalinin/github-markdown-toc) a table of contents from markdown are available. | ||
|
|
||
| - [Title](#title) | ||
| - [Table of Contents](#table-of-contents) | ||
| - [Glossary](#glossary) | ||
| - [Summary](#summary) | ||
| - [Motivation](#motivation) | ||
| - [Goals](#goals) | ||
| - [Non-Goals/Future Work](#non-goalsfuture-work) | ||
| - [Proposal](#proposal) | ||
| - [User Stories](#user-stories) | ||
| - [Story 1](#story-1) | ||
| - [Story 2](#story-2) | ||
| - [Requirements (Optional)](#requirements-optional) | ||
| - [Functional Requirements](#functional-requirements) | ||
| - [FR1](#fr1) | ||
| - [FR2](#fr2) | ||
| - [Non-Functional Requirements](#non-functional-requirements) | ||
| - [NFR1](#nfr1) | ||
| - [NFR2](#nfr2) | ||
| - [Implementation Details/Notes/Constraints](#implementation-detailsnotesconstraints) | ||
| - [Security Model](#security-model) | ||
| - [Risks and Mitigations](#risks-and-mitigations) | ||
| - [Alternatives](#alternatives) | ||
| - [Upgrade Strategy](#upgrade-strategy) | ||
| - [Additional Details](#additional-details) | ||
| - [Test Plan [optional]](#test-plan-optional) | ||
| - [Graduation Criteria [optional]](#graduation-criteria-optional) | ||
| - [Version Skew Strategy [optional]](#version-skew-strategy-optional) | ||
| - [Implementation History](#implementation-history) | ||
|
|
||
| ## Glossary | ||
|
|
||
| Refer to the [Cluster API Book Glossary](https://cluster-api.sigs.k8s.io/reference/glossary.html). | ||
|
|
||
| ### Managed cluster infrastructure | ||
|
|
||
| Cluster infratructure which lifecycle is managed by a provider infraCluster CR. | ||
| E.g in AWS: | ||
| - Network | ||
| - VPC | ||
| - Subnets | ||
| - Internet gateways | ||
| - Nat gateways | ||
| - Route tables | ||
| - Security groups | ||
| - Load balancers | ||
|
|
||
| ### Unmanaged cluster infrastructure | ||
| Cluster infratructure which lifecycle is not managed by CAPI but rather by an external entity. | ||
|
|
||
| ### Data plane | ||
| Kubernetes endpoint accesible by a workload cluster admin which nodes capacity is intended exclusively to run user facing workloads | ||
|
|
||
| ## Summary | ||
|
|
||
| This proposal introduces first class support for "unmanaged" infrastructure for CAPI providers to consolidate the boundaries between managed and unmanaged cluster infrastructure. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Currently CAPI providers support an opinionated happy path to create and manage cluster infrastructure lifecycle. The fundamental use case we want to support is bring your own infrastructure. An "unmanaged" CAPI infraCluster won't reconcile or manage the lifecycle of the cluster infrastructure, but CAPI will be able to create compute nodes within it. | ||
|
|
||
| This will ease adoption of CAPI in heterogeneous real world environments with restricted privileges and where the provider infrastructure for the cluster needs to be managed out of band. | ||
|
|
||
| ### Goals | ||
|
|
||
| - Introduce support for "unmanaged" infrastructure consistently across CAPI providers. | ||
| - The machine controller must be able to operate and manage machines when the infastructure is "unmanaged". | ||
|
|
||
| ### Non-Goals/Future Work | ||
|
|
||
| - Modify existing managed behaviour. | ||
|
|
||
| ## Proposal | ||
|
|
||
| Providers will add a `ManagementPolicy=Managed|Unmanaged` field to the provider infraCluster spec. | ||
|
|
||
| The machine controller must be able to operate without hard dependencies regardless of the cluster infrastructure being managed or unmanaged. | ||
|  | ||
|
|
||
| ### User Stories | ||
|
|
||
| #### Story 1 - Alternate control plane provisioning with user managed infrastructure | ||
| As a cluster provider I want to use CAPI in my service offering to orchestrate Kubernetes bootstraping while letting workload cluster operators own their infrastructure lifecycle for the data plane. | ||
|
|
||
| #### Story 2 - Restricted access to cloud provider APIs | ||
| As a cluster operator I want to use CAPI to orchestrate kubernetes bootstraping while restricting the privileges I need to grant for my cloud provider because of organisational cloud security constraints. | ||
|
|
||
| #### Story 3 - Consuming existing cloud infrastructure | ||
| As a cluster operator I want to use CAPI to orchestate Kubernetes bootstraping while reusing infrastructure that has already been created in the organisation either by me or another team. | ||
|
|
||
| ### Implementation Details/Notes/Constraints | ||
|
|
||
| **Managed** | ||
| - It will be default and will preserve existing behaviour. | ||
|
|
||
| **Unmanaged** | ||
|
|
||
| - The provider infraCluster controller will skip any infrastructure reconciliation. | ||
|
|
||
| - The provider infraCluster will set readiness to true. | ||
|
|
||
| - The provider infraCluster will set a condition unamangedReady to true. | ||
|
|
||
| - CAPI will proceed with further reconciliation as usual. | ||
|
|
||
|
|
||
| ### Security Model | ||
|
|
||
| When unmanaged no additional privileges for a cloud provider need to be given to CAPI other than the required to manage machines. | ||
|
|
||
| #### Multitenancy | ||
|
|
||
| CAPI supports a best effort multitenancy model with deploying multiple controllers. In this scenario adhoc privileges for each controller can be granted to satisfy "managed" or "unmanaged" cluster nifrastr | ||
|
|
||
| ### Risks and Mitigations | ||
|
|
||
|
|
||
| ## Alternatives | ||
|
|
||
| We could have and adhoc CRD https://github.com/kubernetes-sigs/cluster-api/issues/4095 | ||
|
|
||
| This would introduce complexity for the CAPI ecosystem with yet an additional CRD and it woudn't scale well across providers as it would need to contain provider specific information. | ||
|
|
||
| ## Upgrade Strategy | ||
|
|
||
| Support is introduced by adding a new field for the provider infraCluster. | ||
|
|
||
| This makes any transition backward compatible and leave the current managed behaviour untouched. | ||
|
|
||
| The new field will be optional and default to "managed" | ||
|
|
||
| ## Additional Details | ||
|
|
||
| ## Implementation History | ||
|
|
||
| - [ ] MM/DD/YYYY: Proposed idea in an issue or [community meeting] | ||
| - [ ] MM/DD/YYYY: Compile a Google Doc following the CAEP template (link here) | ||
| - [ ] MM/DD/YYYY: First round of feedback from community | ||
| - [ ] MM/DD/YYYY: Present proposal at a [community meeting] | ||
| - [ ] MM/DD/YYYY: Open proposal PR | ||
|
|
||
| <!-- Links --> | ||
| [community meeting]: https://docs.google.com/document/d/1Ys-DOR5UsgbMEeciuG0HOgDQc8kZsaWIWJeKJ1-UfbY |