Merge pull request #1991 from neolit123/1.20-replace-kubeadm-kustomiz…

…e-with-patches

keps/sig-cluster-lifecycle/kubeadm: mark the kustomization KEP as replaced

keps/sig-cluster-lifecycle/kubeadm/20190722-Advanced-configurations-with-kubeadm-(Kustomize).md

@@ -17,8 +17,8 @@ approvers:
   - "@luxas"
 editor: "@fabriziopandini"
 creation-date: 2019-07-22
-last-updated: 2019-09-02
-status: implementable
+last-updated: 2020-09-18
+status: replaced
 ---
 
 # Advanced configurations with kubeadm (Kustomize)
@@ -64,7 +64,10 @@ status: implementable
 
 ## Summary
 
-This KEP is aimed at defining a new kubeadm feature that will allow users to bootstrap 
+**Note:** this KEP was replaced by the KEP tracked in [#1739](https://github.com/kubernetes/enhancements/issues/1739)
+which uses raw patches for customization.
+
+This KEP is aimed at defining a new kubeadm feature that will allow users to bootstrap
 a Kubernetes cluster with static pods customizations not supported by the Kubeadm component configuration.
 
 ## Motivation
@@ -75,7 +78,7 @@ Kubernetes cluster via the Kubeadm component configuration or the corresponding
 More specifically the kubeadm component configuration provides an abstraction that allows to:
 
 1. To define configurations settings at cluster level using the `ClusterConfiguration`
-   config object 
+   config object
 2. To define a limited set of configurations at the node level using the
    `NodeRegistrationOptions` object or the `localAPIEndpoint` object
 
@@ -86,7 +89,7 @@ as of today. Some examples:
 - It is not possible to add sidecars e.g. for authorization web-hooks serving components.
 - It is not possible to set/alter timeouts for liveness probes in control plane components.
 
-This KEP aims to introduce a new feature that will eneable users full control of static 
+This KEP aims to introduce a new feature that will eneable users full control of static
 pod manifest generated by Kubeadm at node level - vs the kubeadm component configuration
 that allows mostly cluster-wide configurations on control-plane/etcd args only -.
 
@@ -106,29 +109,29 @@ to some iterations. The goal of the current iteration is to:
   during cluster lifecycle and more specifically for supporting the kubeadm upgrade workflow.
 - To ensure the proper functioning of “advanced configurations” with kubeadm phases.
 - To clarify what is in the scope of kubeadm and what instead should be the responsibility
-  of the users/of higher-level tools in the stack like e.g. cluster API 
+  of the users/of higher-level tools in the stack like e.g. cluster API
 
 ### Non-Goals
 
-- To provide any validation or guarantee about the security, conformance, 
-  consistency, of “advanced configurations” for control-plane/etcd settings. 
+- To provide any validation or guarantee about the security, conformance,
+  consistency, of “advanced configurations” for control-plane/etcd settings.
   As it is already for `extraArgs` fields in the kubeadm component configuration or in the
-  Kubelet/KubeProxy component config, the responsibility of proper usage of those 
+  Kubelet/KubeProxy component config, the responsibility of proper usage of those
   advanced configuration options belongs to higher-level tools/users.
 - To deprecate the Kubeadm component configuration because:
   - The component configuration provides an abstraction well suited for most common use
     cases (that can be addressed with cluster-wide configurations on control-plane/etcd command line args only)
-  - The component configuration implicitly defines the main cluster variants the kubeadm team 
+  - The component configuration implicitly defines the main cluster variants the kubeadm team
     is committed to support and monitor in the Kubernetes test grid.
 - To define how to manage “advanced configurations” when kubeadm and [`etcdadm`](https://github.com/kubernetes-sigs/etcdadm)
   project will integrate. This will be defined in following iterations of this KEP.
-- To define how to manage “advanced configurations” for the addons 
+- To define how to manage “advanced configurations” for the addons
   (this is postponed until kubeadm - [`addons`](https://github.com/kubernetes-sigs/cluster-addons)
   project integration).
 
 ## Proposal
 
-### User Stories 
+### User Stories
 
 #### Story 1
 As a cluster administrator, I want to add a sidecar to the kube-apiserver pod for running an
@@ -144,7 +147,7 @@ As a cluster administrator, I want to upgrade my cluster preserving all the
 
 ### Implementation Details
 
-This proposal explores as a first option for implementing Kubeadm 
+This proposal explores as a first option for implementing Kubeadm
 “advanced configurations“ the usage of Kustomize; please refer to
 [Declarative application management in Kubernetes](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/architecture/declarative-application-management.md)
 and [Kustomize KEP](https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/0008-kustomize.md)
@@ -169,7 +172,7 @@ This has some implications:
 3. Kubeadm is responsible for coordinating the execution of Kustomize within the
    init/join/upgrade workflows
 
-Additionally, in order to simplify the first implementation of this KEP, this 
+Additionally, in order to simplify the first implementation of this KEP, this
 proposal is going to assume that Kustomize patches for kubeadm are always defined
 specifically for the node where kubeadm is going to be executed.
 
@@ -203,20 +206,20 @@ EOF
 ```
 
 In case higher-level tools/users are providing only strategic merge patches, like in the example above,
-it is not requested to take care of defining a `kustomization.yaml` file. In case the `kustomization.yaml` 
-is missing kubeadm will create it on the fly using all the patches in the folder. 
+it is not requested to take care of defining a `kustomization.yaml` file. In case the `kustomization.yaml`
+is missing kubeadm will create it on the fly using all the patches in the folder.
 
 If instead higher-level tools/users are providing a `kustomization.yaml` file, kubeadm will use it.
 This scenario allows e.g. usage of `patchesJson6902`/other kustomize features.
 
-> In case a `kustomization.yaml` file exist, kubeadm ignores the `resource` value and replaces it with 
+> In case a `kustomization.yaml` file exist, kubeadm ignores the `resource` value and replaces it with
 > the static pod file manifest that should be kustomized by each phase.
 
 2 - Run kubeadm passing the folder where kustomization patches exist, e.g.
 
 ```
 kubeadm init --experimental-kustomize kubeadm-patches/
-or 
+or
 kubeadm init phase control-plane ...  --experimental-kustomize kubeadm-patches/
 ```
 or
@@ -232,7 +235,7 @@ kubeadm upgrade apply --experimental-kustomize kubeadm-patches/
 or
 ```
 kubeadm upgrade node --experimental-kustomize kubeadm-patches/
-or 
+or
 kubeadm upgrade node phase control-plane  --experimental-kustomize kubeadm-patches/
 ```
 
@@ -242,9 +245,9 @@ when beta is reached. `-k` abbreviation can be reserved or even fully connected.
 #### Providing and storing Kustomize patches to kubeadm
 
 Before kubeadm init, Kustomize patches should be eventually provided to kubeadm
-by higher-level tools/users; patches should be defined in a custom location 
+by higher-level tools/users; patches should be defined in a custom location
 on the machine file system and this location could be passed to
-kubeadm init/join with a CLI flag. 
+kubeadm init/join with a CLI flag.
 
 In order to simplify the first implementation of this KEP, this proposal is assuming
 to use the same approach also for kubeadm join; this point could be reconsidered
@@ -258,7 +261,7 @@ for allowing kubeadm to preserve changes during the kubeadm upgrade workflow.
 
 In order to simplify the first implementation of this KEP, this proposal is assuming
 that Kustomize patches will remain stored in the custom location on the machine file system
-for the necessary time, and that this location will be passed to kubeadm upgrade with a CLI 
+for the necessary time, and that this location will be passed to kubeadm upgrade with a CLI
 flag; this point could be reconsidered in the future, by e.g. defining a method for
 allowing higher-level tools/users to define Kustomize patches using a new CRD.
 
@@ -272,14 +275,14 @@ can create confusion in the users.
 kubeadm maintainers should take care of making differences cristal clear in release notes
 and feature announcement:
 
-- The component configuration provides an abstraction well suited for most common use cases 
+- The component configuration provides an abstraction well suited for most common use cases
   that can be addressed with cluster-wide configurations on control-plane/etcd command line args only,
-  while “advanced configurations”/kustomize allows users full control of static 
+  while “advanced configurations”/kustomize allows users full control of static
   pod manifest generated by Kubeadm at node level.
 - The component configuration implicitly defines the main cluster variants the kubeadm team is
   committed to support and monitor in the Kubernetes test grid, while instead higher-level
-  tools/user are responsible for the security, conformance, consistency, of 
-  “advanced configurations” for control-plane/etcd static pod manifests. 
+  tools/user are responsible for the security, conformance, consistency, of
+  “advanced configurations” for control-plane/etcd static pod manifests.
 
 _Misleading expectations on the level of flexibility_
 In order to provide guarantee about kubeadm respecting “advanced configurations” during
@@ -288,8 +291,8 @@ around _what_ can be customized and _how_.
 
 Even if the proposed solution is based on the user feedback/issues, the kubeadm maintainers
 want to be sure the implementation is providing the expected level of flexibility and, in
-order to ensure that, we will wait for at least one K8s release cycle for the users to provide 
-feedback before moving forward in graduating the feature to beta. 
+order to ensure that, we will wait for at least one K8s release cycle for the users to provide
+feedback before moving forward in graduating the feature to beta.
 
 Similarly, the kubeadm maintainers should work with [`etcdadm`](https://github.com/kubernetes-sigs/etcdadm)
 project and [`addons`](https://github.com/kubernetes-sigs/cluster-addons) project
@@ -300,7 +303,7 @@ Static pod manifests are managed by kubeadm, while “advanced configurations”
 will be managed by users.
 
 It might happen that a static pod manifest during upgrades is changed by kubeadm in a way that
-some patches will not cleanly apply anymore. 
+some patches will not cleanly apply anymore.
 
 The kubeadm maintainers will work on release notes to make potential breaking changes more visible;
 additionally, upgrade instructions will be updated adding the recommendation to --dry-run and check
@@ -320,8 +323,8 @@ consistent state in case of errors.
 Add at least one periodic E2E test job exercising  “advanced configurations” during init,
 join and upgrades.
 
-Please note that, in accordance with the split of responsibilities defined in the previous paragraphs, 
-the new E2E test will focus _only_ on the mechanics of applying “advanced configurations”/kustomize 
+Please note that, in accordance with the split of responsibilities defined in the previous paragraphs,
+the new E2E test will focus _only_ on the mechanics of applying “advanced configurations”/kustomize
 patches, not on the possible combination of patches and nor on the security, conformance, consistency,
 of the resulting Kubernetes cluster.