add KEP: Node Maintenance Lease

keps/sig-node/20191211-node-maintenance-lease.md

@@ -0,0 +1,180 @@
+---
+title: Node Maintenance Lease
+authors:
+  - "@michaelgugino"
+owning-sig: sig-cluster-lifecycle
+participating-sigs:
+  - sig-node
+reviewers:
+  - TBD
+approvers:
+  - TBD
+editor: TBD
+creation-date: 2019-12-11
+last-updated: 2019-12-19
+status: provisional
+see-also:
+  - "https://github.com/kubernetes/enhancements/issues/1403"
+---
+
+# Node Maintenance Lease
+
+## Table of Contents
+
+<!-- toc -->
+- [Release Signoff Checklist](#release-signoff-checklist)
+- [Summary](#summary)
+- [Motivation](#motivation)
+  - [Goals](#goals)
+  - [Non-Goals](#non-goals)
+- [Proposal](#proposal)
+  - [User Stories [optional]](#user-stories-optional)
+    - [Story 1](#story-1)
+    - [Story 2](#story-2)
+  - [Implementation Details/Notes/Constraints [optional]](#implementation-detailsnotesconstraints-optional)
+  - [Risks and Mitigations](#risks-and-mitigations)
+- [Design Details](#design-details)
+  - [Test Plan](#test-plan)
+  - [Graduation Criteria](#graduation-criteria)
+    - [Examples](#examples)
+      - [Alpha -&gt; Beta Graduation](#alpha---beta-graduation)
+      - [Beta -&gt; GA Graduation](#beta---ga-graduation)
+      - [Removing a deprecated flag](#removing-a-deprecated-flag)
+  - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
+  - [Version Skew Strategy](#version-skew-strategy)
+- [Implementation History](#implementation-history)
+- [Drawbacks [optional]](#drawbacks-optional)
+- [Alternatives [optional]](#alternatives-optional)
+- [Infrastructure Needed [optional]](#infrastructure-needed-optional)
+<!-- /toc -->
+
+## Release Signoff Checklist
+
+- [ ] kubernetes/enhancements issue in release milestone, which links to KEP (this should be a link to the KEP location in kubernetes/enhancements, not the initial KEP PR)
+- [ ] KEP approvers have set the KEP status to `implementable`
+- [ ] Design details are appropriately documented
+- [ ] Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
+- [ ] Graduation criteria is in place
+- [ ] "Implementation History" section is up-to-date for milestone
+- [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io]
+- [ ] Supporting documentation e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes
+
+## Summary
+
+Node Maintenance Leases are useful to coordinate disruptive actions
+controllers or administrators might wish to take against a node. This lease
+is a voluntary mechanism and allows coordination among different components
+without those components needing knowledge of each other.
+
+## Motivation
+
+Currently, there is no centralized way to inform a controller or user that
+another controller or user is performing a disruptive operation against a node.
+Examples of disruptive operations include:
+1. Rebooting a node
+1. Powering off a node for maintenance
+1. Draining or Cordoning a node
+1. Deleting/deprovisioning a node due to failed health checks
+
+### Goals
+
+1. Define a central point of coordination for various actors.
+1. Define high-level steps for well-behaved clients to implement.
+
+### Non-Goals
+
+Define actors which might utilize the lease.
+
+## Proposal
+
+Utilize the existing `Lease` built-in API in the API group `coordination.k8s.io`.
+Create a new Lease object per-node  with Name equal to Node name in a newly
+created dedicated namespace
+That namespace should be created automatically (similarly to "default" and
+"kube-system", probably by NodeController) and never be deleted (so that nodes
+don't require permission for it).
+
+The Lease object should be created automatically, and the ownerRef should
+be the corresponding node so it is removed when the node is deleted.
+
+### User Stories [optional]
+
+#### Story 1
+
+I have a configuration management system that requires the node to be online to perform configuration updates.
+
+I have an automated-snapshot system that stops an instance and takes a snapshot for backup / data retention purposes.
+
+I do not want snapshots to happen during an update.
+
+#### Story 2
+
+I am investigating a problem with the node. I don't want the node to be disrupted
+by automation while I'm performing my investigation or fixes, and I don't want
+to stop the automation components from acting on other nodes.
+
+### Implementation Details/Notes/Constraints
+
+#### Impact to the Kubelet
+
+None. The kubelet does not need to respond to this lease or interact with it in
+any way.  This is for other components to coordination actions they intend to
+take (or prevent) on the node that might be disruptive.
+
+#### Impact to Scheduleability
+
+None. The existence of acquiring of this lease does not imply any taints,
+annotations, labels, cordoning or draining will take place. The Lease object
+itself is not intended as a mechanism to start any automated process.
+
+A user/controller acquires the Lease object, and then that user/controller
+may add/remove taints, cordon or drain, if they choose to.  Other components
+may wish to integrate with the new Lease object to avoid conflict. EG:
+a component that ensures a taint is always present on a node might stop ensuring
+that taint if there is a current valid lease holder, but that implementation
+is out of scope here.
+
+#### Client Behavior
+
+1. Client has required RBAC permissions for new namespace and Lease object
+1. Client checks for the existence of a lease (to be backwards compatible)
+1. If the lease exists, confirm the lease is not currently held by another user
+1. Attempt to acquire the lease and set an appropriate LeaseDurationSeconds
+1. If acquired, perform the necessary operations
+1. Release the lease
+
+### Risks and Mitigations
+
+Clients might not behave appropriately.  We can mitigate this by providing
+a library with an easy to use interface.
+
+## Alternatives
+
+### Annotations on Node or elsewhere
+
+Use a designated annotation on the node object, or possibly
+elsewhere.  Drawbacks to this include excess updates being applied to the node
+object directly, which might have performance implications. Having such an
+annotation natively seems like an anti-pattern and could easily be disrupted
+by existing controllers/clients attempting to manage those annotations.
+
+### Field in Node Spec Or Status
+
+Extending the node API will be much more difficult.  This behavior is also
+somewhat orthogonal to the node itself.
+
+### Use existing system namespace for lease object
+
+We could utilize an existing system namespace for the lease object. The primary
+issue with this is name collision, especially in the kube-node-lease namespace.
+The kube-node-lease namespace utilizes the same 1-to-1 ratio of Lease objects to
+nodes and also shares node names.  Other existing namespaces might have
+undesirable RBAC requirements.
+
+### Use optional non-system namespace
+
+Administrators that wish to enable this functionality could create a namespace
+with a specific name.  Clients could use a Get-or-Create method to check the
+lease.  This will make installing components that support this
+functionality more difficult as RBAC will not be deterministic at install time
+of those components.