Skip to content

Commit

Permalink
Change to add --config-dir flag
Browse files Browse the repository at this point in the history
  • Loading branch information
yuqi-zhang committed Jun 14, 2023
1 parent 5c31a2b commit 8e75faa
Showing 1 changed file with 11 additions and 10 deletions.
21 changes: 11 additions & 10 deletions keps/sig-node/3983-drop-in-configuration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Items marked with (R) are required *prior to targeting to a milestone / release*

## Summary

Add support for a drop-in configuration directory for the Kubelet. This directory will be in `/etc/kubernetes/kubelet.conf.d` and configuration files will be processed in alphanumeric order. Establishment of conventions for configuration processing will be done, and further work can be done to add this support for other components.
Add support for a drop-in configuration directory for the Kubelet. This directory can be specified via a "--config-dir" flag, and configuration files will be processed in alphanumeric order. The flag will be empty by default and if not specified, drop-in support will not be enabled. Establishment of conventions for configuration processing will be done, and further work can be done to add this support for other components.


## Motivation
Expand All @@ -75,28 +75,29 @@ Components in Kubernetes can similarly be configured by multiple entities and pr

### Goals

* Add support for a `/etc/kubernetes/kubelet.conf.d` directory, which will override the configuration for the Kubelet located at `/etc/kubernetes/kubelet.conf`
* Add support for a "--config-dir" flag to the kubelet to allow users to specify a drop-in directory, which will override the configuration for the Kubelet located at `/etc/kubernetes/kubelet.conf`
* Extend API machinery code to reconcile multiple configuration files.
* Define Kubernetes best-practices for configuration definitions, similarly to [API conventions](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/api-conventions.md).

### Non-Goals

* Add support for drop-in configuration for Kubernetes components other than the Kubelet.
* Dynamically reconfiguring running kubelets if drop-in contents change.

## Proposal

This proposal aims to add support for a drop-in configuration directory for the kubelet in `/etc/kubernetes/kubelet.conf.d`. Users are able to specify individually configurable kubelet config snippets in files, formatted in the same way as the existing kubelet.conf file. The kubelet will process the configuration provided in `/etc/kubernetes/kubelet.conf.d` in alphanumeric order:
This proposal aims to add support for a drop-in configuration directory for the kubelet via specifying a "--config-dir" flag (for example, `/etc/kubernetes/kubelet.conf.d`). Users are able to specify individually configurable kubelet config snippets in files, formatted in the same way as the existing kubelet.conf file. The kubelet will process the configuration provided in the drop-in directory in alphanumeric order:


1. If no other configuration for the subfield(s) exist, append to the base configuration
2. If the subfield(s) exists in the base configuration at `/etc/kubernetes/kubelet.conf` file or another file in `/etc/kubernetes/kubelet.conf.d` with lesser alphanumeric ordering, overwrite it
2. If the subfield(s) exists in the base configuration at `/etc/kubernetes/kubelet.conf` file or another file in the drop-in directory with lesser alphanumeric ordering, overwrite it

* If the subfield(s) exist as a list, overwrite instead of attempting to merge. This makes it easier to delete items from lists defined in the base kubelet.conf or other drop-ins without having to modify other files.


If there are any issues with the drop-ins (e.g. formatting errors), the error will be reported in the same way as a misconfigured kubelet.conf file

This drop-in directory is purely optional and if empty/does not exist, the base configuration is used and no behavior changes will be introduced.
This drop-in directory is purely optional and if empty, the base configuration is used and no behavior changes will be introduced. The "--config-dir" flag will be empty by default and if not specified, drop-in support will not be enabled. This aims to align with "--config" flag defaults.


### User Stories
Expand Down Expand Up @@ -152,7 +153,7 @@ As a cluster admin, I would like to have cgroup management and log size manageme

### Upgrade / Downgrade Strategy

Upgrades and downgrades are safe as far as Kubelet stability is concerned. It’s possible a vendor may ship vital pieces of configuration within a drop-in directory. If the Kubelet downgrades to a version that doesn’t support reading the drop-in directory, it will only be configured with the original `/etc/kubelet/kubelet.conf`. However, assuming that vendor left that file in a valid state, the Kubelet should not be at risk of crashing. Other than vital configuration being missed, there’s no risk to the stability of the Kubelet on upgrades and downgrades.
Upgrades and downgrades are safe as far as Kubelet stability is concerned. It’s possible a vendor may ship vital pieces of configuration within a drop-in directory. If the Kubelet downgrades to a version that doesn’t support reading the drop-in directory, the kubelet will not recognize the "--config-dir" flag and risk failing. However, assuming that vendor left that the original `/etc/kubelet/kubelet.conf` is in a valid state, and the flag isn't specified, there should be no risk to the system. Any configuration that exists in a drop-in dir won't be applied, but that would not affect kubelet stability.


### Version Skew Strategy
Expand All @@ -168,22 +169,22 @@ All behavior change is encapsulated within the Kubelet, so there is no version s

###### How can this feature be enabled / disabled in a live cluster?

There will be no feature gate. On a live cluster, admins can disable by not using the drop-in configuration directory.
There will be no feature gate. On a live cluster, admins can disable by removing the flag.


###### Does enabling the feature change any default behavior?

Yes, upgrading to a version of the Kubelet with this feature will enable the Kubelet to be configured with the drop-in directory
No, upgrading to a version of the Kubelet with this feature will not enable the Kubelet to be configured with the drop-in directory if no flag is specified.


###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)?

No, only a rollback of the Kubelet can roll back the enablement.
Yes. Roll back and remove the `--config-dir` flag from the kubelet's CLI.


###### What happens if we reenable the feature if it was previously rolled back?

This will be a Kubelet upgrade, which will re-enable the feature as expected.
This feature will be re-enabled via adding back the `--config-dir` flag to the CLI, as mentioned above.


###### Are there any tests for feature enablement/disablement?
Expand Down

0 comments on commit 8e75faa

Please sign in to comment.