From 694d3c7bdc0dd6989d8f4c601f6fd68f164a3f29 Mon Sep 17 00:00:00 2001 From: Qiming Teng Date: Thu, 15 Sep 2022 21:42:31 +0800 Subject: [PATCH] Create new page for feature gates that were removed As the upstream code evolves, many feature gates (deprecated or GA'ed) are removed. That means they are no longer recognized by the Kubernetes components. This PR moves such feature gates into a dedicated page for reference so that we won't bother revisit them again for future releases. --- .../feature-gates-removed.md | 668 ++++++++++++++++++ .../feature-gates.md | 560 +-------------- 2 files changed, 677 insertions(+), 551 deletions(-) create mode 100644 content/en/docs/reference/command-line-tools-reference/feature-gates-removed.md diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates-removed.md b/content/en/docs/reference/command-line-tools-reference/feature-gates-removed.md new file mode 100644 index 0000000000000..6a24e538f044d --- /dev/null +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates-removed.md @@ -0,0 +1,668 @@ +--- +title: Feature Gates (removed) +weight: 15 +content_type: concept +--- + + + +This page contains list of feature gates that have been removed. The information on this page is for reference. +A removed feature gate is different from a GA'ed or deprecated one in that a removed one is +no longer recognized as a valid feature gate. +However, a GA'ed or a deprecated feature gate is still recognized by the corresponding Kubernetes +components although they are unable to cause any behavior differences in a cluster. + +For feature gates that are still recognized by the Kubernetes components, please refer to +the [Alpha/Beta feature gate table](/docs/reference/command-line-tools/reference/feature-gates/#feature-gates-for-alpha-or-beta-features) +or the [Graduated/Deprecated feature gate table](/docs/reference/command-line-tools/reference/feature-gates/#feature-gates-for-graduated-or-deprecated-features) + +### Feature gates that are removed + +In the following table: + +- The "From" column contains the Kubernetes release when a feature is introduced + or its release stage is changed. +- The "To" column, if not empty, contains the last Kubernetes release in which + you can still use a feature gate. If the feature stage is either "Deprecated" + or "GA", the "To" column is the Kubernetes release when the feature is removed. + +{{< table caption="Feature Gates Removed" >}} + +| Feature | Default | Stage | From | To | +|---------|---------|-------|-------|-------| +| `Accelerators` | `false` | Alpha | 1.6 | 1.10 | +| `Accelerators` | - | Deprecated | 1.11 | 1.11 | +| `AffinityInAnnotations` | `false` | Alpha | 1.6 | 1.7 | +| `AffinityInAnnotations` | - | Deprecated | 1.8 | 1.8 | +| `AllowExtTrafficLocalEndpoints` | `false` | Beta | 1.4 | 1.6 | +| `AllowExtTrafficLocalEndpoints` | `true` | GA | 1.7 | 1.9 | +| `AttachVolumeLimit` | `false` | Alpha | 1.11 | 1.11 | +| `AttachVolumeLimit` | `true` | Beta | 1.12 | 1.16 | +| `AttachVolumeLimit` | `true` | GA | 1.17 | 1.21 | +| `BalanceAttachedNodeVolumes` | `false` | Alpha | 1.11 | 1.21 | +| `BalanceAttachedNodeVolumes` | `false` | Deprecated | 1.22 | 1.22 | +| `BlockVolume` | `false` | Alpha | 1.9 | 1.12 | +| `BlockVolume` | `true` | Beta | 1.13 | 1.17 | +| `BlockVolume` | `true` | GA | 1.18 | 1.21 | +| `BoundServiceAccountTokenVolume` | `false` | Alpha | 1.13 | 1.20 | +| `BoundServiceAccountTokenVolume` | `true` | Beta | 1.21 | 1.21 | +| `BoundServiceAccountTokenVolume` | `true` | GA | 1.22 | 1.23 | +| `CRIContainerLogRotation` | `false` | Alpha | 1.10 | 1.10 | +| `CRIContainerLogRotation` | `true` | Beta | 1.11 | 1.20 | +| `CRIContainerLogRotation` | `true` | GA | 1.21 | 1.22 | +| `CSIBlockVolume` | `false` | Alpha | 1.11 | 1.13 | +| `CSIBlockVolume` | `true` | Beta | 1.14 | 1.17 | +| `CSIBlockVolume` | `true` | GA | 1.18 | 1.21 | +| `CSIDriverRegistry` | `false` | Alpha | 1.12 | 1.13 | +| `CSIDriverRegistry` | `true` | Beta | 1.14 | 1.17 | +| `CSIDriverRegistry` | `true` | GA | 1.18 | 1.21 | +| `CSIMigrationAWSComplete` | `false` | Alpha | 1.17 | 1.20 | +| `CSIMigrationAWSComplete` | - | Deprecated | 1.21 | 1.21 | +| `CSIMigrationAzureDiskComplete` | `false` | Alpha | 1.17 | 1.20 | +| `CSIMigrationAzureDiskComplete` | - | Deprecated | 1.21 | 1.21 | +| `CSIMigrationAzureFileComplete` | `false` | Alpha | 1.17 | 1.20 | +| `CSIMigrationAzureFileComplete` | - | Deprecated | 1.21 | 1.21 | +| `CSIMigrationGCEComplete` | `false` | Alpha | 1.17 | 1.20 | +| `CSIMigrationGCEComplete` | - | Deprecated | 1.21 | 1.21 | +| `CSIMigrationOpenStackComplete` | `false` | Alpha | 1.17 | 1.20 | +| `CSIMigrationOpenStackComplete` | - | Deprecated | 1.21 | 1.21 | +| `CSIMigrationvSphereComplete` | `false` | Beta | 1.19 | 1.21 | +| `CSIMigrationvSphereComplete` | - | Deprecated | 1.22 | 1.22 | +| `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 | +| `CSINodeInfo` | `true` | Beta | 1.14 | 1.16 | +| `CSINodeInfo` | `true` | GA | 1.17 | 1.22 | +| `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 | +| `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 | +| `CSIPersistentVolume` | `true` | GA | 1.13 | 1.16 | +| `CSIServiceAccountToken` | `false` | Alpha | 1.20 | 1.20 | +| `CSIServiceAccountToken` | `true` | Beta | 1.21 | 1.21 | +| `CSIServiceAccountToken` | `true` | GA | 1.22 | 1.24 | +| `CSIVolumeFSGroupPolicy` | `false` | Alpha | 1.19 | 1.19 | +| `CSIVolumeFSGroupPolicy` | `true` | Beta | 1.20 | 1.22 | +| `CSIVolumeFSGroupPolicy` | `true` | GA | 1.23 | 1.25 | +| `ConfigurableFSGroupPolicy` | `false` | Alpha | 1.18 | 1.19 | +| `ConfigurableFSGroupPolicy` | `true` | Beta | 1.20 | 1.22 | +| `ConfigurableFSGroupPolicy` | `true` | GA | 1.23 | 1.25 | +| `CronJobControllerV2` | `false` | Alpha | 1.20 | 1.20 | +| `CronJobControllerV2` | `true` | Beta | 1.21 | 1.21 | +| `CronJobControllerV2` | `true` | GA | 1.22 | 1.23 | +| `CustomPodDNS` | `false` | Alpha | 1.9 | 1.9 | +| `CustomPodDNS` | `true` | Beta| 1.10 | 1.13 | +| `CustomPodDNS` | `true` | GA | 1.14 | 1.16 | +| `CustomResourceDefaulting` | `false` | Alpha| 1.15 | 1.15 | +| `CustomResourceDefaulting` | `true` | Beta | 1.16 | 1.16 | +| `CustomResourceDefaulting` | `true` | GA | 1.17 | 1.18 | +| `CustomResourcePublishOpenAPI` | `false` | Alpha| 1.14 | 1.14 | +| `CustomResourcePublishOpenAPI` | `true` | Beta| 1.15 | 1.15 | +| `CustomResourcePublishOpenAPI` | `true` | GA | 1.16 | 1.18 | +| `CustomResourceSubresources` | `false` | Alpha | 1.10 | 1.10 | +| `CustomResourceSubresources` | `true` | Beta | 1.11 | 1.15 | +| `CustomResourceSubresources` | `true` | GA | 1.16 | 1.18 | +| `CustomResourceValidation` | `false` | Alpha | 1.8 | 1.8 | +| `CustomResourceValidation` | `true` | Beta | 1.9 | 1.15 | +| `CustomResourceValidation` | `true` | GA | 1.16 | 1.18 | +| `CustomResourceWebhookConversion` | `false` | Alpha | 1.13 | 1.14 | +| `CustomResourceWebhookConversion` | `true` | Beta | 1.15 | 1.15 | +| `CustomResourceWebhookConversion` | `true` | GA | 1.16 | 1.18 | +| `DynamicAuditing` | `false` | Alpha | 1.13 | 1.18 | +| `DynamicAuditing` | - | Deprecated | 1.19 | 1.19 | +| `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 | +| `DynamicProvisioningScheduling` | - | Deprecated| 1.12 | - | +| `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 | +| `DynamicVolumeProvisioning` | `true` | GA | 1.8 | 1.12 | +| `EnableAggregatedDiscoveryTimeout` | `true` | Deprecated | 1.16 | 1.17 | +| `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | 1.12 | +| `EnableEquivalenceClassCache` | - | Deprecated | 1.13 | 1.23 | +| `EndpointSlice` | `false` | Alpha | 1.16 | 1.16 | +| `EndpointSlice` | `false` | Beta | 1.17 | 1.17 | +| `EndpointSlice` | `true` | Beta | 1.18 | 1.20 | +| `EndpointSlice` | `true` | GA | 1.21 | 1.24 | +| `EndpointSliceNodeName` | `false` | Alpha | 1.20 | 1.20 | +| `EndpointSliceNodeName` | `true` | GA | 1.21 | 1.24 | +| `EndpointSliceProxying` | `false` | Alpha | 1.18 | 1.18 | +| `EndpointSliceProxying` | `true` | Beta | 1.19 | 1.21 | +| `EndpointSliceProxying` | `true` | GA | 1.22 | 1.24 | +| `EvenPodsSpread` | `false` | Alpha | 1.16 | 1.17 | +| `EvenPodsSpread` | `true` | Beta | 1.18 | 1.18 | +| `EvenPodsSpread` | `true` | GA | 1.19 | 1.21 | +| `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | 1.12 | +| `ExperimentalCriticalPodAnnotation` | `false` | Deprecated | 1.13 | 1.16 | +| `ExternalPolicyForExternalIP` | `true` | GA | 1.18 | 1.22 | +| `GCERegionalPersistentDisk` | `true` | Beta | 1.10 | 1.12 | +| `GCERegionalPersistentDisk` | `true` | GA | 1.13 | 1.16 | +| `GenericEphemeralVolume` | `false` | Alpha | 1.19 | 1.20 | +| `GenericEphemeralVolume` | `true` | Beta | 1.21 | 1.22 | +| `GenericEphemeralVolume` | `true` | GA | 1.23 | 1.24 | +| `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | 1.18 | +| `HugePageStorageMediumSize` | `true` | Beta | 1.19 | 1.21 | +| `HugePageStorageMediumSize` | `true` | GA | 1.22 | 1.24 | +| `HugePages` | `false` | Alpha | 1.8 | 1.9 | +| `HugePages` | `true` | Beta| 1.10 | 1.13 | +| `HugePages` | `true` | GA | 1.14 | 1.16 | +| `HyperVContainer` | `false` | Alpha | 1.10 | 1.19 | +| `HyperVContainer` | `false` | Deprecated | 1.20 | 1.20 | +| `IPv6DualStack` | `false` | Alpha | 1.15 | 1.20 | +| `IPv6DualStack` | `true` | Beta | 1.21 | 1.22 | +| `IPv6DualStack` | `true` | GA | 1.23 | 1.24 | +| `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | 1.18 | +| `ImmutableEphemeralVolumes` | `true` | Beta | 1.19 | 1.20 | +| `ImmutableEphemeralVolumes` | `true` | GA | 1.21 | 1.24 | +| `IngressClassNamespacedParams` | `false` | Alpha | 1.21 | 1.21 | +| `IngressClassNamespacedParams` | `true` | Beta | 1.22 | 1.22 | +| `IngressClassNamespacedParams` | `true` | GA | 1.23 | 1.24 | +| `Initializers` | `false` | Alpha | 1.7 | 1.13 | +| `Initializers` | - | Deprecated | 1.14 | 1.14 | +| `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 | +| `KubeletConfigFile` | - | Deprecated | 1.10 | 1.10 | +| `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 | +| `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 | +| `KubeletPluginsWatcher` | `true` | GA | 1.13 | 1.16 | +| `LegacyNodeRoleBehavior` | `false` | Alpha | 1.16 | 1.18 | +| `LegacyNodeRoleBehavior` | `true` | Beta | 1.19 | 1.20 | +| `LegacyNodeRoleBehavior` | `false` | GA | 1.21 | 1.22 | +| `MountContainers` | `false` | Alpha | 1.9 | 1.16 | +| `MountContainers` | `false` | Deprecated | 1.17 | 1.17 | +| `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | +| `MountPropagation` | `true` | Beta | 1.10 | 1.11 | +| `MountPropagation` | `true` | GA | 1.12 | 1.14 | +| `NamespaceDefaultLabelName` | `true` | Beta | 1.21 | 1.21 | +| `NamespaceDefaultLabelName` | `true` | GA | 1.22 | 1.23 | +| `NodeDisruptionExclusion` | `false` | Alpha | 1.16 | 1.18 | +| `NodeDisruptionExclusion` | `true` | Beta | 1.19 | 1.20 | +| `NodeDisruptionExclusion` | `true` | GA | 1.21 | 1.22 | +| `NodeLease` | `false` | Alpha | 1.12 | 1.13 | +| `NodeLease` | `true` | Beta | 1.14 | 1.16 | +| `NodeLease` | `true` | GA | 1.17 | 1.23 | +| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | +| `PVCProtection` | - | Deprecated | 1.10 | 1.10 | +| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | +| `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | +| `PersistentLocalVolumes` | `true` | GA | 1.14 | 1.16 | +| `PodDisruptionBudget` | `false` | Alpha | 1.3 | 1.4 | +| `PodDisruptionBudget` | `true` | Beta | 1.5 | 1.20 | +| `PodDisruptionBudget` | `true` | GA | 1.21 | 1.25 | +| `PodPriority` | `false` | Alpha | 1.8 | 1.10 | +| `PodPriority` | `true` | Beta | 1.11 | 1.13 | +| `PodPriority` | `true` | GA | 1.14 | 1.18 | +| `PodReadinessGates` | `false` | Alpha | 1.11 | 1.11 | +| `PodReadinessGates` | `true` | Beta | 1.12 | 1.13 | +| `PodReadinessGates` | `true` | GA | 1.14 | 1.16 | +| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | +| `PodShareProcessNamespace` | `true` | Beta | 1.12 | 1.16 | +| `PodShareProcessNamespace` | `true` | GA | 1.17 | 1.19 | +| `RequestManagement` | `false` | Alpha | 1.15 | 1.16 | +| `RequestManagement` | - | Deprecated | 1.17 | 1.17 | +| `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | 1.18 | +| `ResourceLimitsPriorityFunction` | - | Deprecated | 1.19 | 1.19 | +| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 | +| `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | 1.16 | +| `ResourceQuotaScopeSelectors` | `true` | GA | 1.17 | 1.18 | +| `RootCAConfigMap` | `false` | Alpha | 1.13 | 1.19 | +| `RootCAConfigMap` | `true` | Beta | 1.20 | 1.20 | +| `RootCAConfigMap` | `true` | GA | 1.21 | 1.12 | +| `RotateKubeletClientCertificate` | `true` | Beta | 1.8 | 1.18 | +| `RotateKubeletClientCertificate` | `true` | GA | 1.19 | 1.21 | +| `RunAsGroup` | `true` | Beta | 1.14 | 1.20 | +| `RunAsGroup` | `true` | GA | 1.21 | 1.22 | +| `RuntimeClass` | `false` | Alpha | 1.12 | 1.13 | +| `RuntimeClass` | `true` | Beta | 1.14 | 1.19 | +| `RuntimeClass` | `true` | GA | 1.20 | 1.24 | +| `SCTPSupport` | `false` | Alpha | 1.12 | 1.18 | +| `SCTPSupport` | `true` | Beta | 1.19 | 1.19 | +| `SCTPSupport` | `true` | GA | 1.20 | 1.22 | +| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 | +| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | 1.16 | +| `ScheduleDaemonSetPods` | `true` | GA | 1.17 | 1.18 | +| `SelectorIndex` | `false` | Alpha | 1.18 | 1.18 | +| `SelectorIndex` | `true` | Beta | 1.19 | 1.19 | +| `SelectorIndex` | `true` | GA | 1.20 | 1.25 | +| `ServiceAccountIssuerDiscovery` | `false` | Alpha | 1.18 | 1.19 | +| `ServiceAccountIssuerDiscovery` | `true` | Beta | 1.20 | 1.20 | +| `ServiceAccountIssuerDiscovery` | `true` | GA | 1.21 | 1.23 | +| `ServiceAppProtocol` | `false` | Alpha | 1.18 | 1.18 | +| `ServiceAppProtocol` | `true` | Beta | 1.19 | 1.19 | +| `ServiceAppProtocol` | `true` | GA | 1.20 | 1.22 | +| `ServiceLoadBalancerFinalizer` | `false` | Alpha | 1.15 | 1.15 | +| `ServiceLoadBalancerFinalizer` | `true` | Beta | 1.16 | 1.16 | +| `ServiceLoadBalancerFinalizer` | `true` | GA | 1.17 | 1.20 | +| `ServiceNodeExclusion` | `false` | Alpha | 1.8 | 1.18 | +| `ServiceNodeExclusion` | `true` | Beta | 1.19 | 1.20 | +| `ServiceNodeExclusion` | `true` | GA | 1.21 | 1.22 | +| `ServiceTopology` | `false` | Alpha | 1.17 | 1.19 | +| `ServiceTopology` | `false` | Deprecated | 1.20 | 1.22 | +| `SetHostnameAsFQDN` | `false` | Alpha | 1.19 | 1.19 | +| `SetHostnameAsFQDN` | `true` | Beta | 1.20 | 1.21 | +| `SetHostnameAsFQDN` | `true` | GA | 1.22 | 1,24 | +| `StartupProbe` | `false` | Alpha | 1.16 | 1.17 | +| `StartupProbe` | `true` | Beta | 1.18 | 1.19 | +| `StartupProbe` | `true` | GA | 1.20 | 1.23 | +| `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 | +| `StorageObjectInUseProtection` | `true` | GA | 1.11 | 1.24 | +| `StreamingProxyRedirects` | `false` | Beta | 1.5 | 1.5 | +| `StreamingProxyRedirects` | `true` | Beta | 1.6 | 1.17 | +| `StreamingProxyRedirects` | `true` | Deprecated | 1.18 | 1.21 | +| `StreamingProxyRedirects` | `false` | Deprecated | 1.22 | 1.24 | +| `SupportIPVSProxyMode` | `false` | Alpha | 1.8 | 1.8 | +| `SupportIPVSProxyMode` | `false` | Beta | 1.9 | 1.9 | +| `SupportIPVSProxyMode` | `true` | Beta | 1.10 | 1.10 | +| `SupportIPVSProxyMode` | `true` | GA | 1.11 | 1.20 | +| `SupportNodePidsLimit` | `false` | Alpha | 1.14 | 1.14 | +| `SupportNodePidsLimit` | `true` | Beta | 1.15 | 1.19 | +| `SupportNodePidsLimit` | `true` | GA | 1.20 | 1.23 | +| `SupportPodPidsLimit` | `false` | Alpha | 1.10 | 1.13 | +| `SupportPodPidsLimit` | `true` | Beta | 1.14 | 1.19 | +| `SupportPodPidsLimit` | `true` | GA | 1.20 | 1.23 | +| `Sysctls` | `true` | Beta | 1.11 | 1.20 | +| `Sysctls` | `true` | GA | 1.21 | 1.22 | +| `TTLAfterFinished` | `false` | Alpha | 1.12 | 1.20 | +| `TTLAfterFinished` | `true` | Beta | 1.21 | 1.22 | +| `TTLAfterFinished` | `true` | GA | 1.23 | 1.24 | +| `TaintBasedEvictions` | `false` | Alpha | 1.6 | 1.12 | +| `TaintBasedEvictions` | `true` | Beta | 1.13 | 1.17 | +| `TaintBasedEvictions` | `true` | GA | 1.18 | 1.20 | +| `TaintNodesByCondition` | `false` | Alpha | 1.8 | 1.11 | +| `TaintNodesByCondition` | `true` | Beta | 1.12 | 1.16 | +| `TaintNodesByCondition` | `true` | GA | 1.17 | 1.18 | +| `TokenRequest` | `false` | Alpha | 1.10 | 1.11 | +| `TokenRequest` | `true` | Beta | 1.12 | 1.19 | +| `TokenRequest` | `true` | GA | 1.20 | 1.21 | +| `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 | +| `TokenRequestProjection` | `true` | Beta | 1.12 | 1.19 | +| `TokenRequestProjection` | `true` | GA | 1.20 | 1.21 | +| `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | +| `ValidateProxyRedirects` | `true` | Beta | 1.14 | 1.21 | +| `ValidateProxyRedirects` | `true` | Deprecated | 1.22 | 1.24 | +| `VolumePVCDataSource` | `false` | Alpha | 1.15 | 1.15 | +| `VolumePVCDataSource` | `true` | Beta | 1.16 | 1.17 | +| `VolumePVCDataSource` | `true` | GA | 1.18 | 1.21 | +| `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 | +| `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 | +| `VolumeScheduling` | `true` | GA | 1.13 | 1.16 | +| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | 1.16 | +| `VolumeSnapshotDataSource` | `true` | Beta | 1.17 | 1.19 | +| `VolumeSnapshotDataSource` | `true` | GA | 1.20 | 1.22 | +| `VolumeSubpath` | `true` | GA | 1.10 | 1.24 | +| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | 1.14 | +| `VolumeSubpathEnvExpansion` | `true` | Beta | 1.15 | 1.16 | +| `VolumeSubpathEnvExpansion` | `true` | GA | 1.17 | 1.24 | +| `WarningHeaders` | `true` | Beta | 1.19 | 1.21 | +| `WarningHeaders` | `true` | GA | 1.22 | 1.24 | +| `WindowsEndpointSliceProxying` | `false` | Alpha | 1.19 | 1.20 | +| `WindowsEndpointSliceProxying` | `true` | Beta | 1.21 | 1.21 | +| `WindowsEndpointSliceProxying` | `true` | GA | 1.22| 1.24 | +| `WindowsGMSA` | `false` | Alpha | 1.14 | 1.15 | +| `WindowsGMSA` | `true` | Beta | 1.16 | 1.17 | +| `WindowsGMSA` | `true` | GA | 1.18 | 1.20 | +| `WindowsRunAsUserName` | `false` | Alpha | 1.16 | 1.16 | +| `WindowsRunAsUserName` | `true` | Beta | 1.17 | 1.17 | +| `WindowsRunAsUserName` | `true` | GA | 1.18 | 1.20 | + +## Descriptions for removed feature gates + +- `Accelerators`: Provided an early form of plugin to enable Nvidia GPU support when using + Docker Engine; no longer available. See + [Device Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) for + an alternative. + +- `AffinityInAnnotations`: Enable setting + [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). + +- `AllowExtTrafficLocalEndpoints`: Enable a service to route external requests to node local endpoints. + +- `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes + that can be attached to a node. + See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) + for more details. + +- `BalanceAttachedNodeVolumes`: Include volume count on node to be considered for + balanced resource allocation while scheduling. A node which has closer CPU, + memory utilization, and volume count is favored by the scheduler while making decisions. + +- `BlockVolume`: Enable the definition and consumption of raw block devices in Pods. + See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) + for more details. + +- `BoundServiceAccountTokenVolume`: Migrate ServiceAccount volumes to use a projected volume + consisting of a ServiceAccountTokenVolumeProjection. Cluster admins can use metric + `serviceaccount_stale_tokens_total` to monitor workloads that are depending on the extended + tokens. If there are no such workloads, turn off extended tokens by starting `kube-apiserver` with + flag `--service-account-extend-token-expiration=false`. + Check [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) + for more details. + +- `CRIContainerLogRotation`: Enable container log rotation for CRI container runtime. + The default max size of a log file is 10MB and the default max number of + log files allowed for a container is 5. + These values can be configured in the kubelet config. + See [logging at node level](/docs/concepts/cluster-administration/logging/#logging-at-the-node-level) + for more details. + +- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. + See [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) + for more details. + +- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in + `csi.storage.k8s.io`. + +- `CSIMigrationAWSComplete`: Stops registering the EBS in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to + route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. + Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI + plugin installed and configured on all nodes in the cluster. This flag has + been deprecated in favor of the `InTreePluginAWSUnregister` feature flag + which prevents the registration of in-tree EBS plugin. + +- `CSIMigrationAzureDiskComplete`: Stops registering the Azure-Disk in-tree + plugin in kubelet and volume controllers and enables shims and translation + logic to route volume operations from the Azure-Disk in-tree plugin to + AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature + flags enabled and AzureDisk CSI plugin installed and configured on all nodes + in the cluster. This flag has been deprecated in favor of the + `InTreePluginAzureDiskUnregister` feature flag which prevents the registration + of in-tree AzureDisk plugin. + +- `CSIMigrationAzureFileComplete`: Stops registering the Azure-File in-tree + plugin in kubelet and volume controllers and enables shims and translation + logic to route volume operations from the Azure-File in-tree plugin to + AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature + flags enabled and AzureFile CSI plugin installed and configured on all nodes + in the cluster. This flag has been deprecated in favor of the + `InTreePluginAzureFileUnregister` feature flag which prevents the registration + of in-tree AzureFile plugin. + +- `CSIMigrationGCEComplete`: Stops registering the GCE-PD in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to + route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. + Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI + plugin installed and configured on all nodes in the cluster. This flag has + been deprecated in favor of the `InTreePluginGCEUnregister` feature flag which + prevents the registration of in-tree GCE PD plugin. + +- `CSIMigrationOpenStackComplete`: Stops registering the Cinder in-tree plugin in + kubelet and volume controllers and enables shims and translation logic to route + volume operations from the Cinder in-tree plugin to Cinder CSI plugin. + Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder + CSI plugin installed and configured on all nodes in the cluster. This flag has + been deprecated in favor of the `InTreePluginOpenStackUnregister` feature flag + which prevents the registration of in-tree openstack cinder plugin. + +- `CSIMigrationvSphereComplete`: Stops registering the vSphere in-tree plugin in kubelet + and volume controllers and enables shims and translation logic to route volume operations + from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and + CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and + configured on all nodes in the cluster. This flag has been deprecated in favor + of the `InTreePluginvSphereUnregister` feature flag which prevents the + registration of in-tree vsphere plugin. + +- `CSINodeInfo`: Enable all logic related to the CSINodeInfo API object in `csi.storage.k8s.io`. + +- `CSIPersistentVolume`: Enable discovering and mounting volumes provisioned through a + [CSI (Container Storage Interface)](https://git.k8s.io/design-proposals-archive/storage/container-storage-interface.md) + compatible volume plugin. + +- `CSIServiceAccountToken`: Enable CSI drivers to receive the pods' service account token + that they mount volumes for. See + [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html). + +- `CSIVolumeFSGroupPolicy`: Allows CSIDrivers to use the `fsGroupPolicy` field. + This field controls whether volumes created by a CSIDriver support volume ownership + and permission modifications when these volumes are mounted. + +- `ConfigurableFSGroupPolicy`: Allows user to configure volume permission change policy + for fsGroups when mounting a volume in a Pod. See + [Configure volume permission and ownership change policy for Pods](/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods) + for more details. + +- `CronJobControllerV2`: Use an alternative implementation of the + {{< glossary_tooltip text="CronJob" term_id="cronjob" >}} controller. Otherwise, + version 1 of the same controller is selected. + +- `CustomPodDNS`: Enable customizing the DNS settings for a Pod using its `dnsConfig` property. + Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config) + for more details. + +- `CustomResourceDefaulting`: Enable CRD support for default values in OpenAPI v3 validation schemas. + +- `CustomResourcePublishOpenAPI`: Enables publishing of CRD OpenAPI specs. + +- `CustomResourceSubresources`: Enable `/status` and `/scale` subresources + on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). + +- `CustomResourceValidation`: Enable schema based validation on resources created from + [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). + +- `CustomResourceWebhookConversion`: Enable webhook-based conversion + on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). + +- `DynamicAuditing`: Used to enable dynamic auditing before v1.19. + +- `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of + volume topology and handle PV provisioning. + This feature was superseded by the `VolumeScheduling` feature in v1.12. + +- `DynamicVolumeProvisioning`: Enable the + [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/) of persistent volumes to Pods. + +- `EnableAggregatedDiscoveryTimeout`: Enable the five second + timeout on aggregated discovery calls. + +- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of + nodes when scheduling Pods. + +- `EndpointSlice`: Enables EndpointSlices for more scalable and extensible + network endpoints. See [Enabling EndpointSlices](/docs/concepts/services-networking/endpoint-slices/). + +- `EndpointSliceNodeName`: Enables EndpointSlice `nodeName` field. + +- `EndpointSliceProxying`: When enabled, kube-proxy running + on Linux will use EndpointSlices as the primary data source instead of + Endpoints, enabling scalability and performance improvements. See + [Enabling Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/). + +- `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See + [Pod Topology Spread Constraints](/docs/concepts/scheduling-eviction/topology-spread-constraints/). + +- `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* + so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). + This feature is deprecated by Pod Priority and Preemption as of v1.13. + +- `ExternalPolicyForExternalIP`: Fix a bug where ExternalTrafficPolicy is not + applied to Service ExternalIPs. + +- `GCERegionalPersistentDisk`: Enable the regional PD feature on GCE. + +- `GenericEphemeralVolume`: Enables ephemeral, inline volumes that support all features + of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, + restore from snapshot, etc.). + See [Ephemeral Volumes](/docs/concepts/storage/ephemeral-volumes/). + +- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated + [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). + +- `HugePages`: Enable the allocation and consumption of pre-allocated + [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). + +- `HyperVContainer`: Enable + [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) + for Windows containers. + +- `IPv6DualStack`: Enable [dual stack](/docs/concepts/services-networking/dual-stack/) + support for IPv6. + +- `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as + immutable for better safety and performance. + +- `IngressClassNamespacedParams`: Allow namespace-scoped parameters reference in + `IngressClass` resource. This feature adds two fields - `Scope` and `Namespace` + to `IngressClass.spec.parameters`. + +- `Initializers`: Allow asynchronous coordination of object creation using the + Initializers admission plugin. + +- `KubeletConfigFile`: Enable loading kubelet configuration from + a file specified using a config file. + See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/) + for more details. + +- `KubeletPluginsWatcher`: Enable probe-based plugin watcher utility to enable kubelet + to discover plugins such as [CSI volume drivers](/docs/concepts/storage/volumes/#csi). + +- `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and + node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the + feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. + +- `MountContainers`: Enable using utility containers on host as the volume mounter. + +- `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods. + For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation). + +- `NamespaceDefaultLabelName`: Configure the API Server to set an immutable + {{< glossary_tooltip text="label" term_id="label" >}} `kubernetes.io/metadata.name` + on all namespaces, containing the namespace name. + +- `NodeDisruptionExclusion`: Enable use of the Node label `node.kubernetes.io/exclude-disruption` + which prevents nodes from being evacuated during zone failures. + +- `NodeLease`: Enable the new Lease API to report node heartbeats, which could be used as a node health signal. + +- `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from + being deleted when it is still used by any Pod. + +- `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods. + Pod affinity has to be specified if requesting a `local` volume. + +- `PodDisruptionBudget`: Enable the [PodDisruptionBudget](/docs/tasks/run-application/configure-pdb/) feature. + +- `PodPriority`: Enable the descheduling and preemption of Pods based on their + [priorities](/docs/concepts/scheduling-eviction/pod-priority-preemption/). + +- `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending + Pod readiness evaluation. See [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate) + for more details. + +- `PodShareProcessNamespace`: Enable the setting of `shareProcessNamespace` in a Pod for sharing + a single process namespace between containers running in a pod. More details can be found in + [Share Process Namespace between Containers in a Pod](/docs/tasks/configure-pod-container/share-process-namespace/). + +- `RequestManagement`: Enables managing request concurrency with prioritization and fairness + at each API server. Deprecated by `APIPriorityAndFairness` since 1.17. + +- `ResourceLimitsPriorityFunction`: Enable a scheduler priority function that + assigns a lowest possible score of 1 to a node that satisfies at least one of + the input Pod's cpu and memory limits. The intent is to break ties between + nodes with same scores. + +- `ResourceQuotaScopeSelectors`: Enable resource quota scope selectors. + +- `RootCAConfigMap`: Configure the `kube-controller-manager` to publish a + {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `kube-root-ca.crt` + to every namespace. This ConfigMap contains a CA bundle used for verifying connections + to the kube-apiserver. See + [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) + for more details. + +- `RotateKubeletClientCertificate`: Enable the rotation of the client TLS certificate on the kubelet. + See [kubelet configuration](/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/#kubelet-configuration) + for more details. + +- `RunAsGroup`: Enable control over the primary group ID set on the init processes of containers. + +- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature for + selecting container runtime configurations. + +- `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, Endpoints, EndpointSlice, + and NetworkPolicy definitions. + +- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler instead + of the DaemonSet controller. + +- `SelectorIndex`: Allows label and field based indexes in API server watch cache to accelerate + list operations. + +- `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the + service account issuer in the API server. See + [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) + for more details. + +- `ServiceAppProtocol`: Enables the `appProtocol` field on Services and Endpoints. + +- `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers. + +- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers created by a cloud provider. + A node is eligible for exclusion if labelled with "`node.kubernetes.io/exclude-from-external-load-balancers`". + +- `ServiceTopology`: Enable service to route traffic based upon the Node topology of the cluster. + See [ServiceTopology](/docs/concepts/services-networking/service-topology/) for more details. + +- `SetHostnameAsFQDN`: Enable the ability of setting Fully Qualified Domain Name(FQDN) as the + hostname of a pod. See + [Pod's `setHostnameAsFQDN` field](/docs/concepts/services-networking/dns-pod-service/#pod-sethostnameasfqdn-field). + +- `StartupProbe`: Enable the [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) + probe in the kubelet. + +- `StorageObjectInUseProtection`: Postpone the deletion of PersistentVolume or + PersistentVolumeClaim objects if they are still being used. + +- `StreamingProxyRedirects`: Instructs the API server to intercept (and follow) redirects from the + backend (kubelet) for streaming requests. Examples of streaming requests include the `exec`, + `attach` and `port-forward` requests. + +- `SupportIPVSProxyMode`: Enable providing in-cluster service load balancing using IPVS. + See [service proxies](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies) for more details. + +- `SupportNodePidsLimit`: Enable the support to limiting PIDs on the Node. The parameter + `pid=` in the `--system-reserved` and `--kube-reserved` options can be specified to + ensure that the specified number of process IDs will be reserved for the system as a whole and for + Kubernetes system daemons respectively. + +- `SupportPodPidsLimit`: Enable the support to limiting PIDs in Pods. + +- `Sysctls`: Enable support for namespaced kernel parameters (sysctls) that can be set for each + pod. See [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/) for more details. + +- `TTLAfterFinished`: Allow a [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) + to clean up resources after they finish execution. + +- `TaintBasedEvictions`: Enable evicting pods from nodes based on taints on Nodes and tolerations + on Pods. See [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) + for more details. + +- `TaintNodesByCondition`: Enable automatic tainting nodes based on + [node conditions](/docs/concepts/architecture/nodes/#condition). + +- `TokenRequest`: Enable the `TokenRequest` endpoint on service account resources. + +- `TokenRequestProjection`: Enable the injection of service account tokens into a Pod through a + [`projected` volume](/docs/concepts/storage/volumes/#projected). + +- `ValidateProxyRedirects`: This flag controls whether the API server should validate that redirects + are only followed to the same host. Only used if the `StreamingProxyRedirects` flag is enabled. + +- `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource. + +- `VolumeScheduling`: Enable volume topology aware scheduling and make the PersistentVolumeClaim + (PVC) binding aware of scheduling decisions. It also enables the usage of + [`local`](/docs/concepts/storage/volumes/#local) volume type when used together with the + `PersistentLocalVolumes` feature gate. + +- `VolumeSnapshotDataSource`: Enable volume snapshot data source support. + +- `VolumeSubpath`: Allow mounting a subpath of a volume in a container. + +- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment + variables into a `subPath`. + +- `WarningHeaders`: Allow sending warning headers in API responses. + +- `WindowsEndpointSliceProxying`: When enabled, kube-proxy running on Windows will use + EndpointSlices as the primary data source instead of Endpoints, enabling scalability and + performance improvements. See + [Enabling Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/). + +- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. + +- `WindowsRunAsUserName` : Enable support for running applications in Windows containers with as a + non-default user. See [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) + for more details. + diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index 41564e1e32d18..131fba6b774b0 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -21,7 +21,6 @@ Feature gates are a set of key=value pairs that describe Kubernetes features. You can turn these features on or off using the `--feature-gates` command line flag on each Kubernetes component. - Each Kubernetes component lets you enable or disable a set of feature gates that are relevant to that component. Use `-h` flag to see a full set of feature gates for all components. @@ -46,6 +45,11 @@ different Kubernetes components. - The [Graduated/Deprecated feature gate table](#feature-gates-for-graduated-or-deprecated-features) also lists deprecated and withdrawn features. +{{< note >}} +For a reference to old feature gates that are removed, please refer to +[feature gates removed](/docs/reference/command-line-tools-reference/feature-gates-removed/). +{{< /note >}} + ### Feature gates for Alpha or Beta features {{< table caption="Feature gates for features in Alpha or Beta states" >}} @@ -207,35 +211,9 @@ different Kubernetes components. | Feature | Default | Stage | Since | Until | |---------|---------|-------|-------|-------| -| `Accelerators` | `false` | Alpha | 1.6 | 1.10 | -| `Accelerators` | - | Deprecated | 1.11 | - | | `AdvancedAuditing` | `false` | Alpha | 1.7 | 1.7 | | `AdvancedAuditing` | `true` | Beta | 1.8 | 1.11 | | `AdvancedAuditing` | `true` | GA | 1.12 | - | -| `AffinityInAnnotations` | `false` | Alpha | 1.6 | 1.7 | -| `AffinityInAnnotations` | - | Deprecated | 1.8 | - | -| `AllowExtTrafficLocalEndpoints` | `false` | Beta | 1.4 | 1.6 | -| `AllowExtTrafficLocalEndpoints` | `true` | GA | 1.7 | - | -| `AttachVolumeLimit` | `false` | Alpha | 1.11 | 1.11 | -| `AttachVolumeLimit` | `true` | Beta | 1.12 | 1.16 | -| `AttachVolumeLimit` | `true` | GA | 1.17 | - | -| `BalanceAttachedNodeVolumes` | `false` | Alpha | 1.11 | 1.21 | -| `BalanceAttachedNodeVolumes` | `false` | Deprecated | 1.22 | | -| `BlockVolume` | `false` | Alpha | 1.9 | 1.12 | -| `BlockVolume` | `true` | Beta | 1.13 | 1.17 | -| `BlockVolume` | `true` | GA | 1.18 | - | -| `BoundServiceAccountTokenVolume` | `false` | Alpha | 1.13 | 1.20 | -| `BoundServiceAccountTokenVolume` | `true` | Beta | 1.21 | 1.21 | -| `BoundServiceAccountTokenVolume` | `true` | GA | 1.22 | - | -| `CRIContainerLogRotation` | `false` | Alpha | 1.10 | 1.10 | -| `CRIContainerLogRotation` | `true` | Beta | 1.11 | 1.20 | -| `CRIContainerLogRotation` | `true` | GA | 1.21 | - | -| `CSIBlockVolume` | `false` | Alpha | 1.11 | 1.13 | -| `CSIBlockVolume` | `true` | Beta | 1.14 | 1.17 | -| `CSIBlockVolume` | `true` | GA | 1.18 | - | -| `CSIDriverRegistry` | `false` | Alpha | 1.12 | 1.13 | -| `CSIDriverRegistry` | `true` | Beta | 1.14 | 1.17 | -| `CSIDriverRegistry` | `true` | GA | 1.18 | - | | `CSIInlineVolume` | `false` | Alpha | 1.15 | 1.15 | | `CSIInlineVolume` | `true` | Beta | 1.16 | 1.24 | | `CSIInlineVolume` | `true` | GA | 1.25 | - | @@ -246,75 +224,27 @@ different Kubernetes components. | `CSIMigrationAWS` | `false` | Beta | 1.17 | 1.22 | | `CSIMigrationAWS` | `true` | Beta | 1.23 | 1.24 | | `CSIMigrationAWS` | `true` | GA | 1.25 | - | -| `CSIMigrationAWSComplete` | `false` | Alpha | 1.17 | 1.20 | -| `CSIMigrationAWSComplete` | - | Deprecated | 1.21 | - | | `CSIMigrationAzureDisk` | `false` | Alpha | 1.15 | 1.18 | | `CSIMigrationAzureDisk` | `false` | Beta | 1.19 | 1.22 | | `CSIMigrationAzureDisk` | `true` | Beta | 1.23 | 1.23 | | `CSIMigrationAzureDisk` | `true` | GA | 1.24 | | -| `CSIMigrationAzureDiskComplete` | `false` | Alpha | 1.17 | 1.20 | -| `CSIMigrationAzureDiskComplete` | - | Deprecated | 1.21 | - | -| `CSIMigrationAzureFileComplete` | `false` | Alpha | 1.17 | 1.20 | -| `CSIMigrationAzureFileComplete` | - | Deprecated | 1.21 | - | | `CSIMigrationGCE` | `false` | Alpha | 1.14 | 1.16 | | `CSIMigrationGCE` | `false` | Beta | 1.17 | 1.22 | | `CSIMigrationGCE` | `true` | Beta | 1.23 | 1.24 | | `CSIMigrationGCE` | `true` | GA | 1.25 | - | -| `CSIMigrationGCEComplete` | `false` | Alpha | 1.17 | 1.20 | -| `CSIMigrationGCEComplete` | - | Deprecated | 1.21 | - | | `CSIMigrationOpenStack` | `false` | Alpha | 1.14 | 1.17 | | `CSIMigrationOpenStack` | `true` | Beta | 1.18 | 1.23 | | `CSIMigrationOpenStack` | `true` | GA | 1.24 | | -| `CSIMigrationOpenStackComplete` | `false` | Alpha | 1.17 | 1.20 | -| `CSIMigrationOpenStackComplete` | - | Deprecated | 1.21 | - | -| `CSIMigrationvSphereComplete` | `false` | Beta | 1.19 | 1.21 | -| `CSIMigrationvSphereComplete` | - | Deprecated | 1.22 | - | -| `CSINodeInfo` | `false` | Alpha | 1.12 | 1.13 | -| `CSINodeInfo` | `true` | Beta | 1.14 | 1.16 | -| `CSINodeInfo` | `true` | GA | 1.17 | - | -| `CSIPersistentVolume` | `false` | Alpha | 1.9 | 1.9 | -| `CSIPersistentVolume` | `true` | Beta | 1.10 | 1.12 | -| `CSIPersistentVolume` | `true` | GA | 1.13 | - | -| `CSIServiceAccountToken` | `false` | Alpha | 1.20 | 1.20 | -| `CSIServiceAccountToken` | `true` | Beta | 1.21 | 1.21 | -| `CSIServiceAccountToken` | `true` | GA | 1.22 | - | | `CSIStorageCapacity` | `false` | Alpha | 1.19 | 1.20 | | `CSIStorageCapacity` | `true` | Beta | 1.21 | 1.23 | | `CSIStorageCapacity` | `true` | GA | 1.24 | - | -| `CSIVolumeFSGroupPolicy` | `false` | Alpha | 1.19 | 1.19 | -| `CSIVolumeFSGroupPolicy` | `true` | Beta | 1.20 | 1.22 | -| `CSIVolumeFSGroupPolicy` | `true` | GA | 1.23 | | | `CSRDuration` | `true` | Beta | 1.22 | 1.23 | | `CSRDuration` | `true` | GA | 1.24 | - | -| `ConfigurableFSGroupPolicy` | `false` | Alpha | 1.18 | 1.19 | -| `ConfigurableFSGroupPolicy` | `true` | Beta | 1.20 | 1.22 | -| `ConfigurableFSGroupPolicy` | `true` | GA | 1.23 | - | | `ControllerManagerLeaderMigration` | `false` | Alpha | 1.21 | 1.21 | | `ControllerManagerLeaderMigration` | `true` | Beta | 1.22 | 1.23 | | `ControllerManagerLeaderMigration` | `true` | GA | 1.24 | - | -| `CronJobControllerV2` | `false` | Alpha | 1.20 | 1.20 | -| `CronJobControllerV2` | `true` | Beta | 1.21 | 1.21 | -| `CronJobControllerV2` | `true` | GA | 1.22 | - | | `CronJobTimeZone` | `false` | Alpha | 1.24 | 1.24 | | `CronJobTimeZone` | `true` | Beta | 1.25 | | -| `CustomPodDNS` | `false` | Alpha | 1.9 | 1.9 | -| `CustomPodDNS` | `true` | Beta| 1.10 | 1.13 | -| `CustomPodDNS` | `true` | GA | 1.14 | - | -| `CustomResourceDefaulting` | `false` | Alpha| 1.15 | 1.15 | -| `CustomResourceDefaulting` | `true` | Beta | 1.16 | 1.16 | -| `CustomResourceDefaulting` | `true` | GA | 1.17 | - | -| `CustomResourcePublishOpenAPI` | `false` | Alpha| 1.14 | 1.14 | -| `CustomResourcePublishOpenAPI` | `true` | Beta| 1.15 | 1.15 | -| `CustomResourcePublishOpenAPI` | `true` | GA | 1.16 | - | -| `CustomResourceSubresources` | `false` | Alpha | 1.10 | 1.10 | -| `CustomResourceSubresources` | `true` | Beta | 1.11 | 1.15 | -| `CustomResourceSubresources` | `true` | GA | 1.16 | - | -| `CustomResourceValidation` | `false` | Alpha | 1.8 | 1.8 | -| `CustomResourceValidation` | `true` | Beta | 1.9 | 1.15 | -| `CustomResourceValidation` | `true` | GA | 1.16 | - | -| `CustomResourceWebhookConversion` | `false` | Alpha | 1.13 | 1.14 | -| `CustomResourceWebhookConversion` | `true` | Beta | 1.15 | 1.15 | -| `CustomResourceWebhookConversion` | `true` | GA | 1.16 | - | | `DaemonSetUpdateSurge` | `false` | Alpha | 1.21 | 1.21 | | `DaemonSetUpdateSurge` | `true` | Beta | 1.22 | 1.24 | | `DaemonSetUpdateSurge` | `true` | GA | 1.25 | - | @@ -327,36 +257,15 @@ different Kubernetes components. | `DryRun` | `false` | Alpha | 1.12 | 1.12 | | `DryRun` | `true` | Beta | 1.13 | 1.18 | | `DryRun` | `true` | GA | 1.19 | - | -| `DynamicAuditing` | `false` | Alpha | 1.13 | 1.18 | -| `DynamicAuditing` | - | Deprecated | 1.19 | - | | `DynamicKubeletConfig` | `false` | Alpha | 1.4 | 1.10 | | `DynamicKubeletConfig` | `true` | Beta | 1.11 | 1.21 | | `DynamicKubeletConfig` | `false` | Deprecated | 1.22 | - | -| `DynamicProvisioningScheduling` | `false` | Alpha | 1.11 | 1.11 | -| `DynamicProvisioningScheduling` | - | Deprecated| 1.12 | - | -| `DynamicVolumeProvisioning` | `true` | Alpha | 1.3 | 1.7 | -| `DynamicVolumeProvisioning` | `true` | GA | 1.8 | - | | `EfficientWatchResumption` | `false` | Alpha | 1.20 | 1.20 | | `EfficientWatchResumption` | `true` | Beta | 1.21 | 1.23 | | `EfficientWatchResumption` | `true` | GA | 1.24 | - | -| `EnableAggregatedDiscoveryTimeout` | `true` | Deprecated | 1.16 | - | -| `EnableEquivalenceClassCache` | `false` | Alpha | 1.8 | 1.14 | -| `EnableEquivalenceClassCache` | - | Deprecated | 1.15 | - | -| `EndpointSlice` | `false` | Alpha | 1.16 | 1.16 | -| `EndpointSlice` | `false` | Beta | 1.17 | 1.17 | -| `EndpointSlice` | `true` | Beta | 1.18 | 1.20 | -| `EndpointSlice` | `true` | GA | 1.21 | - | -| `EndpointSliceNodeName` | `false` | Alpha | 1.20 | 1.20 | -| `EndpointSliceNodeName` | `true` | GA | 1.21 | - | -| `EndpointSliceProxying` | `false` | Alpha | 1.18 | 1.18 | -| `EndpointSliceProxying` | `true` | Beta | 1.19 | 1.21 | -| `EndpointSliceProxying` | `true` | GA | 1.22 | - | | `EphemeralContainers` | `false` | Alpha | 1.16 | 1.22 | | `EphemeralContainers` | `true` | Beta | 1.23 | 1.24 | | `EphemeralContainers` | `true` | GA | 1.25 | - | -| `EvenPodsSpread` | `false` | Alpha | 1.16 | 1.17 | -| `EvenPodsSpread` | `true` | Beta | 1.18 | 1.18 | -| `EvenPodsSpread` | `true` | GA | 1.19 | - | | `ExecProbeTimeout` | `true` | GA | 1.20 | - | | `ExpandCSIVolumes` | `false` | Alpha | 1.14 | 1.15 | | `ExpandCSIVolumes` | `true` | Beta | 1.16 | 1.23 | @@ -367,225 +276,54 @@ different Kubernetes components. | `ExpandPersistentVolumes` | `false` | Alpha | 1.8 | 1.10 | | `ExpandPersistentVolumes` | `true` | Beta | 1.11 | 1.23 | | `ExpandPersistentVolumes` | `true` | GA | 1.24 |- | -| `ExperimentalCriticalPodAnnotation` | `false` | Alpha | 1.5 | 1.12 | -| `ExperimentalCriticalPodAnnotation` | `false` | Deprecated | 1.13 | - | -| `ExternalPolicyForExternalIP` | `true` | GA | 1.18 | - | -| `GCERegionalPersistentDisk` | `true` | Beta | 1.10 | 1.12 | -| `GCERegionalPersistentDisk` | `true` | GA | 1.13 | - | -| `GenericEphemeralVolume` | `false` | Alpha | 1.19 | 1.20 | -| `GenericEphemeralVolume` | `true` | Beta | 1.21 | 1.22 | -| `GenericEphemeralVolume` | `true` | GA | 1.23 | - | -| `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | 1.18 | -| `HugePageStorageMediumSize` | `true` | Beta | 1.19 | 1.21 | -| `HugePageStorageMediumSize` | `true` | GA | 1.22 | - | -| `HugePages` | `false` | Alpha | 1.8 | 1.9 | -| `HugePages` | `true` | Beta| 1.10 | 1.13 | -| `HugePages` | `true` | GA | 1.14 | - | -| `HyperVContainer` | `false` | Alpha | 1.10 | 1.19 | -| `HyperVContainer` | `false` | Deprecated | 1.20 | - | -| `IPv6DualStack` | `false` | Alpha | 1.15 | 1.20 | -| `IPv6DualStack` | `true` | Beta | 1.21 | 1.22 | -| `IPv6DualStack` | `true` | GA | 1.23 | - | | `IdentifyPodOS` | `false` | Alpha | 1.23 | 1.23 | | `IdentifyPodOS` | `true` | Beta | 1.24 | 1.24 | | `IdentifyPodOS` | `true` | GA | 1.25 | - | -| `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | 1.18 | -| `ImmutableEphemeralVolumes` | `true` | Beta | 1.19 | 1.20 | -| `ImmutableEphemeralVolumes` | `true` | GA | 1.21 | | | `IndexedJob` | `false` | Alpha | 1.21 | 1.21 | | `IndexedJob` | `true` | Beta | 1.22 | 1.23 | | `IndexedJob` | `true` | GA | 1.24 | - | -| `IngressClassNamespacedParams` | `false` | Alpha | 1.21 | 1.21 | -| `IngressClassNamespacedParams` | `true` | Beta | 1.22 | 1.22 | -| `IngressClassNamespacedParams` | `true` | GA | 1.23 | - | -| `Initializers` | `false` | Alpha | 1.7 | 1.13 | -| `Initializers` | - | Deprecated | 1.14 | - | -| `KubeletConfigFile` | `false` | Alpha | 1.8 | 1.9 | -| `KubeletConfigFile` | - | Deprecated | 1.10 | - | -| `KubeletPluginsWatcher` | `false` | Alpha | 1.11 | 1.11 | -| `KubeletPluginsWatcher` | `true` | Beta | 1.12 | 1.12 | -| `KubeletPluginsWatcher` | `true` | GA | 1.13 | - | -| `LegacyNodeRoleBehavior` | `false` | Alpha | 1.16 | 1.18 | -| `LegacyNodeRoleBehavior` | `true` | Beta | 1.19 | 1.20 | -| `LegacyNodeRoleBehavior` | `false` | GA | 1.21 | - | | `LocalStorageCapacityIsolation` | `false` | Alpha | 1.7 | 1.9 | | `LocalStorageCapacityIsolation` | `true` | Beta | 1.10 | 1.24 | | `LocalStorageCapacityIsolation` | `true` | GA | 1.25 | - | -| `MountContainers` | `false` | Alpha | 1.9 | 1.16 | -| `MountContainers` | `false` | Deprecated | 1.17 | - | -| `MountPropagation` | `false` | Alpha | 1.8 | 1.9 | -| `MountPropagation` | `true` | Beta | 1.10 | 1.11 | -| `MountPropagation` | `true` | GA | 1.12 | - | -| `NamespaceDefaultLabelName` | `true` | Beta | 1.21 | 1.21 | -| `NamespaceDefaultLabelName` | `true` | GA | 1.22 | - | | `NetworkPolicyEndPort` | `false` | Alpha | 1.21 | 1.21 | | `NetworkPolicyEndPort` | `true` | Beta | 1.22 | 1.24 | | `NetworkPolicyEndPort` | `true` | GA | 1.25 | - | -| `NodeDisruptionExclusion` | `false` | Alpha | 1.16 | 1.18 | -| `NodeDisruptionExclusion` | `true` | Beta | 1.19 | 1.20 | -| `NodeDisruptionExclusion` | `true` | GA | 1.21 | - | -| `NodeLease` | `false` | Alpha | 1.12 | 1.13 | -| `NodeLease` | `true` | Beta | 1.14 | 1.16 | -| `NodeLease` | `true` | GA | 1.17 | - | | `NonPreemptingPriority` | `false` | Alpha | 1.15 | 1.18 | | `NonPreemptingPriority` | `true` | Beta | 1.19 | 1.23 | | `NonPreemptingPriority` | `true` | GA | 1.24 | - | -| `PVCProtection` | `false` | Alpha | 1.9 | 1.9 | -| `PVCProtection` | - | Deprecated | 1.10 | - | -| `PersistentLocalVolumes` | `false` | Alpha | 1.7 | 1.9 | -| `PersistentLocalVolumes` | `true` | Beta | 1.10 | 1.13 | -| `PersistentLocalVolumes` | `true` | GA | 1.14 | - | | `PodAffinityNamespaceSelector` | `false` | Alpha | 1.21 | 1.21 | | `PodAffinityNamespaceSelector` | `true` | Beta | 1.22 | 1.23 | | `PodAffinityNamespaceSelector` | `true` | GA | 1.24 | - | -| `PodDisruptionBudget` | `false` | Alpha | 1.3 | 1.4 | -| `PodDisruptionBudget` | `true` | Beta | 1.5 | 1.20 | -| `PodDisruptionBudget` | `true` | GA | 1.21 | - | | `PodOverhead` | `false` | Alpha | 1.16 | 1.17 | | `PodOverhead` | `true` | Beta | 1.18 | 1.23 | | `PodOverhead` | `true` | GA | 1.24 | - | -| `PodPriority` | `false` | Alpha | 1.8 | 1.10 | -| `PodPriority` | `true` | Beta | 1.11 | 1.13 | -| `PodPriority` | `true` | GA | 1.14 | - | -| `PodReadinessGates` | `false` | Alpha | 1.11 | 1.11 | -| `PodReadinessGates` | `true` | Beta | 1.12 | 1.13 | -| `PodReadinessGates` | `true` | GA | 1.14 | - | | `PodSecurity` | `false` | Alpha | 1.22 | 1.22 | | `PodSecurity` | `true` | Beta | 1.23 | 1.24 | | `PodSecurity` | `true` | GA | 1.25 | | -| `PodShareProcessNamespace` | `false` | Alpha | 1.10 | 1.11 | -| `PodShareProcessNamespace` | `true` | Beta | 1.12 | 1.16 | -| `PodShareProcessNamespace` | `true` | GA | 1.17 | - | | `PreferNominatedNode` | `false` | Alpha | 1.21 | 1.21 | | `PreferNominatedNode` | `true` | Beta | 1.22 | 1.23 | | `PreferNominatedNode` | `true` | GA | 1.24 | - | | `RemoveSelfLink` | `false` | Alpha | 1.16 | 1.19 | | `RemoveSelfLink` | `true` | Beta | 1.20 | 1.23 | | `RemoveSelfLink` | `true` | GA | 1.24 | - | -| `RequestManagement` | `false` | Alpha | 1.15 | 1.16 | -| `RequestManagement` | - | Deprecated | 1.17 | - | -| `ResourceLimitsPriorityFunction` | `false` | Alpha | 1.9 | 1.18 | -| `ResourceLimitsPriorityFunction` | - | Deprecated | 1.19 | - | -| `ResourceQuotaScopeSelectors` | `false` | Alpha | 1.11 | 1.11 | -| `ResourceQuotaScopeSelectors` | `true` | Beta | 1.12 | 1.16 | -| `ResourceQuotaScopeSelectors` | `true` | GA | 1.17 | - | -| `RootCAConfigMap` | `false` | Alpha | 1.13 | 1.19 | -| `RootCAConfigMap` | `true` | Beta | 1.20 | 1.20 | -| `RootCAConfigMap` | `true` | GA | 1.21 | - | -| `RotateKubeletClientCertificate` | `true` | Beta | 1.8 | 1.18 | -| `RotateKubeletClientCertificate` | `true` | GA | 1.19 | - | -| `RunAsGroup` | `true` | Beta | 1.14 | 1.20 | -| `RunAsGroup` | `true` | GA | 1.21 | - | -| `RuntimeClass` | `false` | Alpha | 1.12 | 1.13 | -| `RuntimeClass` | `true` | Beta | 1.14 | 1.19 | -| `RuntimeClass` | `true` | GA | 1.20 | - | -| `SCTPSupport` | `false` | Alpha | 1.12 | 1.18 | -| `SCTPSupport` | `true` | Beta | 1.19 | 1.19 | -| `SCTPSupport` | `true` | GA | 1.20 | - | -| `ScheduleDaemonSetPods` | `false` | Alpha | 1.11 | 1.11 | -| `ScheduleDaemonSetPods` | `true` | Beta | 1.12 | 1.16 | -| `ScheduleDaemonSetPods` | `true` | GA | 1.17 | - | -| `SelectorIndex` | `false` | Alpha | 1.18 | 1.18 | -| `SelectorIndex` | `true` | Beta | 1.19 | 1.19 | -| `SelectorIndex` | `true` | GA | 1.20 | - | | `ServerSideApply` | `false` | Alpha | 1.14 | 1.15 | | `ServerSideApply` | `true` | Beta | 1.16 | 1.21 | | `ServerSideApply` | `true` | GA | 1.22 | - | -| `ServiceAccountIssuerDiscovery` | `false` | Alpha | 1.18 | 1.19 | -| `ServiceAccountIssuerDiscovery` | `true` | Beta | 1.20 | 1.20 | -| `ServiceAccountIssuerDiscovery` | `true` | GA | 1.21 | - | -| `ServiceAppProtocol` | `false` | Alpha | 1.18 | 1.18 | -| `ServiceAppProtocol` | `true` | Beta | 1.19 | 1.19 | -| `ServiceAppProtocol` | `true` | GA | 1.20 | - | | `ServiceLBNodePortControl` | `false` | Alpha | 1.20 | 1.21 | | `ServiceLBNodePortControl` | `true` | Beta | 1.22 | 1.23 | | `ServiceLBNodePortControl` | `true` | GA | 1.24 | - | | `ServiceLoadBalancerClass` | `false` | Alpha | 1.21 | 1.21 | | `ServiceLoadBalancerClass` | `true` | Beta | 1.22 | 1.23 | | `ServiceLoadBalancerClass` | `true` | GA | 1.24 | - | -| `ServiceLoadBalancerFinalizer` | `false` | Alpha | 1.15 | 1.15 | -| `ServiceLoadBalancerFinalizer` | `true` | Beta | 1.16 | 1.16 | -| `ServiceLoadBalancerFinalizer` | `true` | GA | 1.17 | - | -| `ServiceNodeExclusion` | `false` | Alpha | 1.8 | 1.18 | -| `ServiceNodeExclusion` | `true` | Beta | 1.19 | 1.20 | -| `ServiceNodeExclusion` | `true` | GA | 1.21 | - | -| `ServiceTopology` | `false` | Alpha | 1.17 | 1.19 | -| `ServiceTopology` | `false` | Deprecated | 1.20 | - | -| `SetHostnameAsFQDN` | `false` | Alpha | 1.19 | 1.19 | -| `SetHostnameAsFQDN` | `true` | Beta | 1.20 | 1.21 | -| `SetHostnameAsFQDN` | `true` | GA | 1.22 | - | -| `StartupProbe` | `false` | Alpha | 1.16 | 1.17 | -| `StartupProbe` | `true` | Beta | 1.18 | 1.19 | -| `StartupProbe` | `true` | GA | 1.20 | - | | `StatefulSetMinReadySeconds` | `false` | Alpha | 1.22 | 1.22 | | `StatefulSetMinReadySeconds` | `true` | Beta | 1.23 | 1.24 | | `StatefulSetMinReadySeconds` | `true` | GA | 1.25 | - | -| `StorageObjectInUseProtection` | `true` | Beta | 1.10 | 1.10 | -| `StorageObjectInUseProtection` | `true` | GA | 1.11 | - | -| `StreamingProxyRedirects` | `false` | Beta | 1.5 | 1.5 | -| `StreamingProxyRedirects` | `true` | Beta | 1.6 | 1.17 | -| `StreamingProxyRedirects` | `true` | Deprecated | 1.18 | 1.21 | -| `StreamingProxyRedirects` | `false` | Deprecated | 1.22 | - | -| `SupportIPVSProxyMode` | `false` | Alpha | 1.8 | 1.8 | -| `SupportIPVSProxyMode` | `false` | Beta | 1.9 | 1.9 | -| `SupportIPVSProxyMode` | `true` | Beta | 1.10 | 1.10 | -| `SupportIPVSProxyMode` | `true` | GA | 1.11 | - | -| `SupportNodePidsLimit` | `false` | Alpha | 1.14 | 1.14 | -| `SupportNodePidsLimit` | `true` | Beta | 1.15 | 1.19 | -| `SupportNodePidsLimit` | `true` | GA | 1.20 | - | -| `SupportPodPidsLimit` | `false` | Alpha | 1.10 | 1.13 | -| `SupportPodPidsLimit` | `true` | Beta | 1.14 | 1.19 | -| `SupportPodPidsLimit` | `true` | GA | 1.20 | - | | `SuspendJob` | `false` | Alpha | 1.21 | 1.21 | | `SuspendJob` | `true` | Beta | 1.22 | 1.23 | | `SuspendJob` | `true` | GA | 1.24 | - | -| `Sysctls` | `true` | Beta | 1.11 | 1.20 | -| `Sysctls` | `true` | GA | 1.21 | - | -| `TTLAfterFinished` | `false` | Alpha | 1.12 | 1.20 | -| `TTLAfterFinished` | `true` | Beta | 1.21 | 1.22 | -| `TTLAfterFinished` | `true` | GA | 1.23 | - | -| `TaintBasedEvictions` | `false` | Alpha | 1.6 | 1.12 | -| `TaintBasedEvictions` | `true` | Beta | 1.13 | 1.17 | -| `TaintBasedEvictions` | `true` | GA | 1.18 | - | -| `TaintNodesByCondition` | `false` | Alpha | 1.8 | 1.11 | -| `TaintNodesByCondition` | `true` | Beta | 1.12 | 1.16 | -| `TaintNodesByCondition` | `true` | GA | 1.17 | - | -| `TokenRequest` | `false` | Alpha | 1.10 | 1.11 | -| `TokenRequest` | `true` | Beta | 1.12 | 1.19 | -| `TokenRequest` | `true` | GA | 1.20 | - | -| `TokenRequestProjection` | `false` | Alpha | 1.11 | 1.11 | -| `TokenRequestProjection` | `true` | Beta | 1.12 | 1.19 | -| `TokenRequestProjection` | `true` | GA | 1.20 | - | -| `ValidateProxyRedirects` | `false` | Alpha | 1.12 | 1.13 | -| `ValidateProxyRedirects` | `true` | Beta | 1.14 | 1.21 | -| `ValidateProxyRedirects` | `true` | Deprecated | 1.22 | - | -| `VolumePVCDataSource` | `false` | Alpha | 1.15 | 1.15 | -| `VolumePVCDataSource` | `true` | Beta | 1.16 | 1.17 | -| `VolumePVCDataSource` | `true` | GA | 1.18 | - | -| `VolumeScheduling` | `false` | Alpha | 1.9 | 1.9 | -| `VolumeScheduling` | `true` | Beta | 1.10 | 1.12 | -| `VolumeScheduling` | `true` | GA | 1.13 | - | -| `VolumeSnapshotDataSource` | `false` | Alpha | 1.12 | 1.16 | -| `VolumeSnapshotDataSource` | `true` | Beta | 1.17 | 1.19 | -| `VolumeSnapshotDataSource` | `true` | GA | 1.20 | - | -| `VolumeSubpath` | `true` | GA | 1.10 | - | -| `VolumeSubpathEnvExpansion` | `false` | Alpha | 1.14 | 1.14 | -| `VolumeSubpathEnvExpansion` | `true` | Beta | 1.15 | 1.16 | -| `VolumeSubpathEnvExpansion` | `true` | GA | 1.17 | - | -| `WarningHeaders` | `true` | Beta | 1.19 | 1.21 | -| `WarningHeaders` | `true` | GA | 1.22 | - | | `WatchBookmark` | `false` | Alpha | 1.15 | 1.15 | | `WatchBookmark` | `true` | Beta | 1.16 | 1.16 | | `WatchBookmark` | `true` | GA | 1.17 | - | -| `WindowsEndpointSliceProxying` | `false` | Alpha | 1.19 | 1.20 | -| `WindowsEndpointSliceProxying` | `true` | Beta | 1.21 | 1.21 | -| `WindowsEndpointSliceProxying` | `true` | GA | 1.22| - | -| `WindowsGMSA` | `false` | Alpha | 1.14 | 1.15 | -| `WindowsGMSA` | `true` | Beta | 1.16 | 1.17 | -| `WindowsGMSA` | `true` | GA | 1.18 | - | -| `WindowsRunAsUserName` | `false` | Alpha | 1.16 | 1.16 | -| `WindowsRunAsUserName` | `true` | Beta | 1.17 | 1.17 | -| `WindowsRunAsUserName` | `true` | GA | 1.18 | - | {{< /table >}} ## Using a feature @@ -639,37 +377,13 @@ Each feature gate is designed for enabling/disabling a specific feature: - `APIServerIdentity`: Assign each API server an ID in a cluster. - `APIServerTracing`: Add support for distributed tracing in the API server. See [Traces for Kubernetes System Components](/docs/concepts/cluster-administration/system-traces) for more details. -- `Accelerators`: Provided an early form of plugin to enable Nvidia GPU support when using - Docker Engine; no longer available. See - [Device Plugins](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/) for - an alternative. - `AdvancedAuditing`: Enable [advanced auditing](/docs/tasks/debug/debug-cluster/audit/#advanced-audit) -- `AffinityInAnnotations`: Enable setting - [Pod affinity or anti-affinity](/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity). -- `AllowExtTrafficLocalEndpoints`: Enable a service to route external requests to node local endpoints. - `AllowInsecureBackendProxy`: Enable the users to skip TLS verification of kubelets on Pod log requests. - `AnyVolumeDataSource`: Enable use of any custom resource as the `DataSource` of a {{< glossary_tooltip text="PVC" term_id="persistent-volume-claim" >}}. - `AppArmor`: Enable use of AppArmor mandatory access control for Pods running on Linux nodes. See [AppArmor Tutorial](/docs/tutorials/security/apparmor/) for more details. -- `AttachVolumeLimit`: Enable volume plugins to report limits on number of volumes - that can be attached to a node. - See [dynamic volume limits](/docs/concepts/storage/storage-limits/#dynamic-volume-limits) - for more details. -- `BalanceAttachedNodeVolumes`: Include volume count on node to be considered for - balanced resource allocation while scheduling. A node which has closer CPU, - memory utilization, and volume count is favored by the scheduler while making decisions. -- `BlockVolume`: Enable the definition and consumption of raw block devices in Pods. - See [Raw Block Volume Support](/docs/concepts/storage/persistent-volumes/#raw-block-volume-support) - for more details. -- `BoundServiceAccountTokenVolume`: Migrate ServiceAccount volumes to use a projected volume - consisting of a ServiceAccountTokenVolumeProjection. Cluster admins can use metric - `serviceaccount_stale_tokens_total` to monitor workloads that are depending on the extended - tokens. If there are no such workloads, turn off extended tokens by starting `kube-apiserver` with - flag `--service-account-extend-token-expiration=false`. - Check [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) - for more details. - `ContainerCheckpoint`: Enables the kubelet `checkpoint` API. See [Kubelet Checkpoint API](/docs/reference/node/kubelet-checkpoint-api/) for more details. - `ControllerManagerLeaderMigration`: Enables Leader Migration for @@ -689,17 +403,6 @@ Each feature gate is designed for enabling/disabling a specific feature: This feature gate guards *a group* of CPUManager options whose quality level is beta. This feature gate will never graduate to stable. - `CPUManagerPolicyOptions`: Allow fine-tuning of CPUManager policies. -- `CRIContainerLogRotation`: Enable container log rotation for CRI container runtime. - The default max size of a log file is 10MB and the default max number of - log files allowed for a container is 5. - These values can be configured in the kubelet config. - See [logging at node level](/docs/concepts/cluster-administration/logging/#logging-at-the-node-level) - for more details. -- `CSIBlockVolume`: Enable external CSI volume drivers to support block storage. - See [`csi` raw block volume support](/docs/concepts/storage/volumes/#csi-raw-block-volume-support) - for more details. -- `CSIDriverRegistry`: Enable all logic related to the CSIDriver API object in - csi.storage.k8s.io. - `CSIInlineVolume`: Enable CSI Inline volumes support for pods. - `CSIMigration`: Enables shims and translation logic to route volume operations from in-tree plugins to corresponding pre-installed CSI plugins @@ -709,13 +412,6 @@ Each feature gate is designed for enabling/disabling a specific feature: the feature disabled or that do not have EBS CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. -- `CSIMigrationAWSComplete`: Stops registering the EBS in-tree plugin in - kubelet and volume controllers and enables shims and translation logic to - route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. - Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI - plugin installed and configured on all nodes in the cluster. This flag has - been deprecated in favor of the `InTreePluginAWSUnregister` feature flag - which prevents the registration of in-tree EBS plugin. - `CSIMigrationAzureDisk`: Enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Supports falling back to in-tree AzureDisk plugin for mount operations to @@ -723,14 +419,6 @@ Each feature gate is designed for enabling/disabling a specific feature: installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAzureDiskComplete`: Stops registering the Azure-Disk in-tree - plugin in kubelet and volume controllers and enables shims and translation - logic to route volume operations from the Azure-Disk in-tree plugin to - AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature - flags enabled and AzureDisk CSI plugin installed and configured on all nodes - in the cluster. This flag has been deprecated in favor of the - `InTreePluginAzureDiskUnregister` feature flag which prevents the registration - of in-tree AzureDisk plugin. - `CSIMigrationAzureFile`: Enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Supports falling back to in-tree AzureFile plugin for mount operations to @@ -738,14 +426,6 @@ Each feature gate is designed for enabling/disabling a specific feature: installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationAzureFileComplete`: Stops registering the Azure-File in-tree - plugin in kubelet and volume controllers and enables shims and translation - logic to route volume operations from the Azure-File in-tree plugin to - AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature - flags enabled and AzureFile CSI plugin installed and configured on all nodes - in the cluster. This flag has been deprecated in favor of the - `InTreePluginAzureFileUnregister` feature flag which prevents the registration - of in-tree AzureFile plugin. - `CSIMigrationGCE`: Enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Supports falling back to in-tree GCE plugin for mount operations to nodes that have the @@ -753,13 +433,6 @@ Each feature gate is designed for enabling/disabling a specific feature: Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationGCEComplete`: Stops registering the GCE-PD in-tree plugin in - kubelet and volume controllers and enables shims and translation logic to - route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. - Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI - plugin installed and configured on all nodes in the cluster. This flag has - been deprecated in favor of the `InTreePluginGCEUnregister` feature flag which - prevents the registration of in-tree GCE PD plugin. - `CSIMigrationOpenStack`: Enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Supports falling back to in-tree Cinder plugin for mount operations to nodes that have @@ -767,13 +440,6 @@ Each feature gate is designed for enabling/disabling a specific feature: configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationOpenStackComplete`: Stops registering the Cinder in-tree plugin in - kubelet and volume controllers and enables shims and translation logic to route - volume operations from the Cinder in-tree plugin to Cinder CSI plugin. - Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder - CSI plugin installed and configured on all nodes in the cluster. This flag has - been deprecated in favor of the `InTreePluginOpenStackUnregister` feature flag - which prevents the registration of in-tree openstack cinder plugin. - `csiMigrationRBD`: Enables shims and translation logic to route volume operations from the RBD in-tree plugin to Ceph RBD CSI plugin. Requires CSIMigration and csiMigrationRBD feature flags enabled and Ceph CSI plugin @@ -787,63 +453,28 @@ Each feature gate is designed for enabling/disabling a specific feature: Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled. -- `CSIMigrationvSphereComplete`: Stops registering the vSphere in-tree plugin in kubelet - and volume controllers and enables shims and translation logic to route volume operations - from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and - CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and - configured on all nodes in the cluster. This flag has been deprecated in favor - of the `InTreePluginvSphereUnregister` feature flag which prevents the - registration of in-tree vsphere plugin. - `CSIMigrationPortworx`: Enables shims and translation logic to route volume operations from the Portworx in-tree plugin to Portworx CSI plugin. Requires Portworx CSI driver to be installed and configured in the cluster. -- `CSINodeInfo`: Enable all logic related to the CSINodeInfo API object in `csi.storage.k8s.io`. - `CSINodeExpandSecret`: Enable passing secret authentication data to a CSI driver for use during a `NodeExpandVolume` CSI operation. -- `CSIPersistentVolume`: Enable discovering and mounting volumes provisioned through a - [CSI (Container Storage Interface)](https://git.k8s.io/design-proposals-archive/storage/container-storage-interface.md) - compatible volume plugin. -- `CSIServiceAccountToken`: Enable CSI drivers to receive the pods' service account token - that they mount volumes for. See - [Token Requests](https://kubernetes-csi.github.io/docs/token-requests.html). - `CSIStorageCapacity`: Enables CSI drivers to publish storage capacity information and the Kubernetes scheduler to use that information when scheduling pods. See [Storage Capacity](/docs/concepts/storage/storage-capacity/). Check the [`csi` volume type](/docs/concepts/storage/volumes/#csi) documentation for more details. -- `CSIVolumeFSGroupPolicy`: Allows CSIDrivers to use the `fsGroupPolicy` field. - This field controls whether volumes created by a CSIDriver support volume ownership - and permission modifications when these volumes are mounted. - `CSIVolumeHealth`: Enable support for CSI volume health monitoring on node. - `CSRDuration`: Allows clients to request a duration for certificates issued via the Kubernetes CSR API. -- `ConfigurableFSGroupPolicy`: Allows user to configure volume permission change policy - for fsGroups when mounting a volume in a Pod. See - [Configure volume permission and ownership change policy for Pods](/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods) - for more details. - `ContextualLogging`: When you enable this feature gate, Kubernetes components that support contextual logging add extra detail to log output. - `ControllerManagerLeaderMigration`: Enables leader migration for `kube-controller-manager` and `cloud-controller-manager`. -- `CronJobControllerV2`: Use an alternative implementation of the - {{< glossary_tooltip text="CronJob" term_id="cronjob" >}} controller. Otherwise, - version 1 of the same controller is selected. - `CronJobTimeZone`: Allow the use of the `timeZone` optional field in [CronJobs](/docs/concepts/workloads/controllers/cron-jobs/) - `CustomCPUCFSQuotaPeriod`: Enable nodes to change `cpuCFSQuotaPeriod` in [kubelet config](/docs/tasks/administer-cluster/kubelet-config-file/). - `CustomResourceValidationExpressions`: Enable expression language validation in CRD which will validate customer resource based on validation rules written in the `x-kubernetes-validations` extension. -- `CustomPodDNS`: Enable customizing the DNS settings for a Pod using its `dnsConfig` property. - Check [Pod's DNS Config](/docs/concepts/services-networking/dns-pod-service/#pods-dns-config) - for more details. -- `CustomResourceDefaulting`: Enable CRD support for default values in OpenAPI v3 validation schemas. -- `CustomResourcePublishOpenAPI`: Enables publishing of CRD OpenAPI specs. -- `CustomResourceSubresources`: Enable `/status` and `/scale` subresources - on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). -- `CustomResourceValidation`: Enable schema based validation on resources created from - [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). -- `CustomResourceWebhookConversion`: Enable webhook-based conversion - on resources created from [CustomResourceDefinition](/docs/concepts/extend-kubernetes/api-extension/custom-resources/). - `DaemonSetUpdateSurge`: Enables the DaemonSet workloads to maintain availability during update per node. See [Perform a Rolling Update on a DaemonSet](/docs/tasks/manage-daemon/update-daemon-set/). @@ -865,36 +496,16 @@ Each feature gate is designed for enabling/disabling a specific feature: [downward API](/docs/tasks/inject-data-application/downward-api-volume-expose-pod-information). - `DryRun`: Enable server-side [dry run](/docs/reference/using-api/api-concepts/#dry-run) requests so that validation, merging, and mutation can be tested without committing. -- `DynamicAuditing`: Used to enable dynamic auditing before v1.19. - `DynamicKubeletConfig`: Enable the dynamic configuration of kubelet. The feature is no longer supported outside of supported skew policy. The feature gate was removed from kubelet in 1.24. See [Reconfigure kubelet](/docs/tasks/administer-cluster/reconfigure-kubelet/). -- `DynamicProvisioningScheduling`: Extend the default scheduler to be aware of - volume topology and handle PV provisioning. - This feature is superseded by the `VolumeScheduling` feature completely in v1.12. -- `DynamicVolumeProvisioning`: Enable the - [dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/) of persistent volumes to Pods. -- `EfficientWatchResumption`: Allows for storage-originated bookmark (progress - notify) events to be delivered to the users. This is only applied to watch - operations. -- `EnableAggregatedDiscoveryTimeout`: Enable the five second - timeout on aggregated discovery calls. -- `EnableEquivalenceClassCache`: Enable the scheduler to cache equivalence of - nodes when scheduling Pods. -- `EndpointSlice`: Enables EndpointSlices for more scalable and extensible - network endpoints. See [Enabling EndpointSlices](/docs/concepts/services-networking/endpoint-slices/). -- `EndpointSliceNodeName`: Enables EndpointSlice `nodeName` field. -- `EndpointSliceProxying`: When enabled, kube-proxy running - on Linux will use EndpointSlices as the primary data source instead of - Endpoints, enabling scalability and performance improvements. See - [Enabling Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/). - `EndpointSliceTerminatingCondition`: Enables EndpointSlice `terminating` and `serving` condition fields. +- `EfficientWatchResumption`: Allows for storage-originated bookmark (progress + notify) events to be delivered to the users. This is only applied to watch operations. - `EphemeralContainers`: Enable the ability to add {{< glossary_tooltip text="ephemeral containers" term_id="ephemeral-container" >}} to running pods. -- `EvenPodsSpread`: Enable pods to be scheduled evenly across topology domains. See - [Pod Topology Spread Constraints](/docs/concepts/scheduling-eviction/topology-spread-constraints/). - `ExecProbeTimeout`: Ensure kubelet respects exec probe timeouts. This feature gate exists in case any of your existing workloads depend on a now-corrected fault where Kubernetes ignored exec probe timeouts. See @@ -908,21 +519,11 @@ Each feature gate is designed for enabling/disabling a specific feature: [Resizing an in-use PersistentVolumeClaim](/docs/concepts/storage/persistent-volumes/#resizing-an-in-use-persistentvolumeclaim). - `ExpandPersistentVolumes`: Enable the expanding of persistent volumes. See [Expanding Persistent Volumes Claims](/docs/concepts/storage/persistent-volumes/#expanding-persistent-volumes-claims). -- `ExperimentalCriticalPodAnnotation`: Enable annotating specific pods as *critical* - so that their [scheduling is guaranteed](/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/). - This feature is deprecated by Pod Priority and Preemption as of v1.13. - `ExperimentalHostUserNamespaceDefaulting`: Enabling the defaulting user namespace to host. This is for containers that are using other host namespaces, host mounts, or containers that are privileged or using specific non-namespaced capabilities (e.g. `MKNODE`, `SYS_MODULE` etc.). This should only be enabled if user namespace remapping is enabled in the Docker daemon. -- `ExternalPolicyForExternalIP`: Fix a bug where ExternalTrafficPolicy is not - applied to Service ExternalIPs. -- `GCERegionalPersistentDisk`: Enable the regional PD feature on GCE. -- `GenericEphemeralVolume`: Enables ephemeral, inline volumes that support all features - of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, - restore from snapshot, etc.). - See [Ephemeral Volumes](/docs/concepts/storage/ephemeral-volumes/). - `GracefulNodeShutdown`: Enables support for graceful shutdown in kubelet. During a system shutdown, kubelet will attempt to detect the shutdown event and gracefully terminate pods running on the node. See @@ -940,29 +541,13 @@ Each feature gate is designed for enabling/disabling a specific feature: metrics from individual containers in target pods. - `HPAScaleToZero`: Enables setting `minReplicas` to 0 for `HorizontalPodAutoscaler` resources when using custom or external metrics. -- `HugePages`: Enable the allocation and consumption of pre-allocated - [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). -- `HugePageStorageMediumSize`: Enable support for multiple sizes pre-allocated - [huge pages](/docs/tasks/manage-hugepages/scheduling-hugepages/). -- `HyperVContainer`: Enable - [Hyper-V isolation](https://docs.microsoft.com/en-us/virtualization/windowscontainers/manage-containers/hyperv-container) - for Windows containers. -- `IPv6DualStack`: Enable [dual stack](/docs/concepts/services-networking/dual-stack/) - support for IPv6. - `IPTablesOwnershipCleanup`: This causes kubelet to no longer create legacy IPTables rules. - `IdentifyPodOS`: Allows the Pod OS field to be specified. This helps in identifying the OS of the pod authoritatively during the API server admission time. In Kubernetes {{< skew currentVersion >}}, the allowed values for the `pod.spec.os.name` are `windows` and `linux`. -- `ImmutableEphemeralVolumes`: Allows for marking individual Secrets and ConfigMaps as - immutable for better safety and performance. - `IndexedJob`: Allows the [Job](/docs/concepts/workloads/controllers/job/) controller to manage Pod completions per completion index. -- `IngressClassNamespacedParams`: Allow namespace-scoped parameters reference in - `IngressClass` resource. This feature adds two fields - `Scope` and `Namespace` - to `IngressClass.spec.parameters`. -- `Initializers`: Allow asynchronous coordination of object creation using the - Initializers admission plugin. - `InTreePluginAWSUnregister`: Stops registering the aws-ebs in-tree plugin in kubelet and volume controllers. - `InTreePluginAzureDiskUnregister`: Stops registering the azuredisk in-tree plugin in kubelet @@ -992,17 +577,11 @@ Each feature gate is designed for enabling/disabling a specific feature: completions without relying on Pods remaining in the cluster indefinitely. The Job controller uses Pod finalizers and a field in the Job status to keep track of the finished Pods to count towards completion. -- `KubeletConfigFile`: Enable loading kubelet configuration from - a file specified using a config file. - See [setting kubelet parameters via a config file](/docs/tasks/administer-cluster/kubelet-config-file/) - for more details. - `KubeletCredentialProviders`: Enable kubelet exec credential providers for image pull credentials. - `KubeletInUserNamespace`: Enables support for running kubelet in a {{}}. See [Running Kubernetes Node Components as a Non-root User](/docs/tasks/administer-cluster/kubelet-in-userns/). -- `KubeletPluginsWatcher`: Enable probe-based plugin watcher utility to enable kubelet - to discover plugins such as [CSI volume drivers](/docs/concepts/storage/volumes/#csi). - `KubeletPodResources`: Enable the kubelet's pod resources gRPC endpoint. See [Support Device Monitoring](https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/606-compute-device-assignment/README.md) for more details. @@ -1011,9 +590,6 @@ Each feature gate is designed for enabling/disabling a specific feature: [resource allocation reporting](/docs/concepts/extend-kubernetes/compute-storage-net/device-plugins/#monitoring-device-plugin-resources) with informations about the allocatable resources, enabling clients to properly track the free compute resources on a node. -- `LegacyNodeRoleBehavior`: When disabled, legacy behavior in service load balancers and - node disruption will ignore the `node-role.kubernetes.io/master` label in favor of the - feature-specific labels provided by `NodeDisruptionExclusion` and `ServiceNodeExclusion`. - `KubeletTracing`: Add support for distributed tracing in the kubelet. When enabled, kubelet CRI interface and authenticated http servers are instrumented to generate OpenTelemetry trace spans. @@ -1047,22 +623,13 @@ Each feature gate is designed for enabling/disabling a specific feature: [Pod topology spread constraints](/docs/concepts/scheduling-eviction/topology-spread-constraints/). - `MixedProtocolLBService`: Enable using different protocols in the same `LoadBalancer` type Service instance. -- `MountContainers`: Enable using utility containers on host as the volume mounter. -- `MountPropagation`: Enable sharing volume mounted by one container to other containers or pods. - For more details, please see [mount propagation](/docs/concepts/storage/volumes/#mount-propagation). - `MultiCIDRRangeAllocator`: Enables the MultiCIDR range allocator. -- `NamespaceDefaultLabelName`: Configure the API Server to set an immutable - {{< glossary_tooltip text="label" term_id="label" >}} `kubernetes.io/metadata.name` - on all namespaces, containing the namespace name. - `NetworkPolicyEndPort`: Enable use of the field `endPort` in NetworkPolicy objects, allowing the selection of a port range instead of a single port. - `NetworkPolicyStatus`: Enable the `status` subresource for NetworkPolicy objects. - `NodeInclusionPolicyInPodTopologySpread`: Enable using `nodeAffinityPolicy` and `nodeTaintsPolicy` in [Pod topology spread constraints](/docs/concepts/scheduling-eviction/topology-spread-constraints/) when calculating pod topology spread skew. -- `NodeDisruptionExclusion`: Enable use of the Node label `node.kubernetes.io/exclude-disruption` - which prevents nodes from being evacuated during zone failures. -- `NodeLease`: Enable the new Lease API to report node heartbeats, which could be used as a node health signal. - `NodeOutOfServiceVolumeDetach`: When a Node is marked out-of-service using the `node.kubernetes.io/out-of-service` taint, Pods on the node will be forcefully deleted if they can not tolerate this taint, and the volume detach operations for Pods terminating @@ -1076,28 +643,17 @@ Each feature gate is designed for enabling/disabling a specific feature: - `OpenAPIV3`: Enables the API server to publish OpenAPI v3. - `PodDeletionCost`: Enable the [Pod Deletion Cost](/docs/concepts/workloads/controllers/replicaset/#pod-deletion-cost) feature which allows users to influence ReplicaSet downscaling order. -- `PersistentLocalVolumes`: Enable the usage of `local` volume type in Pods. - Pod affinity has to be specified if requesting a `local` volume. - `PodAffinityNamespaceSelector`: Enable the [Pod Affinity Namespace Selector](/docs/concepts/scheduling-eviction/assign-pod-node/#namespace-selector) and [CrossNamespacePodAffinity](/docs/concepts/policy/resource-quotas/#cross-namespace-pod-affinity-quota) quota scope features. - `PodAndContainerStatsFromCRI`: Configure the kubelet to gather container and pod stats from the CRI container runtime rather than gathering them from cAdvisor. -- `PodDisruptionBudget`: Enable the [PodDisruptionBudget](/docs/tasks/run-application/configure-pdb/) feature. - `PodDisruptionConditions`: Enables support for appending a dedicated pod condition indicating that the pod is being deleted due to a disruption. - `PodHasNetworkCondition`: Enable the kubelet to mark the [PodHasNetwork](/docs/concepts/workloads/pods/pod-lifecycle/#pod-has-network) condition on pods. - `PodOverhead`: Enable the [PodOverhead](/docs/concepts/scheduling-eviction/pod-overhead/) feature to account for pod overheads. -- `PodPriority`: Enable the descheduling and preemption of Pods based on their - [priorities](/docs/concepts/scheduling-eviction/pod-priority-preemption/). -- `PodReadinessGates`: Enable the setting of `PodReadinessGate` field for extending - Pod readiness evaluation. See [Pod readiness gate](/docs/concepts/workloads/pods/pod-lifecycle/#pod-readiness-gate) - for more details. - `PodSecurity`: Enables the `PodSecurity` admission plugin. -- `PodShareProcessNamespace`: Enable the setting of `shareProcessNamespace` in a Pod for sharing - a single process namespace between containers running in a pod. More details can be found in - [Share Process Namespace between Containers in a Pod](/docs/tasks/configure-pod-container/share-process-namespace/). - `PreferNominatedNode`: This flag tells the scheduler whether the nominated nodes will be checked first before looping through all the other nodes in the cluster. @@ -1109,8 +665,6 @@ Each feature gate is designed for enabling/disabling a specific feature: by setting the `procMount` field of a SecurityContext. - `ProxyTerminatingEndpoints`: Enable the kube-proxy to handle terminating endpoints when `ExternalTrafficPolicy=Local`. -- `PVCProtection`: Enable the prevention of a PersistentVolumeClaim (PVC) from - being deleted when it is still used by any Pod. - `QOSReserved`: Allows resource reservations at the QoS level preventing pods at lower QoS levels from bursting into resources requested at higher QoS levels (memory only for now). @@ -1127,42 +681,16 @@ Each feature gate is designed for enabling/disabling a specific feature: objects and collections. This field has been deprecated since the Kubernetes v1.16 release. When this feature is enabled, the `.metadata.selfLink` field remains part of the Kubernetes API, but is always unset. -- `RequestManagement`: Enables managing request concurrency with prioritization and fairness - at each API server. Deprecated by `APIPriorityAndFairness` since 1.17. -- `ResourceLimitsPriorityFunction`: Enable a scheduler priority function that - assigns a lowest possible score of 1 to a node that satisfies at least one of - the input Pod's cpu and memory limits. The intent is to break ties between - nodes with same scores. -- `ResourceQuotaScopeSelectors`: Enable resource quota scope selectors. - `RetroactiveDefaultStorageClass`: Allow assigning StorageClass to unbound PVCs retroactively. -- `RootCAConfigMap`: Configure the `kube-controller-manager` to publish a - {{< glossary_tooltip text="ConfigMap" term_id="configmap" >}} named `kube-root-ca.crt` - to every namespace. This ConfigMap contains a CA bundle used for verifying connections - to the kube-apiserver. See - [Bound Service Account Tokens](https://github.com/kubernetes/enhancements/blob/master/keps/sig-auth/1205-bound-service-account-tokens/README.md) - for more details. -- `RotateKubeletClientCertificate`: Enable the rotation of the client TLS certificate on the kubelet. - See [kubelet configuration](/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/#kubelet-configuration) - for more details. - `RotateKubeletServerCertificate`: Enable the rotation of the server TLS certificate on the kubelet. See [kubelet configuration](/docs/reference/access-authn-authz/kubelet-tls-bootstrapping/#kubelet-configuration) for more details. -- `RunAsGroup`: Enable control over the primary group ID set on the init - processes of containers. -- `RuntimeClass`: Enable the [RuntimeClass](/docs/concepts/containers/runtime-class/) feature - for selecting container runtime configurations. -- `SCTPSupport`: Enables the _SCTP_ `protocol` value in Pod, Service, - Endpoints, EndpointSlice, and NetworkPolicy definitions. - `SELinuxMountReadWriteOncePod`: Speed up container startup by mounting volumes with the correct SELinux label instead of changing each file on the volumes recursively. The initial implementation focused on ReadWriteOncePod volumes. -- `ScheduleDaemonSetPods`: Enable DaemonSet Pods to be scheduled by the default scheduler - instead of the DaemonSet controller. - `SeccompDefault`: Enables the use of `RuntimeDefault` as the default seccomp profile for all workloads. The seccomp profile is specified in the `securityContext` of a Pod and/or a Container. -- `SelectorIndex`: Allows label and field based indexes in API server watch - cache to accelerate list operations. - `SELinuxMountReadWriteOncePod`: Allows kubelet to mount volumes for a Pod directly with the right SELinux label instead of applying the SELinux label recursively on every file on the volume. @@ -1171,75 +699,27 @@ Each feature gate is designed for enabling/disabling a specific feature: - `ServerSideFieldValidation`: Enables server-side field validation. This means the validation of resource schema is performed at the API server side rather than the client side (for example, the `kubectl create` or `kubectl apply` command line). -- `ServiceAccountIssuerDiscovery`: Enable OIDC discovery endpoints (issuer and - JWKS URLs) for the service account issuer in the API server. See - [Configure Service Accounts for Pods](/docs/tasks/configure-pod-container/configure-service-account/#service-account-issuer-discovery) - for more details. -- `ServiceAppProtocol`: Enables the `appProtocol` field on Services and Endpoints. - `ServiceInternalTrafficPolicy`: Enables the `internalTrafficPolicy` field on Services - `ServiceLBNodePortControl`: Enables the `allocateLoadBalancerNodePorts` field on Services. - `ServiceLoadBalancerClass`: Enables the `loadBalancerClass` field on Services. See [Specifying class of load balancer implementation](/docs/concepts/services-networking/service/#load-balancer-class) for more details. -- `ServiceLoadBalancerFinalizer`: Enable finalizer protection for Service load balancers. -- `ServiceNodeExclusion`: Enable the exclusion of nodes from load balancers - created by a cloud provider. A node is eligible for exclusion if labelled with - "`node.kubernetes.io/exclude-from-external-load-balancers`". -- `ServiceTopology`: Enable service to route traffic based upon the Node - topology of the cluster. See - [ServiceTopology](/docs/concepts/services-networking/service-topology/) - for more details. - `ServiceIPStaticSubrange`: Enables a strategy for Services ClusterIP allocations, whereby the ClusterIP range is subdivided. Dynamic allocated ClusterIP addresses will be allocated preferently from the upper range allowing users to assign static ClusterIPs from the lower range with a low risk of collision. See [Avoiding collisions](/docs/concepts/services-networking/service/#avoiding-collisions) for more details. -- `SetHostnameAsFQDN`: Enable the ability of setting Fully Qualified Domain - Name(FQDN) as the hostname of a pod. See - [Pod's `setHostnameAsFQDN` field](/docs/concepts/services-networking/dns-pod-service/#pod-sethostnameasfqdn-field). - `SizeMemoryBackedVolumes`: Enable kubelets to determine the size limit for memory-backed volumes (mainly `emptyDir` volumes). -- `StartupProbe`: Enable the - [startup](/docs/concepts/workloads/pods/pod-lifecycle/#when-should-you-use-a-startup-probe) - probe in the kubelet. - `StatefulSetMinReadySeconds`: Allows `minReadySeconds` to be respected by the StatefulSet controller. -- `StorageObjectInUseProtection`: Postpone the deletion of PersistentVolume or - PersistentVolumeClaim objects if they are still being used. - `StorageVersionAPI`: Enable the [storage version API](/docs/reference/generated/kubernetes-api/{{< param "version" >}}/#storageversion-v1alpha1-internal-apiserver-k8s-io). - `StorageVersionHash`: Allow API servers to expose the storage version hash in the discovery. -- `StreamingProxyRedirects`: Instructs the API server to intercept (and follow) - redirects from the backend (kubelet) for streaming requests. - Examples of streaming requests include the `exec`, `attach` and `port-forward` requests. -- `SupportIPVSProxyMode`: Enable providing in-cluster service load balancing using IPVS. - See [service proxies](/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies) for more details. -- `SupportNodePidsLimit`: Enable the support to limiting PIDs on the Node. - The parameter `pid=` in the `--system-reserved` and `--kube-reserved` - options can be specified to ensure that the specified number of process IDs - will be reserved for the system as a whole and for Kubernetes system daemons - respectively. -- `SupportPodPidsLimit`: Enable the support to limiting PIDs in Pods. -- `SuspendJob`: Enable support to suspend and resume Jobs. See - [the Jobs docs](/docs/concepts/workloads/controllers/job/) for - more details. -- `Sysctls`: Enable support for namespaced kernel parameters (sysctls) that can be - set for each pod. See - [sysctls](/docs/tasks/administer-cluster/sysctl-cluster/) for more details. -- `TTLAfterFinished`: Allow a - [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) - to clean up resources after they finish execution. -- `TaintBasedEvictions`: Enable evicting pods from nodes based on taints on Nodes - and tolerations on Pods. - See [taints and tolerations](/docs/concepts/scheduling-eviction/taint-and-toleration/) - for more details. -- `TaintNodesByCondition`: Enable automatic tainting nodes based on - [node conditions](/docs/concepts/architecture/nodes/#condition). -- `TokenRequest`: Enable the `TokenRequest` endpoint on service account resources. -- `TokenRequestProjection`: Enable the injection of service account tokens into a - Pod through a [`projected` volume](/docs/concepts/storage/volumes/#projected). +- `SuspendJob`: Enable support to suspend and resume Jobs. For more details, see + [the Jobs docs](/docs/concepts/workloads/controllers/job/). - `TopologyAwareHints`: Enables topology aware routing based on topology hints in EndpointSlices. See [Topology Aware Hints](/docs/concepts/services-networking/topology-aware-hints/) for more @@ -1248,34 +728,12 @@ Each feature gate is designed for enabling/disabling a specific feature: assignments for different components in Kubernetes. See [Control Topology Management Policies on a node](/docs/tasks/administer-cluster/topology-manager/). - `UserNamespacesStatelessPodsSupport`: Enable user namespace support for stateless Pods. -- `ValidateProxyRedirects`: This flag controls whether the API server should - validate that redirects are only followed to the same host. Only used if the - `StreamingProxyRedirects` flag is enabled. - `VolumeCapacityPriority`: Enable support for prioritizing nodes in different topologies based on available PV capacity. -- `VolumePVCDataSource`: Enable support for specifying an existing PVC as a DataSource. -- `VolumeScheduling`: Enable volume topology aware scheduling and make the - PersistentVolumeClaim (PVC) binding aware of scheduling decisions. It also - enables the usage of [`local`](/docs/concepts/storage/volumes/#local) volume - type when used together with the `PersistentLocalVolumes` feature gate. -- `VolumeSnapshotDataSource`: Enable volume snapshot data source support. -- `VolumeSubpath`: Allow mounting a subpath of a volume in a container. -- `VolumeSubpathEnvExpansion`: Enable `subPathExpr` field for expanding environment - variables into a `subPath`. -- `WarningHeaders`: Allow sending warning headers in API responses. - `WatchBookmark`: Enable support for watch bookmark events. - `WinDSR`: Allows kube-proxy to create DSR loadbalancers for Windows. - `WinOverlay`: Allows kube-proxy to run in overlay mode for Windows. -- `WindowsEndpointSliceProxying`: When enabled, kube-proxy running on Windows - will use EndpointSlices as the primary data source instead of Endpoints, - enabling scalability and performance improvements. See - [Enabling Endpoint Slices](/docs/concepts/services-networking/endpoint-slices/). -- `WindowsGMSA`: Enables passing of GMSA credential specs from pods to container runtimes. - `WindowsHostProcessContainers`: Enables support for Windows HostProcess containers. -- `WindowsRunAsUserName` : Enable support for running applications in Windows containers - with as a non-default user. See - [Configuring RunAsUserName](/docs/tasks/configure-pod-container/configure-runasusername) - for more details. ## {{% heading "whatsnext" %}}