-
Notifications
You must be signed in to change notification settings - Fork 14.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #28870 from shannonxtreme/gc-docs
Create garbage collection docs
- Loading branch information
Showing
10 changed files
with
726 additions
and
284 deletions.
There are no files selected for viewing
164 changes: 164 additions & 0 deletions
164
content/en/docs/concepts/architecture/garbage-collection.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,164 @@ | ||
--- | ||
title: Garbage Collection | ||
content_type: concept | ||
weight: 50 | ||
--- | ||
|
||
<!-- overview --> | ||
{{<glossary_definition term_id="garbage-collection" length="short">}} This | ||
allows the clean up of resources like the following: | ||
|
||
* [Failed pods](/docs/concepts/workloads/pods/pod-lifecycle/#pod-garbage-collection) | ||
* [Completed Jobs](/docs/concepts/workloads/controllers/ttlafterfinished/) | ||
* [Objects without owner references](#owners-dependents) | ||
* [Unused containers and container images](#containers-images) | ||
* [Dynamically provisioned PersistentVolumes with a StorageClass reclaim policy of Delete](/docs/concepts/storage/persistent-volumes/#delete) | ||
* [Stale or expired CertificateSigningRequests (CSRs)](/reference/access-authn-authz/certificate-signing-requests/#request-signing-process) | ||
* {{<glossary_tooltip text="Nodes" term_id="node">}} deleted in the following scenarios: | ||
* On a cloud when the cluster uses a [cloud controller manager](/docs/concepts/architecture/cloud-controller/) | ||
* On-premises when the cluster uses an addon similar to a cloud controller | ||
manager | ||
* [Node Lease objects](/docs/concepts/architecture/nodes/#heartbeats) | ||
|
||
## Owners and dependents {#owners-dependents} | ||
|
||
Many objects in Kubernetes link to each other through [*owner references*](/docs/concepts/overview/working-with-objects/owners-dependents/). | ||
Owner references tell the control plane which objects are dependent on others. | ||
Kubernetes uses owner references to give the control plane, and other API | ||
clients, the opportunity to clean up related resources before deleting an | ||
object. In most cases, Kubernetes manages owner references automatically. | ||
|
||
Ownership is different from the [labels and selectors](/docs/concepts/overview/working-with-objects/labels/) | ||
mechanism that some resources also use. For example, consider a | ||
{{<glossary_tooltip text="Service" term_id="service">}} that creates | ||
`EndpointSlice` objects. The Service uses *labels* to allow the control plane to | ||
determine which `EndpointSlice` objects are used for that Service. In addition | ||
to the labels, each `EndpointSlice` that is managed on behalf of a Service has | ||
an owner reference. Owner references help different parts of Kubernetes avoid | ||
interfering with objects they don’t control. | ||
|
||
## Cascading deletion {#cascading-deletion} | ||
|
||
Kubernetes checks for and deletes objects that no longer have owner | ||
references, like the pods left behind when you delete a ReplicaSet. When you | ||
delete an object, you can control whether Kubernetes deletes the object's | ||
dependents automatically, in a process called *cascading deletion*. There are | ||
two types of cascading deletion, as follows: | ||
|
||
* Foreground cascading deletion | ||
* Background cascading deletion | ||
|
||
You can also control how and when garbage collection deletes resources that have | ||
owner references using Kubernetes {{<glossary_tooltip text="finalizers" term_id="finalizer">}}. | ||
|
||
### Foreground cascading deletion {#foreground-deletion} | ||
|
||
In foreground cascading deletion, the owner object you're deleting first enters | ||
a *deletion in progress* state. In this state, the following happens to the | ||
owner object: | ||
|
||
* The Kubernetes API server sets the object's `metadata.deletionTimestamp` | ||
field to the time the object was marked for deletion. | ||
* The Kubernetes API server also sets the `metadata.finalizers` field to | ||
`foregroundDeletion`. | ||
* The object remains visible through the Kubernetes API until the deletion | ||
process is complete. | ||
|
||
After the owner object enters the deletion in progress state, the controller | ||
deletes the dependents. After deleting all the dependent objects, the controller | ||
deletes the owner object. At this point, the object is no longer visible in the | ||
Kubernetes API. | ||
|
||
During foreground cascading deletion, the only dependents that block owner | ||
deletion are those that have the `ownerReference.blockOwnerDeletion=true` field. | ||
See [Use foreground cascading deletion](/docs/tasks/administer-cluster/use-cascading-deletion/#use-foreground-cascading-deletion) | ||
to learn more. | ||
|
||
### Background cascading deletion {#background-deletion} | ||
|
||
In background cascading deletion, the Kubernetes API server deletes the owner | ||
object immediately and the controller cleans up the dependent objects in | ||
the background. By default, Kubernetes uses background cascading deletion unless | ||
you manually use foreground deletion or choose to orphan the dependent objects. | ||
|
||
See [Use background cascading deletion](/docs/tasks/administer-cluster/use-cascading-deletion/#use-background-cascading-deletion) | ||
to learn more. | ||
|
||
### Orphaned dependents | ||
|
||
When Kubernetes deletes an owner object, the dependents left behind are called | ||
*orphan* objects. By default, Kubernetes deletes dependent objects. To learn how | ||
to override this behaviour, see [Delete owner objects and orphan dependents](/docs/tasks/administer-cluster/use-cascading-deletion/#set-orphan-deletion-policy). | ||
|
||
## Garbage collection of unused containers and images {#containers-images} | ||
|
||
The {{<glossary_tooltip text="kubelet" term_id="kubelet">}} performs garbage | ||
collection on unused images every five minutes and on unused containers every | ||
minute. You should avoid using external garbage collection tools, as these can | ||
break the kubelet behavior and remove containers that should exist. | ||
|
||
To configure options for unused container and image garbage collection, tune the | ||
kubelet using a [configuration file](/docs/tasks/administer-cluster/kubelet-config-file/) | ||
and change the parameters related to garbage collection using the | ||
[`KubeletConfiguration`](/docs/reference/config-api/kubelet-config.v1beta1/#kubelet-config-k8s-io-v1beta1-KubeletConfiguration) | ||
resource type. | ||
|
||
### Container image lifecycle | ||
|
||
Kubernetes manages the lifecycle of all images through its *image manager*, | ||
which is part of the kubelet, with the cooperation of cadvisor. The kubelet | ||
considers the following disk usage limits when making garbage collection | ||
decisions: | ||
|
||
* `HighThresholdPercent` | ||
* `LowThresholdPercent` | ||
|
||
Disk usage above the configured `HighThresholdPercent` value triggers garbage | ||
collection, which deletes images in order based on the last time they were used, | ||
starting with the oldest first. The kubelet deletes images | ||
until disk usage reaches the `LowThresholdPercent` value. | ||
|
||
### Container image garbage collection {#container-image-garbage-collection} | ||
|
||
The kubelet garbage collects unused containers based on the following variables, | ||
which you can define: | ||
|
||
* `MinAge`: the minimum age at which the kubelet can garbage collect a | ||
container. Disable by setting to `0`. | ||
* `MaxPerPodContainer`: the maximum number of dead containers each Pod pair | ||
can have. Disable by setting to less than `0`. | ||
* `MaxContainers`: the maximum number of dead containers the cluster can have. | ||
Disable by setting to less than `0`. | ||
|
||
In addition to these variables, the kubelet garbage collects unidentified and | ||
deleted containers, typically starting with the oldest first. | ||
|
||
`MaxPerPodContainer` and `MaxContainer` may potentially conflict with each other | ||
in situations where retaining the maximum number of containers per Pod | ||
(`MaxPerPodContainer`) would go outside the allowable total of global dead | ||
containers (`MaxContainers`). In this situation, the kubelet adjusts | ||
`MaxPodPerContainer` to address the conflict. A worst-case scenario would be to | ||
downgrade `MaxPerPodContainer` to `1` and evict the oldest containers. | ||
Additionally, containers owned by pods that have been deleted are removed once | ||
they are older than `MinAge`. | ||
|
||
{{<note>}} | ||
The kubelet only garbage collects the containers it manages. | ||
{{</note>}} | ||
|
||
## Configuring garbage collection {#configuring-gc} | ||
|
||
You can tune garbage collection of resources by configuring options specific to | ||
the controllers managing those resources. The following pages show you how to | ||
configure garbage collection: | ||
|
||
* [Configuring cascading deletion of Kubernetes objects](/docs/tasks/administer-cluster/use-cascading-deletion/) | ||
* [Configuring cleanup of finished Jobs](/docs/concepts/workloads/controllers/ttlafterfinished/) | ||
|
||
<!-- * [Configuring unused container and image garbage collection](/docs/tasks/administer-cluster/reconfigure-kubelet/) --> | ||
|
||
## {{% heading "whatsnext" %}} | ||
|
||
* Learn more about [ownership of Kubernetes objects](/docs/concepts/overview/working-with-objects/owners-dependents/). | ||
* Learn more about Kubernetes [finalizers](/docs/concepts/overview/working-with-objects/finalizers/). | ||
* Learn about the [TTL controller](/docs/concepts/workloads/controllers/ttlafterfinished/) (beta) that cleans up finished Jobs. |
98 changes: 0 additions & 98 deletions
98
content/en/docs/concepts/cluster-administration/kubelet-garbage-collection.md
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
80 changes: 80 additions & 0 deletions
80
content/en/docs/concepts/overview/working-with-objects/finalizers.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,80 @@ | ||
--- | ||
title: Finalizers | ||
content_type: concept | ||
weight: 60 | ||
--- | ||
|
||
<!-- overview --> | ||
|
||
{{<glossary_definition term_id="finalizer" length="long">}} | ||
|
||
You can use finalizers to control {{<glossary_tooltip text="garbage collection" term_id="garbage-collection">}} | ||
of resources by alerting {{<glossary_tooltip text="controllers" term_id="controller">}} to perform specific cleanup tasks before | ||
deleting the target resource. | ||
|
||
Finalizers don't usually specify the code to execute. Instead, they are | ||
typically lists of keys on a specific resource similar to annotations. | ||
Kubernetes specifies some finalizers automatically, but you can also specify | ||
your own. | ||
|
||
## How finalizers work | ||
|
||
When you create a resource using a manifest file, you can specify finalizers in | ||
the `metadata.finalizers` field. When you attempt to delete the resource, the | ||
controller that manages it notices the values in the `finalizers` field and does | ||
the following: | ||
|
||
* Modifies the object to add a `metadata.deletionTimestamp` field with the | ||
time you started the deletion. | ||
* Marks the object as read-only until its `metadata.finalizers` field is empty. | ||
|
||
The controller then attempts to satisfy the requirements of the finalizers | ||
specified for that resource. Each time a finalizer condition is satisfied, the | ||
controller removes that key from the resource's `finalizers` field. When the | ||
field is empty, garbage collection continues. You can also use finalizers to | ||
prevent deletion of unmanaged resources. | ||
|
||
A common example of a finalizer is `kubernetes.io/pv-protection`, which prevents | ||
accidental deletion of `PersistentVolume` objects. When a `PersistentVolume` | ||
object is in use by a Pod, Kubernetes adds the `pv-protection` finalizer. If you | ||
try to delete the `PersistentVolume`, it enters a `Terminating` status, but the | ||
controller can't delete it because the finalizer exists. When the Pod stops | ||
using the `PersistentVolume`, Kubernetes clears the `pv-protection` finalizer, | ||
and the controller deletes the volume. | ||
|
||
## Owner references, labels, and finalizers {#owners-labels-finalizers} | ||
|
||
Like {{<glossary_tooltip text="labels" term_id="label">}}, [owner references](/concepts/overview/working-with-objects/owners-dependents/) | ||
describe the relationships between objects in Kubernetes, but are used for a | ||
different purpose. When a | ||
{{<glossary_tooltip text="controllers" term_id="controller">}} manages objects | ||
like Pods, it uses labels to track changes to groups of related objects. For | ||
example, when a {{<glossary_tooltip text="Job" term_id="job">}} creates one or | ||
more Pods, the Job controller applies labels to those pods and tracks changes to | ||
any Pods in the cluster with the same label. | ||
|
||
The Job controller also adds *owner references* to those Pods, pointing at the | ||
Job that created the Pods. If you delete the Job while these Pods are running, | ||
Kubernetes uses the owner references (not labels) to determine which Pods in the | ||
cluster need cleanup. | ||
|
||
Kubernetes also processes finalizers when it identifies owner references on a | ||
resource targeted for deletion. | ||
|
||
In some situations, finalizers can block the deletion of dependent objects, | ||
which can cause the targeted owner object to remain in a read-only state for | ||
longer than expected without being fully deleted. In these situations, you | ||
should check finalizers and owner references on the target owner and dependent | ||
objects to troubleshoot the cause. | ||
|
||
{{<note>}} | ||
In cases where objects are stuck in a deleting state, try to avoid manually | ||
removing finalizers to allow deletion to continue. Finalizers are usually added | ||
to resources for a reason, so forcefully removing them can lead to issues in | ||
your cluster. | ||
{{</note>}} | ||
|
||
## {{% heading "whatsnext" %}} | ||
|
||
* Read [Using Finalizers to Control Deletion](/blog/2021/05/14/using-finalizers-to-control-deletion/) | ||
on the Kubernetes blog. |
Oops, something went wrong.