Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge main into dev-1.29 #44305

Merged
merged 21 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
7a8ed1c
docs: Small fixes to the Resource deletion section of API Concept
austince Oct 25, 2023
5c49e61
Changed yellow color to orange color for better visibility
PranitRout07 Oct 28, 2023
d174742
kubeadm: add section on how to use the "generate-csr" command
neolit123 Nov 9, 2023
bc7c1db
added a new concept page Sidecar containers and revise details detail…
T-Lakshmi Oct 6, 2023
7518e4e
[pt] Update "Operator White Paper" link on operator.md page
Arhell Dec 3, 2023
83b1334
Update service.md
rohan-kapse Dec 8, 2023
6edfcad
Fix Broken voyager link in ingress-controllers.md
Community-Programmer Dec 8, 2023
d2492b5
[zh-cn] sync secret cheatsheet php-apache.yaml
my-git9 Dec 9, 2023
c064885
Merge pull request #43719 from PranitRout07/test1
k8s-ci-robot Dec 9, 2023
f7ead26
Merge pull request #43689 from austince/patch-1
k8s-ci-robot Dec 9, 2023
2ceb4a4
Modified community index and added x-org image
Manish-hr14 Dec 9, 2023
f5ad2c5
Merge pull request #44287 from Manish-hr14/main
k8s-ci-robot Dec 10, 2023
2448333
Merge pull request #44274 from rohan-kapse/patch-7
k8s-ci-robot Dec 10, 2023
4813a41
Merge pull request #44276 from Community-Programmer/main
k8s-ci-robot Dec 11, 2023
43ecd32
Merge pull request #44285 from my-git9/patch-4ro4
k8s-ci-robot Dec 11, 2023
af8de40
Merge pull request #44200 from Arhell/pt-operator
k8s-ci-robot Dec 11, 2023
73e33f6
[zh-cn] Update KubeCon dates (#44280)
Fan-Lin Dec 11, 2023
b23c499
Revise details for Resource sharing within containers
T-Lakshmi Dec 11, 2023
0c5cb41
Merge pull request #43871 from neolit123/1.29-add-task-for-kubeadm-ge…
k8s-ci-robot Dec 11, 2023
9b9d765
Merge pull request #43346 from T-Lakshmi/sidecar-container
k8s-ci-robot Dec 11, 2023
e57cf32
Merge 'dev-1.29' with main
sftim Dec 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions content/en/community/_index.html
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ <h2>Discussions</h2>

<div id="twitter" class="community-resource">
<a href="https://twitter.com/kubernetesio">
<img src="/images/community/twitter.png" alt="Twitter">
<img src="/images/community/x-org.png" alt="𝕏.org">
</a>
<a href="https://twitter.com/kubernetesio">Twitter&nbsp;&#9654;</a>
<a href="https://twitter.com/kubernetesio">𝕏&nbsp;&#9654;</a>
<p><em>#kubernetesio</em></p>
<p>Real-time announcements of blog posts, events, news, ideas.</p>
</div>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Kubernetes as a project supports and maintains [AWS](https://github.com/kubernet
* The [Traefik Kubernetes Ingress provider](https://doc.traefik.io/traefik/providers/kubernetes-ingress/) is an
ingress controller for the [Traefik](https://traefik.io/traefik/) proxy.
* [Tyk Operator](https://github.com/TykTechnologies/tyk-operator) extends Ingress with Custom Resources to bring API Management capabilities to Ingress. Tyk Operator works with the Open Source Tyk Gateway & Tyk Cloud control plane.
* [Voyager](https://appscode.com/products/voyager) is an ingress controller for
* [Voyager](https://voyagermesh.com) is an ingress controller for
[HAProxy](https://www.haproxy.org/#desc).
* [Wallarm Ingress Controller](https://www.wallarm.com/solutions/waf-for-kubernetes) is an Ingress Controller that provides WAAP (WAF) and API Security capabilities.

Expand Down
5 changes: 1 addition & 4 deletions content/en/docs/concepts/services-networking/service.md
Original file line number Diff line number Diff line change
Expand Up @@ -887,10 +887,7 @@ finding a Service: environment variables and DNS.
When a Pod is run on a Node, the kubelet adds a set of environment variables
for each active Service. It adds `{SVCNAME}_SERVICE_HOST` and `{SVCNAME}_SERVICE_PORT` variables,
where the Service name is upper-cased and dashes are converted to underscores.
It also supports variables
(see [makeLinkVariables](https://github.com/kubernetes/kubernetes/blob/dd2d12f6dc0e654c15d5db57a5f9f6ba61192726/pkg/kubelet/envvars/envvars.go#L72))
that are compatible with Docker Engine's
"_[legacy container links](https://docs.docker.com/network/links/)_" feature.


For example, the Service `redis-primary` which exposes TCP port 6379 and has been
allocated cluster IP address 10.0.0.11, produces the following environment
Expand Down
86 changes: 36 additions & 50 deletions content/en/docs/concepts/workloads/pods/init-containers.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ Init containers can contain utilities or setup scripts not present in an app ima
You can specify init containers in the Pod specification alongside the `containers`
array (which describes app containers).

In Kubernetes, a [sidecar container](/docs/concepts/workloads/pods/sidecar-containers/) is a container that
starts before the main application container and _continues to run_. This document is about init containers:
containers that run to completion during Pod initialization.

<!-- body -->

Expand Down Expand Up @@ -48,14 +51,33 @@ including resource limits, [volumes](/docs/concepts/storage/volumes/), and secur
resource requests and limits for an init container are handled differently,
as documented in [Resource sharing within containers](#resource-sharing-within-containers).

Also, init containers do not support `lifecycle`, `livenessProbe`, `readinessProbe`, or
`startupProbe` because they must run to completion before the Pod can be ready.
Regular init containers (in other words: excluding sidecar containers) do not support the
`lifecycle`, `livenessProbe`, `readinessProbe`, or `startupProbe` fields. Init containers
must run to completion before the Pod can be ready; sidecar containers continue running
during a Pod's lifetime, and _do_ support some probes. See [sidecar container](/docs/concepts/workloads/pods/sidecar-containers/)
for further details about sidecar containers.

If you specify multiple init containers for a Pod, kubelet runs each init
container sequentially. Each init container must succeed before the next can run.
When all of the init containers have run to completion, kubelet initializes
the application containers for the Pod and runs them as usual.

### Differences from sidecar containers

Init containers run and complete their tasks before the main application container starts.
Unlike [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers),
init containers are not continuously running alongside the main containers.

Init containers run to completion sequentially, and the main container does not start
until all the init containers have successfully completed.

init containers do not support `lifecycle`, `livenessProbe`, `readinessProbe`, or
`startupProbe` whereas sidecar containers support all these [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe) to control their lifecycle.

Init containers share the same resources (CPU, memory, network) with the main application
containers but do not interact directly with them. They can, however, use shared volumes
for data exchange.

## Using init containers

Because init containers have separate images from app containers, they
Expand Down Expand Up @@ -289,51 +311,9 @@ The Pod which is already running correctly would be killed by `activeDeadlineSec
The name of each app and init container in a Pod must be unique; a
validation error is thrown for any container sharing a name with another.

#### API for sidecar containers

{{< feature-state for_k8s_version="v1.29" state="beta" >}}

Enabled by default with Kubernetes 1.29, a feature gate named `SidecarContainers`
allows you to specify a `restartPolicy` for init containers which is independent of
the Pod and other init containers. Container [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe)
can also be added to control their lifecycle.

If an init container is created with its `restartPolicy` set to `Always`, it will
start and remain running during the entire life of the Pod, which is useful for
running supporting services separated from the main application containers.

If a `readinessProbe` is specified for this init container, its result will be used
to determine the `ready` state of the Pod.

Since these containers are defined as init containers, they benefit from the same
ordering and sequential guarantees as other init containers, allowing them to
be mixed with other init containers into complex Pod initialization flows.

Compared to regular init containers, sidecar-style init containers continue to
run and the next init container can begin starting once the kubelet has set
the `started` container status for the sidecar-style init container to true.
That status either becomes true because there is a process running in the
container and no startup probe defined, or
as a result of its `startupProbe` succeeding.

This feature can be used to implement the sidecar container pattern in a more
robust way, as the kubelet always restarts a sidecar container if it fails.

Here's an example of a Deployment with two containers, one of which is a sidecar:

{{% code_sample language="yaml" file="application/deployment-sidecar.yaml" %}}

This feature is also useful for running Jobs with sidecars, as the sidecar
container will not prevent the Job from completing after the main container
has finished.

Here's an example of a Job with two containers, one of which is a sidecar:

{{% code_sample language="yaml" file="application/job/job-sidecar.yaml" %}}
Comment on lines -292 to -332
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved per #43346


#### Resource sharing within containers
### Resource sharing within containers

Given the ordering and execution for init containers, the following rules
Given the order of execution for init, sidecar and app containers, the following rules
for resource usage apply:

* The highest of any particular resource request or limit defined on all init
Expand All @@ -354,6 +334,10 @@ limit.
Pod level control groups (cgroups) are based on the effective Pod request and
limit, the same as the scheduler.

{{< comment >}}
This section also present under [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/) page.
If you're editing this section, change both places.
{{< /comment >}}

### Pod restart reasons

Expand All @@ -373,7 +357,9 @@ Kubernetes, consult the documentation for the version you are using.

## {{% heading "whatsnext" %}}

* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container)
* Learn how to [debug init containers](/docs/tasks/debug/debug-application/debug-init-containers/)
* Read about an overview of [kubelet](/docs/reference/command-line-tools-reference/kubelet/) and [kubectl](/docs/reference/kubectl/)
* Learn about the [types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
Learn more about the following:
* [Creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container).
* [Debug init containers](/docs/tasks/debug/debug-application/debug-init-containers/).
* Overview of [kubelet](/docs/reference/command-line-tools-reference/kubelet/) and [kubectl](/docs/reference/kubectl/).
* [Types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
* [Sidecar containers](/docs/concepts/workloads/pods/sidecar-containers).
23 changes: 18 additions & 5 deletions content/en/docs/concepts/workloads/pods/pod-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -150,11 +150,22 @@ the `Terminated` state.
The `spec` of a Pod has a `restartPolicy` field with possible values Always, OnFailure,
and Never. The default value is Always.

The `restartPolicy` applies to all containers in the Pod. `restartPolicy` only
refers to restarts of the containers by the kubelet on the same node. After containers
in a Pod exit, the kubelet restarts them with an exponential back-off delay (10s, 20s,
40s, …), that is capped at five minutes. Once a container has executed for 10 minutes
without any problems, the kubelet resets the restart backoff timer for that container.
The `restartPolicy` for a Pod applies to {{< glossary_tooltip text="app containers" term_id="app-container" >}}
in the Pod and to regular [init containers](/docs/concepts/workloads/pods/init-containers/).
[Sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/)
ignore the Pod-level `restartPolicy` field: in Kubernetes, a sidecar is defined as an
entry inside `initContainers` that has its container-level `restartPolicy` set to `Always`.
For init containers that exit with an error, the kubelet restarts the init container if
the Pod level `restartPolicy` is either `OnFailure` or `Always`.

When the kubelet is handling container restarts according to the configured restart
policy, that only applies to restarts that make replacement containers inside the
same Pod and running on the same node. After containers in a Pod exit, the kubelet
restarts them with an exponential back-off delay (10s, 20s,40s, …), that is capped at
five minutes. Once a container has executed for 10 minutes without any problems, the
kubelet resets the restart backoff timer for that container.
[Sidecar containers and Pod lifecycle](/docs/concepts/workloads/pods/sidecar-containers/#sidecar-containers-and-pod-lifecycle)
explains the behaviour of `init containers` when specify `restartpolicy` field on it.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this needs a fix at some point


## Pod conditions

Expand Down Expand Up @@ -600,6 +611,8 @@ for more details.

* Learn more about [container lifecycle hooks](/docs/concepts/containers/container-lifecycle-hooks/).

* Learn more about [sidecar containers](/docs/concepts/workloads/pods/sidecar-containers/).

* For detailed information about Pod and container status in the API, see
the API reference documentation covering
[`status`](/docs/reference/kubernetes-api/workload-resources/pod-v1/#PodStatus) for Pod.
126 changes: 126 additions & 0 deletions content/en/docs/concepts/workloads/pods/sidecar-containers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
---
title: Sidecar Containers
content_type: concept
weight: 50
---

<!-- overview -->
{{< feature-state for_k8s_version="v1.29" state="beta" >}}

Sidecar containers are the secondary containers that run along with the main
application container within the same {{< glossary_tooltip text="Pod" term_id="pod" >}}.
These containers are used to enhance or to extend the functionality of the main application
container by providing additional services, or functionality such as logging, monitoring,
security, or data synchronization, without directly altering the primary application code.

<!-- body -->

## Enabling sidecar containers

Starting with Kubernetes 1.28, a
[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) named
`SidecarContainers` allows you to specify a `restartPolicy` for containers listed in a
Pod's `initContainers` field. These restartable _sidecar_ containers are independent with
other [init containers](/docs/concepts/workloads/pods/init-containers/) and main
application container within the same pod. These can be started, stopped, or restarted
without effecting the main application container and other init containers.

## Sidecar containers and Pod lifecycle

If an init container is created with its `restartPolicy` set to `Always`, it will
start and remain running during the entire life of the Pod. This can be helpful for
running supporting services separated from the main application containers.

If a `readinessProbe` is specified for this init container, its result will be used
to determine the `ready` state of the Pod.

Since these containers are defined as init containers, they benefit from the same
ordering and sequential guarantees as other init containers, allowing them to
be mixed with other init containers into complex Pod initialization flows.

Compared to regular init containers, sidecars defined within `initContainers` continue to
run after they have started. This is important when there is more than one entry inside
`.spec.initContainers` for a Pod. After a sidecar-style init container is running (the kubelet
has set the `started` status for that init container to true), the kubelet then starts the
next init container from the ordered `.spec.initContainers` list.
That status either becomes true because there is a process running in the
container and no startup probe defined, or as a result of its `startupProbe` succeeding.

Here's an example of a Deployment with two containers, one of which is a sidecar:

{{% code_sample language="yaml" file="application/deployment-sidecar.yaml" %}}

This feature is also useful for running Jobs with sidecars, as the sidecar
container will not prevent the Job from completing after the main container
has finished.

Here's an example of a Job with two containers, one of which is a sidecar:

{{% code_sample language="yaml" file="application/job/job-sidecar.yaml" %}}
By default, this feature is not available in Kubernetes. To avail this feature, you
need to enable the [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
named `SidecarContainers`.

## Differences from regular containers

Sidecar containers run alongside regular containers in the same pod. However, they do not
execute the primary application logic; instead, they provide supporting functionality to
the main application.

Sidecar containers have their own independent lifecycles. They can be started, stopped,
and restarted independently of regular containers. This means you can update, scale, or
maintain sidecar containers without affecting the primary application.

Sidecar containers share the same network and storage namespaces with the primary
container This co-location allows them to interact closely and share resources.

## Differences from init containers

Sidecar containers work alongside the main container, extending its functionality and
providing additional services.

Sidecar containers run concurrently with the main application container. They are active
throughout the lifecycle of the pod and can be started and stopped independently of the
main container. Unlike [init containers](/docs/concepts/workloads/pods/init-containers/),
sidecar containers support [probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe) to control their lifecycle.

These containers can interact directly with the main application containers, sharing
the same network namespace, filesystem, and environment variables. They work closely
together to provide additional functionality.

## Resource sharing within containers

{{< comment >}}
This section is also present in the [init containers](/docs/concepts/workloads/pods/init-containers/) page.
If you're editing this section, change both places.
{{< /comment >}}

Given the order of execution for init, sidecar and app containers, the following rules
for resource usage apply:

* The highest of any particular resource request or limit defined on all init
containers is the *effective init request/limit*. If any resource has no
resource limit specified this is considered as the highest limit.
* The Pod's *effective request/limit* for a resource is the sum of
[pod overhead](/docs/concepts/scheduling-eviction/pod-overhead/) and the higher of:
* the sum of all non-init containers(app and sidecar containers) request/limit for a
resource
* the effective init request/limit for a resource
* Scheduling is done based on effective requests/limits, which means
init containers can reserve resources for initialization that are not used
during the life of the Pod.
* The QoS (quality of service) tier of the Pod's *effective QoS tier* is the
QoS tier for all init, sidecar and app containers alike.

Quota and limits are applied based on the effective Pod request and
limit.

Pod level control groups (cgroups) are based on the effective Pod request and
limit, the same as the scheduler.

## {{% heading "whatsnext" %}}

* Read a blog post on [native sidecar containers](/blog/2023/08/25/native-sidecar-containers/).
* Read about [creating a Pod that has an init container](/docs/tasks/configure-pod-container/configure-pod-initialization/#create-a-pod-that-has-an-init-container).
* Learn about the [types of probes](/docs/concepts/workloads/pods/pod-lifecycle/#types-of-probe): liveness, readiness, startup probe.
* Learn about [pod overhead](/docs/concepts/scheduling-eviction/pod-overhead/).
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,9 @@ For more details see
## kubeadm certs generate-csr {#cmd-certs-generate-csr}

This command can be used to generate keys and CSRs for all control-plane certificates and kubeconfig files.
The user can then sign the CSRs with a CA of their choice.
The user can then sign the CSRs with a CA of their choice. To read more information
on how to use the command see
[Signing certificate signing requests (CSR) generated by kubeadm](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#signing-csr).

{{< tabs name="tab-certs-generate-csr" >}}
{{< tab name="generate-csr" include="generated/kubeadm_certs_generate-csr.md" />}}
Expand Down
4 changes: 2 additions & 2 deletions content/en/docs/reference/using-api/api-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -722,13 +722,13 @@ When you **delete** a resource this takes place in two phases.
"kind": "ConfigMap",
"apiVersion": "v1",
"metadata": {
"finalizers": {"url.io/neat-finalization", "other-url.io/my-finalizer"},
"finalizers": ["url.io/neat-finalization", "other-url.io/my-finalizer"],
"deletionTimestamp": nil,
}
}
```

When a client first sends a **delete** to request removal of a resource, the `.metadata.deletionTimestamp` is set to the current time.
When a client first sends a **delete** to request the removal of a resource, the `.metadata.deletionTimestamp` is set to the current time.
Once the `.metadata.deletionTimestamp` is set, external controllers that act on finalizers
may start performing their cleanup work at any time, in any order.

Expand Down
Loading