From c07ce392e4cabebd147b8fd22982c507c5d9ac0a Mon Sep 17 00:00:00 2001 From: Chris Henzie Date: Tue, 10 Oct 2023 07:55:28 -0700 Subject: [PATCH] Graduate ReadWriteOncePod to GA Included is a task for migrating existing PersistentVolumes to use ReadWriteOncePod, taken from the alpha blog post. --- .../concepts/storage/persistent-volumes.md | 26 ++- .../workloads/controllers/statefulset.md | 6 + .../feature-gates.md | 5 +- .../change-pv-access-mode-readwriteoncepod.md | 187 ++++++++++++++++++ .../configure-persistent-volume-storage.md | 6 + 5 files changed, 219 insertions(+), 11 deletions(-) create mode 100644 content/en/docs/tasks/administer-cluster/change-pv-access-mode-readwriteoncepod.md diff --git a/content/en/docs/concepts/storage/persistent-volumes.md b/content/en/docs/concepts/storage/persistent-volumes.md index 13b81f9c1f708..b6d18462c4abc 100644 --- a/content/en/docs/concepts/storage/persistent-volumes.md +++ b/content/en/docs/concepts/storage/persistent-volumes.md @@ -39,8 +39,8 @@ NFS, iSCSI, or a cloud-provider-specific storage system. A _PersistentVolumeClaim_ (PVC) is a request for storage by a user. It is similar to a Pod. Pods consume node resources and PVCs consume PV resources. Pods can request specific levels of resources (CPU and Memory). Claims can request specific -size and access modes (e.g., they can be mounted ReadWriteOnce, ReadOnlyMany or -ReadWriteMany, see [AccessModes](#access-modes)). +size and access modes (e.g., they can be mounted ReadWriteOnce, ReadOnlyMany, +ReadWriteMany, or ReadWriteOncePod, see [AccessModes](#access-modes)). While PersistentVolumeClaims allow a user to consume abstract storage resources, it is common that users need PersistentVolumes with varying properties, such as @@ -626,7 +626,8 @@ The access modes are: `ReadWriteOnce` : the volume can be mounted as read-write by a single node. ReadWriteOnce access - mode still can allow multiple pods to access the volume when the pods are running on the same node. + mode still can allow multiple pods to access the volume when the pods are + running on the same node. For single pod access, please see ReadWriteOncePod. `ReadOnlyMany` : the volume can be mounted as read-only by many nodes. @@ -635,15 +636,22 @@ The access modes are: : the volume can be mounted as read-write by many nodes. `ReadWriteOncePod` -: {{< feature-state for_k8s_version="v1.27" state="beta" >}} +: {{< feature-state for_k8s_version="v1.29" state="stable" >}} the volume can be mounted as read-write by a single Pod. Use ReadWriteOncePod access mode if you want to ensure that only one pod across the whole cluster can - read that PVC or write to it. This is only supported for CSI volumes and - Kubernetes version 1.22+. + read that PVC or write to it. -The blog article -[Introducing Single Pod Access Mode for PersistentVolumes](/blog/2021/09/13/read-write-once-pod-access-mode-alpha/) -covers this in more detail. +{{< note >}} +The `ReadWriteOncePod` access mode is only supported for +{{< glossary_tooltip text="CSI" term_id="csi" >}} volumes and Kubernetes version +1.22+. To use this feature you will need to update the following +[CSI sidecars](https://kubernetes-csi.github.io/docs/sidecar-containers.html) +to these versions or greater: + +* [csi-provisioner:v3.0.0+](https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.0.0) +* [csi-attacher:v3.3.0+](https://github.com/kubernetes-csi/external-attacher/releases/tag/v3.3.0) +* [csi-resizer:v1.3.0+](https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.3.0) +{{< /note >}} In the CLI, the access modes are abbreviated to: diff --git a/content/en/docs/concepts/workloads/controllers/statefulset.md b/content/en/docs/concepts/workloads/controllers/statefulset.md index 927b2e53f3ea3..b0176108cea75 100644 --- a/content/en/docs/concepts/workloads/controllers/statefulset.md +++ b/content/en/docs/concepts/workloads/controllers/statefulset.md @@ -116,6 +116,12 @@ spec: storage: 1Gi ``` +{{< note >}} +This example uses the `ReadWriteOnce` access mode, for simplicity. For +production use, the Kubernetes project recommends using the `ReadWriteOncePod` +access mode instead. +{{< /note >}} + In the above example: * A Headless Service, named `nginx`, is used to control the network domain. diff --git a/content/en/docs/reference/command-line-tools-reference/feature-gates.md b/content/en/docs/reference/command-line-tools-reference/feature-gates.md index f5ff4c9a85748..a897300b063a6 100644 --- a/content/en/docs/reference/command-line-tools-reference/feature-gates.md +++ b/content/en/docs/reference/command-line-tools-reference/feature-gates.md @@ -170,8 +170,6 @@ For a reference to old feature gates that are removed, please refer to | `PodSchedulingReadiness` | `true` | Beta | 1.27 | | | `ProcMountType` | `false` | Alpha | 1.12 | | | `QOSReserved` | `false` | Alpha | 1.11 | | -| `ReadWriteOncePod` | `false` | Alpha | 1.22 | 1.26 | -| `ReadWriteOncePod` | `true` | Beta | 1.27 | | | `RecoverVolumeExpansionFailure` | `false` | Alpha | 1.23 | | | `RemainingItemCount` | `false` | Alpha | 1.15 | 1.15 | | `RemainingItemCount` | `true` | Beta | 1.16 | | @@ -311,6 +309,9 @@ For a reference to old feature gates that are removed, please refer to | `ProxyTerminatingEndpoints` | `false` | Alpha | 1.22 | 1.25 | | `ProxyTerminatingEndpoints` | `true` | Beta | 1.26 | 1.27 | | `ProxyTerminatingEndpoints` | `true` | GA | 1.28 | | +| `ReadWriteOncePod` | `false` | Alpha | 1.22 | 1.26 | +| `ReadWriteOncePod` | `true` | Beta | 1.27 | 1.28 | +| `ReadWriteOncePod` | `true` | GA | 1.29 | | | `RemoveSelfLink` | `false` | Alpha | 1.16 | 1.19 | | `RemoveSelfLink` | `true` | Beta | 1.20 | 1.23 | | `RemoveSelfLink` | `true` | GA | 1.24 | | diff --git a/content/en/docs/tasks/administer-cluster/change-pv-access-mode-readwriteoncepod.md b/content/en/docs/tasks/administer-cluster/change-pv-access-mode-readwriteoncepod.md new file mode 100644 index 0000000000000..a2dffdc702c22 --- /dev/null +++ b/content/en/docs/tasks/administer-cluster/change-pv-access-mode-readwriteoncepod.md @@ -0,0 +1,187 @@ +--- +title: Change the Access Mode of a PersistentVolume to ReadWriteOncePod +content_type: task +weight: 90 +min-kubernetes-server-version: v1.22 +--- + + + +This page shows how to change the access mode on an existing PersistentVolume to +use `ReadWriteOncePod`. + +## {{% heading "prerequisites" %}} + +{{< include "task-tutorial-prereqs.md" >}} {{< version-check >}} + +{{< note >}} +The `ReadWriteOncePod` access mode graduated to stable in the Kubernetes v1.29 +release. If you are running a version of Kubernetes older than v1.29, you might +need to enable a feature gate. Check the documentation for your version of +Kubernetes. +{{< /note >}} + +{{< note >}} +The `ReadWriteOncePod` access mode is only supported for +{{< glossary_tooltip text="CSI" term_id="csi" >}} volumes. +To use this volume access mode you will need to update the following +[CSI sidecars](https://kubernetes-csi.github.io/docs/sidecar-containers.html) +to these versions or greater: + +* [csi-provisioner:v3.0.0+](https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.0.0) +* [csi-attacher:v3.3.0+](https://github.com/kubernetes-csi/external-attacher/releases/tag/v3.3.0) +* [csi-resizer:v1.3.0+](https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.3.0) +{{< /note >}} + +## Why should I use `ReadWriteOncePod`? + +Prior to Kubernetes v1.22, the `ReadWriteOnce` access mode was commonly used to +restrict PersistentVolume access for workloads that required single-writer +access to storage. However, this access mode had a limitation: it restricted +volume access to a single *node*, allowing multiple pods on the same node to +read from and write to the same volume simultaneously. This could pose a risk +for applications that demand strict single-writer access for data safety. + +If ensuring single-writer access is critical for your workloads, consider +migrating your volumes to `ReadWriteOncePod`. + + + +## Migrating existing PersistentVolumes + +If you have existing PersistentVolumes, they can be migrated to use +`ReadWriteOncePod`. Only migrations from `ReadWriteOnce` to `ReadWriteOncePod` +are supported. + +In this example, there is already a `ReadWriteOnce` "cat-pictures-pvc" +PersistentVolumeClaim that is bound to a "cat-pictures-pv" PersistentVolume, +and a "cat-pictures-writer" Deployment that uses this PersistentVolumeClaim. + +{{< note >}} +If your storage plugin supports +[Dynamic provisioning](/docs/concepts/storage/dynamic-provisioning/), +the "cat-picutres-pv" will be created for you, but its name may differ. To get +your PersistentVolume's name run: + +```shell +kubectl get pvc cat-pictures-pvc -o jsonpath='{.spec.volumeName}' +``` +{{< /note >}} + +And you can view the PVC before you make changes. Either view the manifest +locally, or run `kubectl get pvc -o yaml`. The output is similar +to: + +```yaml +# cat-pictures-pvc.yaml +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: cat-pictures-pvc +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi +``` + +Here's an example Deployment that relies on that PersistentVolumeClaim: + +```yaml +# cat-pictures-writer-deployment.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: cat-pictures-writer +spec: + replicas: 3 + selector: + matchLabels: + app: cat-pictures-writer + template: + metadata: + labels: + app: cat-pictures-writer + spec: + containers: + - name: nginx + image: nginx:1.14.2 + ports: + - containerPort: 80 + volumeMounts: + - name: cat-pictures + mountPath: /mnt + volumes: + - name: cat-pictures + persistentVolumeClaim: + claimName: cat-pictures-pvc + readOnly: false +``` + +As a first step, you need to edit your PersistentVolume's +`spec.persistentVolumeReclaimPolicy` and set it to `Retain`. This ensures your +PersistentVolume will not be deleted when you delete the corresponding +PersistentVolumeClaim: + +```shell +kubectl patch pv cat-pictures-pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' +``` + +Next you need to stop any workloads that are using the PersistentVolumeClaim +bound to the PersistentVolume you want to migrate, and then delete the +PersistentVolumeClaim. Avoid making any other changes to the +PersistentVolumeClaim, such as volume resizes, until after the migration is +complete. + +Once that is done, you need to clear your PersistentVolume's `spec.claimRef.uid` +to ensure PersistentVolumeClaims can bind to it upon recreation: + +```shell +kubectl scale --replicas=0 deployment cat-pictures-writer +kubectl delete pvc cat-pictures-pvc +kubectl patch pv cat-pictures-pv -p '{"spec":{"claimRef":{"uid":""}}}' +``` + +After that, replace the PersistentVolume's list of valid access modes to be +(only) `ReadWriteOncePod`: + +```shell +kubectl patch pv cat-pictures-pv -p '{"spec":{"accessModes":["ReadWriteOncePod"]}}' +``` + +{{< note >}} +The `ReadWriteOncePod` access mode cannot be combined with other access modes. +Make sure `ReadWriteOncePod` is the only access mode on the PersistentVolume +when updating, otherwise the request will fail. +{{< /note >}} + +Next you need to modify your PersistentVolumeClaim to set `ReadWriteOncePod` as +the only access mode. You should also set the PersistentVolumeClaim's +`spec.volumeName` to the name of your PersistentVolume to ensure it binds to +this specific PersistentVolume. + +Once this is done, you can recreate your PersistentVolumeClaim and start up your +workloads: + +```shell +# IMPORTANT: Make sure to edit your PVC in cat-pictures-pvc.yaml before applying. You need to: +# - Set ReadWriteOncePod as the only access mode +# - Set spec.volumeName to "cat-pictures-pv" + +kubectl apply -f cat-pictures-pvc.yaml +kubectl apply -f cat-pictures-writer-deployment.yaml +``` + +Lastly you may edit your PersistentVolume's `spec.persistentVolumeReclaimPolicy` +and set to it back to `Delete` if you previously changed it. + +```shell +kubectl patch pv cat-pictures-pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}' +``` + +## {{% heading "whatsnext" %}} + +* Learn more about [PersistentVolumes](/docs/concepts/storage/persistent-volumes/). +* Learn more about [PersistentVolumeClaims](/docs/concepts/storage/persistent-volumes/#persistentvolumeclaims). +* Learn more about [Configuring a Pod to Use a PersistentVolume for Storage](/docs/tasks/configure-pod-container/configure-persistent-volume-storage/) diff --git a/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md b/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md index dc1a01abb00ad..0d6c9859dc97f 100644 --- a/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md +++ b/content/en/docs/tasks/configure-pod-container/configure-persistent-volume-storage.md @@ -98,6 +98,12 @@ read-write by a single Node. It defines the [StorageClass name](/docs/concepts/s `manual` for the PersistentVolume, which will be used to bind PersistentVolumeClaim requests to this PersistentVolume. +{{< note >}} +This example uses the `ReadWriteOnce` access mode, for simplicity. For +production use, the Kubernetes project recommends using the `ReadWriteOncePod` +access mode instead. +{{< /note >}} + Create the PersistentVolume: ```shell