kubeadm: add section on how to use the "generate-csr" command

content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md

@@ -60,7 +60,9 @@ For more details see
 ## kubeadm certs generate-csr {#cmd-certs-generate-csr}
 
 This command can be used to generate keys and CSRs for all control-plane certificates and kubeconfig files.
-The user can then sign the CSRs with a CA of their choice.
+The user can then sign the CSRs with a CA of their choice. To read more information
+on how to use the command see
+[Signing certificate signing requests (CSR) generated by kubeadm](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#signing-csr).
 
 {{< tabs name="tab-certs-generate-csr" >}}
 {{< tab name="generate-csr" include="generated/kubeadm_certs_generate-csr.md" />}}

content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md

@@ -225,28 +225,11 @@ A CSR represents a request to a CA for a signed certificate for a client.
 In kubeadm terms, any certificate that would normally be signed by an on-disk CA can be produced
 as a CSR instead. A CA, however, cannot be produced as a CSR.
 
-### Create certificate signing requests (CSR)
-
-You can create certificate signing requests with `kubeadm certs renew --csr-only`.
-
-Both the CSR and the accompanying private key are given in the output.
-You can pass in a directory with `--csr-dir` to output the CSRs to the specified location.
-If `--csr-dir` is not specified, the default certificate directory (`/etc/kubernetes/pki`) is used.
-
-Certificates can be renewed with `kubeadm certs renew --csr-only`.
-As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag.
-
-A CSR contains a certificate's name, domains, and IPs, but it does not specify usages.
-It is the responsibility of the CA to specify [the correct cert usages](/docs/setup/best-practices/certificates/#all-certificates)
-when issuing a certificate.
+### Renewal by using certificate signing requests (CSR)
 
-* In `openssl` this is done with the
-  [`openssl ca` command](https://superuser.com/questions/738612/openssl-ca-keyusage-extension).
-* In `cfssl` you specify
-  [usages in the config file](https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170).
-
-After a certificate is signed using your preferred method, the certificate and the private key
-must be copied to the PKI directory (by default `/etc/kubernetes/pki`).
+Renewal of ceritficates is possible by generating new CSRs and signing them with the external CA.
+For more details about working with CSRs generated by kubeadm see the section
+[Signing certificate signing requests (CSR) generated by kubeadm](#signing-csr).
 
 ## Certificate authority (CA) rotation {#certificate-authority-rotation}
 
@@ -373,3 +356,246 @@ The following example will generate a kubeconfig file with administrator credent
 ```shell
 kubeadm kubeconfig user --config example.yaml --client-name admin --validity-period 168h
 ```
+
+## Signing certificate signing requests (CSR) generated by kubeadm {#signing-csr}
+
+You can create certificate signing requests with `kubeadm certs generate-csr`.
+Calling this command will generate `.csr` / `.key` file pairs for regular
+certificates. For certificates embedded in kubeconfig files, the command will
+generate a `.csr` / `.conf` pair where the key is already embedded in the `.conf` file.
+
+A CSR file contains all relevant information for a CA to sign a certificate.
+kubeadm uses a
+[well defined specification](/docs/setup/best-practices/certificates/#all-certificates)
+for all its certificates and CSRs.
+
+The default certificate directory is `/etc/kubernetes/pki`, while the default
+directory for kubeconfig files is `/etc/kubernetes`. These defaults can be
+overridden with the flags `--cert-dir` and `--kubeconfig-dir`, respectively.
+
+To pass custom options to `kubeadm certs generate-csr` use the `--config` flag,
+which accepts a [kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/)
+file, similarly to commands such as `kubeadm init`. Any specification such
+as extra SANs and custom IP addresses must be stored in the same configuration
+file and used for all relevant kubeadm commands by passing it as `--config`.
+
+{{< note >}}
+This guide will cover the usage of the `openssl` command for singing the CSRs,
+but you can use your preferred tools.
+{{< /note >}}
+
+{{< note >}}
+This guide will use the default Kubernetes directory `/etc/kubernetes`, which requires
+a super user. If you are following this guide with permissive directories
+(by passing `--cert-dir` and `--kubeconfig-dir`) you can omit the `sudo` command).
+But note that the resulted files must be copied to the `/etc/kubernetes` tree,
+so that `kubeadm init` or `kubeadm join` will find them.
+{{< /note >}}
+
+### Preparing CA and service account files
+
+On the primary control plane node, where `kubeadm init` will be executed, call the following
+commands:
+
+```shell
+sudo kubeadm init phase certs ca
+sudo kubeadm init phase certs etcd-ca
+sudo kubeadm init phase certs front-proxy-ca
+sudo kubeadm init phase certs sa
+```
+
+This will populate the folders `/etc/kubernetes/pki` and `/etc/kubernetes/pki/etcd`
+with all self-signed CA files (certificates and keys) and service account (public and
+private keys) that kubeadm needs for a control plane node.
+
+{{< note >}}
+If you are using an external CA, you must generate the same files out of band and manually
+copy them to the primary control plane node in `/etc/kubernetes`. Once all CSRs
+are signed, you can delete the root CA key (`ca.key`) as noted in the
+[External CA mode](#external-ca-mode) section.
+{{< /note >}}
+
+For secondary control plane nodes (`kubeadm join --control-plane`) there is no need to call
+the above commands. Depending on how you setup the
+[High Availability](/docs/setup/production-environment/tools/kubeadm/high-availability)
+cluster, you either have to manually copy the same files from the primary
+control plane node, or use the automated `--upload-certs` functionality of `kubeadm init`.
+
+### Generate CSRs
+
+The `kubeadm certs generate-csr` command generates CSRs for all known certificates
+managed by kubeadm. Once the command is done you must manually delete `.csr`, `.conf`
+or `.key` files that you don't need.
+
+#### Considerations for kubelet.conf {#considerations-kubelet-conf}
+
+This section applies to both control plane and worker nodes.
+
+If you have deleted the `ca.key` file from control plane nodes
+([External CA mode](#external-ca-mode)), the active kube-controller-manager in
+this cluster will not be able to sign kubelet client certificates. If no external
+method for signing these certificates exists in your setup (such as an
+[external signer](#set-up-a-signer), you could manually sign the `kubelet.conf.csr`
+as explained in this guide.
+
+Note that this also means that the automatic
+[kubelet client certificate rotation](/docs/tasks/tls/certificate-rotation/#enabling-client-certificate-rotation)
+will be disabled. If so, close to certificate expiration, you must generate
+a new `kubelet.conf.csr`, sign the certificate, embed it in `kubelet.conf`
+and restart the kubelet.
+
+If this does not apply to your setup, you can skip processing the `kubelet.conf.csr`
+on secondary control plane and on workers nodes (all nodes tha call `kubeadm join ...`).
+That is because the active kube-controller-manager will be responsible
+for signing new kubelet client certificates.
+
+{{< note >}}
+Processing the `kubelet.conf.csr` on the primary control plane node
+(`kubeadm init`) is required, because that is considered the node that
+bootstraps the cluster and a pre-populated `kubelet.conf` is needed.
+{{< /note >}}
+
+#### Control plane nodes
+
+Execute the following command on primary (`kubeadm init`) and secondary
+(`kubeadm join --control-plane`) control plane nodes to generate all CSR files:
+
+```shell
+sudo kubeadm certs generate-csr
+```
+
+If external etcd is to be used, follow the
+[External etcd with kubeadm](docs/setup/production-environment/tools/kubeadm/high-availability/#external-etcd-nodes)
+guide to understand what CSR files are needed on the kubeadm and etcd nodes. Other
+`.csr` and `.key` files under `/etc/kubernetes/pki/etcd` can be removed.
+
+Based on the explanation in
+[Considerations for kubelet.conf](#considerations-kubelet-conf) keep or delete
+the `kubelet.conf` and `kubelet.conf.csr` files.
+
+#### Worker nodes
+
+Based on the explanation in
+[Considerations for kubelet.conf](#considerations-kubelet-conf), optionally call:
+
+```shell
+sudo kubeadm certs generate-csr
+```
+
+and keep only the `kubelet.conf` and `kubelet.conf.csr` files. Alternatively skip
+the steps for worker nodes entirely.
+
+### Signing CSRs for all certificates
+
+{{< note >}}
+If you are using external CA and already have CA serial number files (`.srl`) for
+`openssl` you can copy such files to a kubeadm node where CSRs will be processed.
+`.srl` files to copy are `/etc/kubernetes/pki/ca.srl`,
+`/etc/kubernetes/pki/front-proxy-ca.srl` and `/etc/kubernetes/pki/etcd/ca.srl`.
+The files can be then moved to a new node where CSR files will be processed.
+
+If a `.srl` file is missing for a CA on a node, the script below will generate a new SRL file
+with a random starting serial number.
+
+To read more about `.srl` files see the
+[`openssl`](https://www.openssl.org/docs/man3.0/man1/openssl-x509.html)
+documentation for the `--CAserial` flag.
+{{< /note >}}
+
+Repeat this step for all nodes that have CSR files.
+
+Write the following script in the `/etc/kubernetes` directory, navigate to the directory
+and execute the script. The script will generate certificates for all CSR files that are
+present in the `/etc/kubernetes` tree.
+
+```bash
+#!/bin/bash
+
+# Set certificate expiration time in days
+DAYS=365
+
+# Process all CSR files except those for front-proxy and etcd
+find ./ -name "*.csr" | grep -v "pki/etcd" | grep -v "front-proxy" | while read -r FILE;
+do
+    echo "* Processing ${FILE} ..."
+    FILE=${FILE%.*} # Trim the extension
+    if [ -f "./pki/ca.srl" ]; then
+        SERIAL_FLAG="-CAserial ./pki/ca.srl"
+    else
+        SERIAL_FLAG="-CAcreateserial"
+    fi
+    openssl x509 -req -days "${DAYS}" -CA ./pki/ca.crt -CAkey ./pki/ca.key ${SERIAL_FLAG} \
+        -in "${FILE}.csr" -out "${FILE}.crt"
+    sleep 2
+done
+
+# Process all etcd CSRs
+find ./pki/etcd -name "*.csr" | while read -r FILE;
+do
+    echo "* Processing ${FILE} ..."
+    FILE=${FILE%.*} # Trim the extension
+    if [ -f "./pki/etcd/ca.srl" ]; then
+        SERIAL_FLAG=-CAserial ./pki/etcd/ca.srl
+    else
+        SERIAL_FLAG=-CAcreateserial
+    fi
+    openssl x509 -req -days "${DAYS}" -CA ./pki/etcd/ca.crt -CAkey ./pki/etcd/ca.key ${SERIAL_FLAG} \
+        -in "${FILE}.csr" -out "${FILE}.crt"
+done
+
+# Process front-proxy CSRs
+echo "* Processing ./pki/front-proxy-client.csr ..."
+openssl x509 -req -days "${DAYS}" -CA ./pki/front-proxy-ca.crt -CAkey ./pki/front-proxy-ca.key -CAcreateserial \
+    -in ./pki/front-proxy-client.csr -out ./pki/front-proxy-client.crt
+```
+
+### Embedding certificates in kubeconfig files
+
+Repeat this step for all nodes that have CSR files.
+
+Write the following script in the `/etc/kubernetes` directory, navigate to the directory
+and execute the script. The script will take the `.crt` files that were signed for
+kubeconfig files from CSRs in the previous step and will embed them in the kubeconfig files.
+
+```bash
+#!/bin/bash
+
+CLUSTER=kubernetes
+find ./ -name "*.conf" | while read -r FILE;
+do
+    echo "* Processing ${FILE} ..."
+    KUBECONFIG="${FILE}" kubectl config set-cluster "${CLUSTER}" --certificate-authority ./pki/ca.crt --embed-certs
+    USER=$(KUBECONFIG="${FILE}" kubectl config view -o jsonpath='{.users[0].name}')
+    KUBECONFIG="${FILE}" kubectl config set-credentials "${USER}" --client-certificate "${FILE}.crt" --embed-certs
+done
+```
+
+### Performing cleanup {#post-csr-cleanup}
+
+Perform this step on all nodes that have CSR files.
+
+Write the following script in the `/etc/kubernetes` directory, navigate to the directory
+and execute the script.
+
+```bash
+#!/bin/bash
+
+# Cleanup CSR files
+rm -f ./*.csr ./pki/*.csr ./pki/etcd/*.csr # Clean all CSR files
+
+# Cleanup CRT files that were already embedded in kubeconfig files
+rm -f ./*.crt
+```
+
+Optionally, move `.srl` files to the next node to be processed.
+
+Optionally, if using external CA remove the `/etc/kubernetes/pki/ca.key` file,
+as explained in the [External CA node](#external-ca-mode) section.
+
+### kubeadm node initialization
+
+Once CSR files have been signed and required certificates are in place on the hosts
+you want to use as nodes, you can use the commands `kubeadm init` and `kubeadm join`
+to create a Kubernetes cluster from these nodes. During `init` and `join`, kubeadm
+uses existing certificates, encryption keys and kubeconfig files that it finds in the
+`/etc/kubernetes` tree on the host's local filesystem.