From d174742c461e8f1ea8806858f9870898ff7dd708 Mon Sep 17 00:00:00 2001 From: "Lubomir I. Ivanov" Date: Thu, 9 Nov 2023 13:44:36 +0200 Subject: [PATCH] kubeadm: add section on how to use the "generate-csr" command The "generate-csr" command is useful in cases users don't wish to use the default certificate duration that kubeadm has hardcoded to 1 year. The command can also be used when the certificate rotation process is done manually, out of bounds with an external CA. --- .../setup-tools/kubeadm/kubeadm-certs.md | 4 +- .../kubeadm/kubeadm-certs.md | 268 ++++++++++++++++-- 2 files changed, 250 insertions(+), 22 deletions(-) diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md index 3bce10ccf0b67..f4951290cf804 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md @@ -60,7 +60,9 @@ For more details see ## kubeadm certs generate-csr {#cmd-certs-generate-csr} This command can be used to generate keys and CSRs for all control-plane certificates and kubeconfig files. -The user can then sign the CSRs with a CA of their choice. +The user can then sign the CSRs with a CA of their choice. To read more information +on how to use the command see +[Signing certificate signing requests (CSR) generated by kubeadm](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#signing-csr). {{< tabs name="tab-certs-generate-csr" >}} {{< tab name="generate-csr" include="generated/kubeadm_certs_generate-csr.md" />}} diff --git a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md index fead85f7e6af4..b745a22792c75 100644 --- a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md +++ b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md @@ -225,28 +225,11 @@ A CSR represents a request to a CA for a signed certificate for a client. In kubeadm terms, any certificate that would normally be signed by an on-disk CA can be produced as a CSR instead. A CA, however, cannot be produced as a CSR. -### Create certificate signing requests (CSR) - -You can create certificate signing requests with `kubeadm certs renew --csr-only`. - -Both the CSR and the accompanying private key are given in the output. -You can pass in a directory with `--csr-dir` to output the CSRs to the specified location. -If `--csr-dir` is not specified, the default certificate directory (`/etc/kubernetes/pki`) is used. - -Certificates can be renewed with `kubeadm certs renew --csr-only`. -As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag. - -A CSR contains a certificate's name, domains, and IPs, but it does not specify usages. -It is the responsibility of the CA to specify [the correct cert usages](/docs/setup/best-practices/certificates/#all-certificates) -when issuing a certificate. +### Renewal by using certificate signing requests (CSR) -* In `openssl` this is done with the - [`openssl ca` command](https://superuser.com/questions/738612/openssl-ca-keyusage-extension). -* In `cfssl` you specify - [usages in the config file](https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170). - -After a certificate is signed using your preferred method, the certificate and the private key -must be copied to the PKI directory (by default `/etc/kubernetes/pki`). +Renewal of ceritficates is possible by generating new CSRs and signing them with the external CA. +For more details about working with CSRs generated by kubeadm see the section +[Signing certificate signing requests (CSR) generated by kubeadm](#signing-csr). ## Certificate authority (CA) rotation {#certificate-authority-rotation} @@ -373,3 +356,246 @@ The following example will generate a kubeconfig file with administrator credent ```shell kubeadm kubeconfig user --config example.yaml --client-name admin --validity-period 168h ``` + +## Signing certificate signing requests (CSR) generated by kubeadm {#signing-csr} + +You can create certificate signing requests with `kubeadm certs generate-csr`. +Calling this command will generate `.csr` / `.key` file pairs for regular +certificates. For certificates embedded in kubeconfig files, the command will +generate a `.csr` / `.conf` pair where the key is already embedded in the `.conf` file. + +A CSR file contains all relevant information for a CA to sign a certificate. +kubeadm uses a +[well defined specification](/docs/setup/best-practices/certificates/#all-certificates) +for all its certificates and CSRs. + +The default certificate directory is `/etc/kubernetes/pki`, while the default +directory for kubeconfig files is `/etc/kubernetes`. These defaults can be +overridden with the flags `--cert-dir` and `--kubeconfig-dir`, respectively. + +To pass custom options to `kubeadm certs generate-csr` use the `--config` flag, +which accepts a [kubeadm configuration](/docs/reference/config-api/kubeadm-config.v1beta3/) +file, similarly to commands such as `kubeadm init`. Any specification such +as extra SANs and custom IP addresses must be stored in the same configuration +file and used for all relevant kubeadm commands by passing it as `--config`. + +{{< note >}} +This guide will cover the usage of the `openssl` command for singing the CSRs, +but you can use your preferred tools. +{{< /note >}} + +{{< note >}} +This guide will use the default Kubernetes directory `/etc/kubernetes`, which requires +a super user. If you are following this guide with permissive directories +(by passing `--cert-dir` and `--kubeconfig-dir`) you can omit the `sudo` command). +But note that the resulted files must be copied to the `/etc/kubernetes` tree, +so that `kubeadm init` or `kubeadm join` will find them. +{{< /note >}} + +### Preparing CA and service account files + +On the primary control plane node, where `kubeadm init` will be executed, call the following +commands: + +```shell +sudo kubeadm init phase certs ca +sudo kubeadm init phase certs etcd-ca +sudo kubeadm init phase certs front-proxy-ca +sudo kubeadm init phase certs sa +``` + +This will populate the folders `/etc/kubernetes/pki` and `/etc/kubernetes/pki/etcd` +with all self-signed CA files (certificates and keys) and service account (public and +private keys) that kubeadm needs for a control plane node. + +{{< note >}} +If you are using an external CA, you must generate the same files out of band and manually +copy them to the primary control plane node in `/etc/kubernetes`. Once all CSRs +are signed, you can delete the root CA key (`ca.key`) as noted in the +[External CA mode](#external-ca-mode) section. +{{< /note >}} + +For secondary control plane nodes (`kubeadm join --control-plane`) there is no need to call +the above commands. Depending on how you setup the +[High Availability](/docs/setup/production-environment/tools/kubeadm/high-availability) +cluster, you either have to manually copy the same files from the primary +control plane node, or use the automated `--upload-certs` functionality of `kubeadm init`. + +### Generate CSRs + +The `kubeadm certs generate-csr` command generates CSRs for all known certificates +managed by kubeadm. Once the command is done you must manually delete `.csr`, `.conf` +or `.key` files that you don't need. + +#### Considerations for kubelet.conf {#considerations-kubelet-conf} + +This section applies to both control plane and worker nodes. + +If you have deleted the `ca.key` file from control plane nodes +([External CA mode](#external-ca-mode)), the active kube-controller-manager in +this cluster will not be able to sign kubelet client certificates. If no external +method for signing these certificates exists in your setup (such as an +[external signer](#set-up-a-signer), you could manually sign the `kubelet.conf.csr` +as explained in this guide. + +Note that this also means that the automatic +[kubelet client certificate rotation](/docs/tasks/tls/certificate-rotation/#enabling-client-certificate-rotation) +will be disabled. If so, close to certificate expiration, you must generate +a new `kubelet.conf.csr`, sign the certificate, embed it in `kubelet.conf` +and restart the kubelet. + +If this does not apply to your setup, you can skip processing the `kubelet.conf.csr` +on secondary control plane and on workers nodes (all nodes tha call `kubeadm join ...`). +That is because the active kube-controller-manager will be responsible +for signing new kubelet client certificates. + +{{< note >}} +Processing the `kubelet.conf.csr` on the primary control plane node +(`kubeadm init`) is required, because that is considered the node that +bootstraps the cluster and a pre-populated `kubelet.conf` is needed. +{{< /note >}} + +#### Control plane nodes + +Execute the following command on primary (`kubeadm init`) and secondary +(`kubeadm join --control-plane`) control plane nodes to generate all CSR files: + +```shell +sudo kubeadm certs generate-csr +``` + +If external etcd is to be used, follow the +[External etcd with kubeadm](docs/setup/production-environment/tools/kubeadm/high-availability/#external-etcd-nodes) +guide to understand what CSR files are needed on the kubeadm and etcd nodes. Other +`.csr` and `.key` files under `/etc/kubernetes/pki/etcd` can be removed. + +Based on the explanation in +[Considerations for kubelet.conf](#considerations-kubelet-conf) keep or delete +the `kubelet.conf` and `kubelet.conf.csr` files. + +#### Worker nodes + +Based on the explanation in +[Considerations for kubelet.conf](#considerations-kubelet-conf), optionally call: + +```shell +sudo kubeadm certs generate-csr +``` + +and keep only the `kubelet.conf` and `kubelet.conf.csr` files. Alternatively skip +the steps for worker nodes entirely. + +### Signing CSRs for all certificates + +{{< note >}} +If you are using external CA and already have CA serial number files (`.srl`) for +`openssl` you can copy such files to a kubeadm node where CSRs will be processed. +`.srl` files to copy are `/etc/kubernetes/pki/ca.srl`, +`/etc/kubernetes/pki/front-proxy-ca.srl` and `/etc/kubernetes/pki/etcd/ca.srl`. +The files can be then moved to a new node where CSR files will be processed. + +If a `.srl` file is missing for a CA on a node, the script below will generate a new SRL file +with a random starting serial number. + +To read more about `.srl` files see the +[`openssl`](https://www.openssl.org/docs/man3.0/man1/openssl-x509.html) +documentation for the `--CAserial` flag. +{{< /note >}} + +Repeat this step for all nodes that have CSR files. + +Write the following script in the `/etc/kubernetes` directory, navigate to the directory +and execute the script. The script will generate certificates for all CSR files that are +present in the `/etc/kubernetes` tree. + +```bash +#!/bin/bash + +# Set certificate expiration time in days +DAYS=365 + +# Process all CSR files except those for front-proxy and etcd +find ./ -name "*.csr" | grep -v "pki/etcd" | grep -v "front-proxy" | while read -r FILE; +do + echo "* Processing ${FILE} ..." + FILE=${FILE%.*} # Trim the extension + if [ -f "./pki/ca.srl" ]; then + SERIAL_FLAG="-CAserial ./pki/ca.srl" + else + SERIAL_FLAG="-CAcreateserial" + fi + openssl x509 -req -days "${DAYS}" -CA ./pki/ca.crt -CAkey ./pki/ca.key ${SERIAL_FLAG} \ + -in "${FILE}.csr" -out "${FILE}.crt" + sleep 2 +done + +# Process all etcd CSRs +find ./pki/etcd -name "*.csr" | while read -r FILE; +do + echo "* Processing ${FILE} ..." + FILE=${FILE%.*} # Trim the extension + if [ -f "./pki/etcd/ca.srl" ]; then + SERIAL_FLAG=-CAserial ./pki/etcd/ca.srl + else + SERIAL_FLAG=-CAcreateserial + fi + openssl x509 -req -days "${DAYS}" -CA ./pki/etcd/ca.crt -CAkey ./pki/etcd/ca.key ${SERIAL_FLAG} \ + -in "${FILE}.csr" -out "${FILE}.crt" +done + +# Process front-proxy CSRs +echo "* Processing ./pki/front-proxy-client.csr ..." +openssl x509 -req -days "${DAYS}" -CA ./pki/front-proxy-ca.crt -CAkey ./pki/front-proxy-ca.key -CAcreateserial \ + -in ./pki/front-proxy-client.csr -out ./pki/front-proxy-client.crt +``` + +### Embedding certificates in kubeconfig files + +Repeat this step for all nodes that have CSR files. + +Write the following script in the `/etc/kubernetes` directory, navigate to the directory +and execute the script. The script will take the `.crt` files that were signed for +kubeconfig files from CSRs in the previous step and will embed them in the kubeconfig files. + +```bash +#!/bin/bash + +CLUSTER=kubernetes +find ./ -name "*.conf" | while read -r FILE; +do + echo "* Processing ${FILE} ..." + KUBECONFIG="${FILE}" kubectl config set-cluster "${CLUSTER}" --certificate-authority ./pki/ca.crt --embed-certs + USER=$(KUBECONFIG="${FILE}" kubectl config view -o jsonpath='{.users[0].name}') + KUBECONFIG="${FILE}" kubectl config set-credentials "${USER}" --client-certificate "${FILE}.crt" --embed-certs +done +``` + +### Performing cleanup {#post-csr-cleanup} + +Perform this step on all nodes that have CSR files. + +Write the following script in the `/etc/kubernetes` directory, navigate to the directory +and execute the script. + +```bash +#!/bin/bash + +# Cleanup CSR files +rm -f ./*.csr ./pki/*.csr ./pki/etcd/*.csr # Clean all CSR files + +# Cleanup CRT files that were already embedded in kubeconfig files +rm -f ./*.crt +``` + +Optionally, move `.srl` files to the next node to be processed. + +Optionally, if using external CA remove the `/etc/kubernetes/pki/ca.key` file, +as explained in the [External CA node](#external-ca-mode) section. + +### kubeadm node initialization + +Once CSR files have been signed and required certificates are in place on the hosts +you want to use as nodes, you can use the commands `kubeadm init` and `kubeadm join` +to create a Kubernetes cluster from these nodes. During `init` and `join`, kubeadm +uses existing certificates, encryption keys and kubeconfig files that it finds in the +`/etc/kubernetes` tree on the host's local filesystem.