diff --git a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md index 3bce10ccf0b67..f4951290cf804 100644 --- a/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md +++ b/content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md @@ -60,7 +60,9 @@ For more details see ## kubeadm certs generate-csr {#cmd-certs-generate-csr} This command can be used to generate keys and CSRs for all control-plane certificates and kubeconfig files. -The user can then sign the CSRs with a CA of their choice. +The user can then sign the CSRs with a CA of their choice. To read more information +on how to use the command see +[Signing certificate signing requests (CSR) generated by kubeadm](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#signing-csr). {{< tabs name="tab-certs-generate-csr" >}} {{< tab name="generate-csr" include="generated/kubeadm_certs_generate-csr.md" />}} diff --git a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md index fead85f7e6af4..5c20975e3d6c9 100644 --- a/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md +++ b/content/en/docs/tasks/administer-cluster/kubeadm/kubeadm-certs.md @@ -225,28 +225,11 @@ A CSR represents a request to a CA for a signed certificate for a client. In kubeadm terms, any certificate that would normally be signed by an on-disk CA can be produced as a CSR instead. A CA, however, cannot be produced as a CSR. -### Create certificate signing requests (CSR) - -You can create certificate signing requests with `kubeadm certs renew --csr-only`. - -Both the CSR and the accompanying private key are given in the output. -You can pass in a directory with `--csr-dir` to output the CSRs to the specified location. -If `--csr-dir` is not specified, the default certificate directory (`/etc/kubernetes/pki`) is used. - -Certificates can be renewed with `kubeadm certs renew --csr-only`. -As with `kubeadm init`, an output directory can be specified with the `--csr-dir` flag. - -A CSR contains a certificate's name, domains, and IPs, but it does not specify usages. -It is the responsibility of the CA to specify [the correct cert usages](/docs/setup/best-practices/certificates/#all-certificates) -when issuing a certificate. +### Renewal by using certificate signing requests (CSR) -* In `openssl` this is done with the - [`openssl ca` command](https://superuser.com/questions/738612/openssl-ca-keyusage-extension). -* In `cfssl` you specify - [usages in the config file](https://github.com/cloudflare/cfssl/blob/master/doc/cmd/cfssl.txt#L170). - -After a certificate is signed using your preferred method, the certificate and the private key -must be copied to the PKI directory (by default `/etc/kubernetes/pki`). +Renewal of ceritficates is possible by generating new CSRs and signing them with the external CA. +For more details about working with CSRs generated by kubeadm see the section +[Signing certificate signing requests (CSR) generated by kubeadm](#signing-csr). ## Certificate authority (CA) rotation {#certificate-authority-rotation} @@ -373,3 +356,214 @@ The following example will generate a kubeconfig file with administrator credent ```shell kubeadm kubeconfig user --config example.yaml --client-name admin --validity-period 168h ``` + +## Signing certificate signing requests (CSR) generated by kubeadm {#signing-csr} + +You can create certificate signing requests with `kubeadm certs generate-csr`. +Calling this command will generate `.csr` / `.key` file pairs for regular +certificates. For certificates embedded in kubeconfig files, the command will +generate a `.csr` / `.conf` pair where the key is already embedded in the `.conf` file. + +The default certificate directory is `/etc/kubernetes/pki`, while the default +directory for kubeconfig files is `/etc/kubernetes`. These defaults can be +overridden with the flags `--cert-dir` and `--kubeconfig-dir`, respectively. + +To pass custom options to `kubeadm certs generate-csr` use the `--config` flag, +which accepts a kubeadm API file, similarly to commands such as `kubeadm init`. +Note that any specification such as extra SAN and custom IP addresses **must be** +stored in the same API file and used for all relevant kubeadm commands. + +A CSR file contains all relevant information for a CA to sign a certificate. +kubeadm uses a +[well defined specification](/docs/setup/best-practices/certificates/#all-certificates) +for all its certificates and CSR. + +{{< note >}} +This guide will cover the usage of the `openssl` command for singing the CSRs, +but you can use your preferred tools. +{{< /note >}} + +{{< note >}} +This guide will use the default Kubernetes path `/etc/kubernetes`, which requires a super +user. If you are following this guide with a permissive path you can omit the `sudo` +command. +{{< /note >}} + +### Preparing CA and service account files + +On the primary control plane node, where `kubeadm init` will be executed, call the following +commands: + +```shell +sudo kubeadm init phase certs ca +sudo kubeadm init phase certs etcd-ca +sudo kubeadm init phase certs front-proxy-ca +sudo kubeadm init phase certs sa +``` + +This will populate the folders `/etc/kubernetes/pki` and `/etc/kubernetes/pki/etcd` +with all self-signed CA files (certificates and keys) and service account (public and +private keys) that kubeadm needs for a control plane node. + +{{< note >}} +If you are using an external CA, you must generate the same files out of band and manually +copy them to the primary control plane node. Once all CSRs are signed you can delete the root +CA key (`ca.key`) as noted in the [External CA mode](#external-ca-mode) section. +{{< /note >}} + +For secondary control plane nodes the above commands does not have to be called. +Depending on how you setup the +[High Availability](/docs/setup/production-environment/tools/kubeadm/high-availability) +cluster, you either have to manually copy the same files from the primary +control plane node, or use the automated `--upload-certs` functionality of `kubeadm init`. + +### Generate CSRs + +{{< note >}} +The `kubeadm certs generate-csr` command generates CSRs for **all** known certificates +managed by kubeadm. You must manually delete CSR files that are not needed. +{{< /note >}} + +#### Control plane nodes + +Execute the following command on primary and secondary control plane +nodes to generate all CSR files: + +```shell +sudo kubeadm certs generate-csr +``` + +If external etcd is to be used, follow the +[External etcd with kubeadm](docs/setup/production-environment/tools/kubeadm/high-availability/#external-etcd-nodes) +guide to understand what CSR files are needed on the kubeadm and etcd nodes. Other etcd +CSR files can be removed. + +If kubelet client certificate rotation is disabled for this cluster you can +leave `kubelet.conf` and `kubelet.conf.csr` on secondary control plane nodes +and generate a certificate for this kubeconfig. + +#### Worker nodes + +Execute the same command on worker nodes, but only if kubelet client certificate rotation +is disabled for this cluster. + +```shell +sudo kubeadm certs generate-csr +``` + +Then delete all CSR and CONF files except `kubelet.conf` and `kubelet.conf.csr`. + +### Signing CSRs for all certificates + +{{< note >}} +If you are using external CA and already have CA serial number files (SRL) for `openssl` +you can copy such files to a kubeadm node where CSRs will be processed. +SRL files to copy are `/etc/kubernetes/pki/ca.srl`, +`/etc/kubernetes/pki/front-proxy-ca.srl` and `/etc/kubernetes/pki/etcd/ca.srl`. +The files can be copied to a new node, once the CSRs on a given node were already +processed. + +If a SRL file is missing for a CA on a node, the script below will generate a new SRL file +with a random starting serial number. +{{< /note >}} + +Repeat this step for all nodes that have CSR files. +Write the following script in the `/etc/kubernetes` directory, navigate to the directory +and execute the script. + +The script will generate certificates for all CSR files that are present in the +`/etc/kubernetes` directory. + +```bash +#!/bin/bash + +# Set certificate expiration time in days +DAYS=365 + +# Process all CSR files except those for front-proxy and etcd +find ./ -name "*.csr" | grep -v "pki/etcd" | grep -v "front-proxy" | while read -r FILE; +do + echo "* Processing ${FILE} ..." + FILE=${FILE%.*} # Trim the extension + if [ -f "./pki/ca.srl" ]; then + SERIAL_FLAG="-CAserial ./pki/ca.srl" + else + SERIAL_FLAG="-CAcreateserial" + fi + openssl x509 -req -days "${DAYS}" -CA ./pki/ca.crt -CAkey ./pki/ca.key ${SERIAL_FLAG} \ + -in "${FILE}.csr" -out "${FILE}.crt" + sleep 2 +done + +# Process all etcd CSRs +find ./pki/etcd -name "*.csr" | while read -r FILE; +do + echo "* Processing ${FILE} ..." + FILE=${FILE%.*} # Trim the extension + if [ -f "./pki/etcd/ca.srl" ]; then + SERIAL_FLAG=-CAserial ./pki/etcd/ca.srl + else + SERIAL_FLAG=-CAcreateserial + fi + openssl x509 -req -days "${DAYS}" -CA ./pki/etcd/ca.crt -CAkey ./pki/etcd/ca.key ${SERIAL_FLAG} \ + -in "${FILE}.csr" -out "${FILE}.crt" +done + +# Process front-proxy CSRs +echo "* Processing ./pki/front-proxy-client.csr ..." +openssl x509 -req -days "${DAYS}" -CA ./pki/front-proxy-ca.crt -CAkey ./pki/front-proxy-ca.key -CAcreateserial \ + -in ./pki/front-proxy-client.csr -out ./pki/front-proxy-client.crt +``` + +### Embedding certificates in kubeconfig files + +Repeat this step for all nodes that have CSR files. +Write the following script in the `/etc/kubernetes` directory, navigate to the directory +and execute the script. + +The script will take the CRT files that were signed for kubeconfig files in +the previous step and will embed them in the kubeconfig files. + +```bash +#!/bin/bash + +CLUSTER=kubernetes +find ./ -name "*.conf" | while read -r FILE; +do + echo "* Processing ${FILE} ..." + KUBECONFIG="${FILE}" kubectl config set-cluster "${CLUSTER}" --certificate-authority ./pki/ca.crt --embed-certs + USER=$(KUBECONFIG="${FILE}" kubectl config view -o jsonpath='{.users[0].name}') + KUBECONFIG="${FILE}" kubectl config set-credentials "${USER}" --client-certificate "${FILE}.crt" --embed-certs +done +``` + +### Performing cleanup + +Perform this step on all nodes that have CSR files. +Write the following script in the `/etc/kubernetes` directory, navigate to the directory +and execute the script. + +```bash +#!/bin/bash + +# Cleanup CSR files +rm -f ./*.csr ./pki/*.csr ./pki/etcd/*.csr # Clean all CSR files + +# Cleanup CRT files that were already embedded in kubeconfig files +rm -f ./*.crt +``` + +Optionally, store SRL files for CSR signing on another node and delete +them from `/etc/kubernetes` paths. This has more value for external CAs that +keep track of signed certificates, and less value for self-signed CAs. + +Optionally, if using external CA remove the `/etc/kubernetes/pki/ca.key` file, +as explained in the [External CA node](#external-ca-mode) section. + +### kubeadm node initialization + +Once CSR files have been signed and required certificates are in place on nodes, +the kubeadm commands `kubeadm init` and `kubeadm join` can be used to create +a Kubernetes cluster from these nodes. During `init` and `join` kubeadm will +use existing certificates, keys and kubeconfig files that it finds in the +`/etc/kubernetes` directory.