Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add document explaining certificates and PKI for Kubernetes #8232

Closed
wants to merge 3 commits into from
Closed

[WIP] Add document explaining certificates and PKI for Kubernetes #8232

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
15f7665
Add document explaining certificates and PKI for Kubernetes
liztio Apr 30, 2018
0be4a24
Docs review feedback
liztio Aug 2, 2018
b1f3cd3
copyedits for clarity, template, partial hugo markdown
Aug 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
114 changes: 59 additions & 55 deletions docs/reference/certificates.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,85 +1,83 @@
---
layout: docwithnav
title: Kubernetes PKI certificates and requirements
permalink: /certs/
approvers:
- efrost
title: PKI Certificates and Requirements
reviewers:
- sig-cluster-lifecycle
content_template: templates/concept
---

# Introduction
{{% capture overview %}}

A tool like [kubeadm][kubeadm] can be used to generate your certificates for you.
But there are also reasons to generate these certificates yourself.
This allows you to keep your private keys off of the api server, for enhanced security.
Kubernetes requires PKI certificates for authentication over TLS.
If you install Kubernetes with [kubeadm][kubeadm], the certificates that your cluster requires are automatically generated.
You can also generate your own certificates -- for example, to keep your private keys more secure by not storing them on the API server.
This page explains the certificates that your cluster requires.

# Certificates in Kubernetes
{{% /capture %}}

Kuberentes uses TLS extensively for authentication.
{{% capture body %}}

## Where they're used
## How certificates are used by your cluster

Kubernetes uses PKI for the following operations:
1. Client certificates for the kubelet to authenticate to the API server
2. Server certificate for the API server endpoint
3. Client certificates for administrators of the cluster authenticating to the API server
4. Client certificates for the API server to talk to the kubelets
5. Client certificate for the API server to talk to etcd
6. Client certificate/kubeconfig for the controller manager to talk to the API server
7. Client certificate/kubeconfig for the scheduler to talk to the API server.
8. Client and server certificates for the [front-proxy][proxy]
Kubernetes requires PKI for the following operations:

* Client certificates for the kubelet to authenticate to the API server
* Server certificate for the API server endpoint
* Client certificates for administrators of the cluster to authenticate to the API server
* Client certificates for the API server to talk to the kubelets
* Client certificate for the API server to talk to etcd
* Client certificate/kubeconfig for the controller manager to talk to the API server
* Client certificate/kubeconfig for the scheduler to talk to the API server.
* Client and server certificates for the [front-proxy][proxy]

In addition, etcd uses mutual TLS for authenticating clients and peers.
etcd also implements mutual TLS to authenticate clients and peers.

## Where to find them
## Where certificates are stored

Kubernetes stores certificates by default in `/etc/kubernetes/pki`. All paths in this documentation are relative to that directory.

## Certificate configuration
## Configure certificates manually

There are multiple levels of control for kubernetes certificates.
At the basic level, a tool like [kubeadm][kubeadm] can generate all required certificates.
If that isn't desireable, a single root CA, controlled by an adminstrator, can create multiple _intermediate CAs_, and delegate all further creation to Kubernetes itself.
If you don't want [kubeadm][kubeadm] to generate the required certificates, you can create them in either of the following ways.

### Single root CA

The CAs needed for this to work are:
You can create a single root CA, controlled by an adminstrator. This root CA can then create multiple intermediate CAs, and delegate all further creation to Kubernetes itself.

Required CAs:

| path | Default CN | description |
|------------------------|---------------------------|----------------------------------|
| ca.crt,key | kubernetes-ca | the kubernetes general CA |
| etcd/ca.crt,key | kubernetes-front-proxy-ca | For all ETCD-related functions |
| front-proxy-ca.crt,key | etcd-ca | for the [front-end proxy][proxy] |

If you don't wish to copy these private keys to your API servers, you can generate all certificates yourself.
In that case, you will also need to make all of these:
| ca.crt,key | kubernetes-ca | Kubernetes general CA |
| etcd/ca.crt,key | etcd-ca | For all etcd-related functions |
| front-proxy-ca.crt,key | kubernetes-front-proxy-ca | For the [front-end proxy][proxy] |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looked as though this and the previous line somehow got reversed in the version that I edited. If I've misunderstood something obscure, my apologies (but if I've misunderstood, we should explain a bit more too).


### All certificates

If you don't wish to copy these private keys to your API servers, you can generate all certificates yourself.

[2]: For a liveness probe, if self-hosted
Required certificates:

| Default CN | Parent CA | O (in Subject) | kind | hosts (SAN) |
|-------------------------------|---------------------------|----------------|----------------------------------------|---------------------------------------------|
| kube-etcd | etcd-ca | | server, client [<sup>1</sup>][etcdbug] | localhost, 127.0.0.1 |
| kube-etcd-peer | etcd-ca | | peer | <hostname>, <Host IP>, localhost, 127.0.0.1 |
| kube-etcd | etcd-ca | | server, client [<sup>1</sup>][etcdbug] | `localhost`, `127.0.0.1` |
| kube-etcd-peer | etcd-ca | | peer | `<hostname>`, `<Host_IP>`, `localhost`, `127.0.0.1` |
| kube-etcd-healthcheck-client | etcd-ca | | client | |
| kube-apiserver-etcd-client | etcd-ca | system:masters | client | |
| kube-apiserver | kubernetes-ca | | server | <hostname>, <Host IP>, <advertise IP> , [1] |
| kube-apiserver | kubernetes-ca | | server | `<hostname>`, `<Host_IP>`, `<advertise_IP>`, `[1]` |
| kube-apiserver-kubelet-client | kubernetes-ca | system:masters | client | |
| front-proxy-client | kubernetes-front-proxy-ca | | client | |

[1]: `kubernetes`, `kubernetes.default`, `kubernetes.default.svc`, `kubernetes.default.svc.cluster`, `kubernetes.default.svc.cluster.local`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed markup for this and other notes because they weren't rendering at all in the original.


[etcdbug]: https://github.com/coreos/etcd/issues/9785
1: "kubernetes", "kubernetes.default", "kubernetes.default.svc", "kubernetes.default.svc.cluster", "kubernetes.default.svc.cluster.local"

where "kind" is are the [x509 key usage][usage] types:
where `kind` is one or more of the [x509 key usage][usage] types:

| kind | TLS role |
|--------|---------------------------------------------------------------------------------------------------|
| server | Digital Signature, Key Encipherment, TLS Web Server Authentication |
| peer | Digital Signature, Key Encipherment, TLS Web Server Authentication, TLS Web Client Authentication |
| client | Digital Signature, Key Encipherment, TLS Web Client Authentication |

### Cert paths
### Certificate paths

Certificates should either be placed in a recommended path (as used by [kubeadm][kubeadm]). Paths should be specified using the given argument regardless of location.

Expand All @@ -98,9 +96,11 @@ Certificates should either be placed in a recommended path (as used by [kubeadm]
| etcd-ca | | etcd/ca.crt | etcdctl[2] | | --cacert |
| kube-etcd-healthcheck-client | etcd/healthcheck-client.key | etcd/healthcheck-client.crt | etcdctl[2] | --key | --cert |

## User configuration
[2]: For a liveness probe, if self-hosted

## Configure certificates for user accounts

The administrator account and service accounts will need to be manually configured.
You must manually configure thesee administrator account and service accounts:

| filename | credential name | Default CN | O (in Subject) |
|-------------------------|----------------------------|--------------------------------|----------------|
Expand All @@ -109,24 +109,28 @@ The administrator account and service accounts will need to be manually configur
| controller-manager.conf | default-controller-manager | system:kube-controller-manager | |
| scheduler.conf | default-manager | system:kube-scheduler | |

The first step is to generate an x509 cert/key pair with the given CN and O. Then, `kubectl` can be used to generate a config (one per line):
1. For each config, generate an x509 cert/key pair with the given CN and O.

1. Run `kubectl` as follows for each config:

```shell
KUBECONFIG=<filename> kubectl config set-cluster default-cluster --server=https://<host ip>:6443 --certificate-authority <path to kubernetes ca> --embed-certs
KUBECONFIG=<filename> kubectl config set-cluster default-cluster --server=https://<host ip>:6443 --certificate-authority <path-to-kubernetes-ca> --embed-certs
KUBECONFIG=<filename> kubectl config set-credentials <credential-name> --client-key <path-to-key>.pem --client-certificate <path-to-cert>.pem --embed-certs
KUBECONFIG=<filename> kubectl config set-context default-system --cluster default-cluster --user <credential-name>
KUBECONFIG=<filename> kubectl config use-context default-system
```

These files are used as follows:

| filename | command | comment |
|-------------------------|-------------------------|-------------------------------------------------------------|
| admin.conf | kubectl | used by a user to administer the cluster |
| kubelet.conf | kubelet | one will be needed for every node in the cluster. |
| controller-manager.conf | kube-controller-manager | add to manifest in `manifests/kube-controller-manager.yaml` |
| scheduler.conf | kube-scheduler | add to manifest in `manifests/kube-scheduler.yaml` |
| filename | command | comment |
|-------------------------|-------------------------|-----------------------------------------------------------------------|
| admin.conf | kubectl | Lets user administer the cluster |
| kubelet.conf | kubelet | One required for each node in the cluster. |
| controller-manager.conf | kube-controller-manager | Must be added to manifest in `manifests/kube-controller-manager.yaml` |
| scheduler.conf | kube-scheduler | Must be added to manifest in `manifests/kube-scheduler.yaml` |

[usage]: https://tools.ietf.org/html/rfc5280#section-4.2.1.3
[kubeadm]: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
[proxy]: https://kubernetes.io/docs/concepts/api-extension/apiserver-aggregation/
[kubeadm]: /docs/reference/setup-tools/kubeadm/kubeadm/
[proxy]: /docs/concepts/extend-kubernetes/api-extension/apiserver-aggregation/

{{% /capture %}}