-
Notifications
You must be signed in to change notification settings - Fork 107
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Ability to change control plane certificates expiration date (#1595)
* Refactored certificates settings * Moved apicerver-certificates tasks to correct role * Added ansible tasks to regenerate certificates * Added custom handler to process openssl output date * Applied created tasks to kubernetes_master role * Added documentation for certificates management * Updated changelog * Renamed config option not to match builtin name * Do not restart kubelet during certificates renewal process * Formatting and syntax fixes * Splitted tasks by empty lines * Renamed config setting * Collect k8s apiserver cert information before renewal * Certificates renewal documentation enhancements * Regenerate certificates tasks improvement Co-authored-by: atsikham <[email protected]>
- Loading branch information
Showing
9 changed files
with
315 additions
and
16 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
21 changes: 21 additions & 0 deletions
21
core/src/epicli/data/common/ansible/playbooks/filter_plugins/date_processing.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| #!/usr/bin/python | ||
|
|
||
| from datetime import datetime | ||
|
|
||
|
|
||
| class FilterModule(object): | ||
| def filters(self): | ||
| return { | ||
| 'openssl_date2days': self.openssl_date2days | ||
| } | ||
|
|
||
| def openssl_date2days(self, openssl_date): | ||
| """ | ||
| This function is used to find difference between openssl's | ||
| '-enddate' or '-startdate' output and today | ||
| :param openssl_date: '-enddate' or '-startdate' output of openssl, example: notAfter=Apr 20 07:06:21 2022 GMT | ||
| :return: result in days | ||
| """ | ||
| date1 = datetime.strptime(openssl_date.split('=')[1], '%b %d %H:%M:%S %Y %Z').date() | ||
| date2 = datetime.now().date() | ||
| return (date1 - date2).days |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
188 changes: 188 additions & 0 deletions
188
...cli/data/common/ansible/playbooks/roles/kubernetes_master/tasks/generate-certificates.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,188 @@ | ||
| --- | ||
| - name: Generate certificates block | ||
| vars: | ||
| # https://kubernetes.io/docs/setup/best-practices/certificates/#all-certificates | ||
| _certificates_opt_mapping: | ||
| - name: admin.conf | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/admin.conf" | ||
| parent_ca: ca | ||
| - name: apiserver-etcd-client | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/apiserver-etcd-client" | ||
| parent_ca: etcd/ca | ||
| - name: apiserver-kubelet-client | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/apiserver-kubelet-client" | ||
| parent_ca: ca | ||
| - name: apiserver | ||
| kind: ['serverAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/apiserver" | ||
| parent_ca: ca | ||
| - name: controller-manager.conf | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/controller-manager.conf" | ||
| parent_ca: ca | ||
| - name: etcd-healthcheck-client | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/etcd/healthcheck-client" | ||
| parent_ca: etcd/ca | ||
| - name: etcd-peer | ||
| kind: ['serverAuth', 'clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/etcd/peer" | ||
| parent_ca: etcd/ca | ||
| - name: etcd-server | ||
| kind: ['serverAuth', 'clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/etcd/server" | ||
| parent_ca: etcd/ca | ||
| - name: front-proxy-client | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/front-proxy-client" | ||
| parent_ca: front-proxy-ca | ||
| - name: scheduler.conf | ||
| kind: ['clientAuth'] | ||
| target: "{{ specification.advanced.certificates.location }}/scheduler.conf" | ||
| parent_ca: ca | ||
| block: | ||
| - name: Create certificates_opt_mapping fact | ||
| block: | ||
| - set_fact: | ||
| certificates_opt_mapping: "{{ certificates_opt_mapping | default([]) + [item] }}" | ||
| when: certificates_renewal_list is defined and item.name in certificates_renewal_list | ||
| with_items: "{{ _certificates_opt_mapping }}" | ||
| - set_fact: | ||
| certificates_opt_mapping: "{{ _certificates_opt_mapping }}" | ||
| when: certificates_renewal_list is not defined | ||
|
|
||
| - name: Save old certificates | ||
| synchronize: | ||
| src: "{{ specification.advanced.certificates.location }}/" | ||
| dest: >- | ||
| "{{ specification.advanced.certificates.location | regex_replace('\\/$', '') }}-backup-{{ ansible_date_time.iso8601_basic_short }}" | ||
| delegate_to: "{{ inventory_hostname }}" | ||
|
|
||
| - name: Ensure necessary directories exist | ||
| file: | ||
| path: "{{ item }}" | ||
| state: directory | ||
| owner: root | ||
| group: root | ||
| mode: u=rw | ||
| with_items: | ||
| - "{{ specification.advanced.certificates.location }}/csr" | ||
| - "{{ specification.advanced.certificates.location }}/ext" | ||
|
|
||
| - name: Generate new CSR | ||
| shell: kubeadm alpha certs renew all --csr-only --csr-dir=csr | ||
| args: | ||
| executable: /bin/bash | ||
| chdir: "{{ specification.advanced.certificates.location }}" | ||
|
|
||
| # ansible openssl modules and openssl tool behave different, extensions file is necessary for openssl | ||
| # https://github.com/openssl/openssl/issues/10458 | ||
| - name: Register SAN extension for all csr files | ||
| shell: |- | ||
| openssl req -text -noout \ | ||
| -reqopt no_subject,no_header,no_version,no_serial,no_signame,no_validity,no_issuer,no_pubkey,no_sigdump,no_aux \ | ||
| -in csr/{{ item.name }}.csr \ | ||
| | sed '1,3d;s/ Address//g;s/^[[:blank:]]*//;s/[[:blank:]]*$//' | ||
| args: | ||
| executable: /bin/bash | ||
| chdir: "{{ specification.advanced.certificates.location }}" | ||
| register: csr_info | ||
| with_items: "{{ certificates_opt_mapping }}" | ||
|
|
||
| - name: Generate extension files | ||
| template: | ||
| src: certificate-v3.ext.j2 | ||
| dest: "{{ specification.advanced.certificates.location }}/ext/{{ item.0.name }}.ext" | ||
| with_together: | ||
| - "{{ certificates_opt_mapping }}" | ||
| - "{{ csr_info.results }}" | ||
|
|
||
| - name: Create signed certificates | ||
| shell: |- | ||
| openssl x509 -req -days {{ valid_days }} -in csr/{{ item.name }}.csr -extfile ext/{{ item.name }}.ext \ | ||
| -CA {{ item.parent_ca }}.crt -CAkey {{ item.parent_ca }}.key -CAcreateserial -out {{ item.target }}.crt | ||
| args: | ||
| executable: /bin/bash | ||
| chdir: "{{ specification.advanced.certificates.location }}" | ||
| with_items: "{{ certificates_opt_mapping }}" | ||
|
|
||
| - name: Copy keys to pki location and ensure that permissions are strict | ||
| copy: | ||
| src: "{{ specification.advanced.certificates.location }}/csr/{{ item.name }}.key" | ||
| remote_src: yes | ||
| dest: "{{ item.target }}.key" | ||
| owner: root | ||
| group: root | ||
| mode: '0600' | ||
| with_items: "{{ certificates_opt_mapping }}" | ||
|
|
||
| - name: Remove csr and ext directories | ||
| file: | ||
| path: "{{ specification.advanced.certificates.location }}/{{ item }}" | ||
| state: absent | ||
| with_items: | ||
| - csr | ||
| - ext | ||
|
|
||
| - name: Search for .conf certificates | ||
| find: | ||
| paths: [ "{{ specification.advanced.certificates.location }}" ] | ||
| pattern: "*.conf.crt" | ||
| register: _conf_certificates | ||
|
|
||
| - name: Set conf_certificates fact | ||
| set_fact: | ||
| conf_certificates: >- | ||
| {{ _conf_certificates.files | ||
| | map(attribute='path') | ||
| | map('basename') | ||
| | map('regex_replace', '\.crt$', '') | ||
| | list }} | ||
| - name: Update conf files with embedded certs | ||
| environment: | ||
| KUBECONFIG: "/etc/kubernetes/{{ item }}" | ||
| vars: | ||
| conf_account_mapping: | ||
| admin.conf: "kubernetes-admin" | ||
| scheduler.conf: "system:kube-scheduler" | ||
| controller-manager.conf: "system:kube-controller-manager" | ||
| shell: | | ||
| kubectl config set-credentials {{ conf_account_mapping[item] }} \ | ||
| --client-key {{ specification.advanced.certificates.location }}/{{ item }}.key \ | ||
| --client-certificate {{ specification.advanced.certificates.location }}/{{ item }}.crt --embed-certs | ||
| args: | ||
| executable: /bin/bash | ||
| with_items: "{{ conf_certificates }}" | ||
|
|
||
| - name: Remove conf certificates | ||
| file: | ||
| path: "{{ specification.advanced.certificates.location }}/{{ item.0 }}.{{ item.1 }}" | ||
| state: absent | ||
| with_nested: | ||
| - - 'admin.conf' | ||
| - 'scheduler.conf' | ||
| - 'controller-manager.conf' | ||
| - - 'crt' | ||
| - 'key' | ||
|
|
||
| - name: Restart systemd services | ||
| when: restart_services is defined and (restart_services | difference(['docker', 'kubelet']) | length == 0) | ||
| block: | ||
| - name: Restart | ||
| systemd: | ||
| name: "{{ item }}" | ||
| state: restarted | ||
| with_items: "{{ restart_services }}" | ||
|
|
||
| - name: Wait until cluster is available | ||
| environment: | ||
| KUBECONFIG: /etc/kubernetes/admin.conf | ||
| shell: kubectl cluster-info | ||
| retries: 50 | ||
| delay: 1 | ||
| register: output | ||
| until: output is succeeded |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 changes: 5 additions & 0 deletions
5
...cli/data/common/ansible/playbooks/roles/kubernetes_master/templates/certificate-v3.ext.j2
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| keyUsage = critical, digitalSignature, keyEncipherment | ||
| extendedKeyUsage = {{ item.0.kind | join(',') }} | ||
| {% if item.1.stdout %} | ||
| subjectAltName = {{ item.1.stdout }} | ||
| {% endif %} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # PKI certificates management | ||
|
|
||
| ## TLS certificates in a cluster | ||
|
|
||
| It's possible to regenerate kubernetes control plane certificates with epiphany. | ||
| To do so, additional configuration should be specified. | ||
|
|
||
| ```yaml | ||
| kind: configuration/kubernetes-master | ||
| title: "Kubernetes Master Config" | ||
| name: default | ||
| provider: <provider> | ||
| specification: | ||
| advanced: | ||
| certificates: | ||
| expiration_days: <int> | ||
| renew: true | ||
| ``` | ||
| Parameters (optional): | ||
| 1. expiration_days - days to expire in, default value is `365` | ||
| 2. renew - whether to renew certificates or not, default value is `false` | ||
|
|
||
| When `epicly apply` executes, if `renew` option is set to `true`, following certificates will be renewed with expiration period defined by `expiration_days`: | ||
|
|
||
| 1. admin.conf | ||
| 2. apiserver | ||
| 3. apiserver-etcd-client | ||
| 4. apiserver-kubelet-client | ||
| 5. controller-manager.conf | ||
| 6. etcd-healthcheck-client | ||
| 7. etcd-peer | ||
| 8. etcd-server | ||
| 9. front-proxy-client | ||
| 10. scheduler.conf | ||
|
|
||
| --- | ||
| **NOTE** | ||
|
|
||
| kubelet.conf is not renewed because kubelet is configured for automatic certificate renewal. | ||
| To verify that, navigate to `/var/lib/kubelet/` and check `config.yaml` file, where `rotateCertificates` setting is `true` by default. | ||
|
|
||
| --- | ||
|
|
||
| ## CA certificates rotation | ||
|
|
||
| This part cannot be done by epiphany. Refer to official kubernetes [documentation](https://kubernetes.io/docs/tasks/tls/manual-rotation-of-ca-certificates/) to perform this task. | ||
|
|
||
| ## References | ||
|
|
||
| 1. [Best practices](https://kubernetes.io/docs/setup/best-practices/certificates/) | ||
| 2. [Certificates management by kubeadm](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/) | ||
| 3. [Kubernetes the hard way](https://github.com/kelseyhightower/kubernetes-the-hard-way/blob/master/docs/04-certificate-authority.md) | ||
| 4. [Certificates generation with cfssl](https://gist.github.com/detiber/81b515df272f5911959e81e39137a8bb) |