Skip to content

Commit

Permalink
Merge pull request #43540 from neolit123/1.29-add-super-admin-kubeconfig
Browse files Browse the repository at this point in the history 
kubeadm: introduce documentation changes for super-admin.conf
  • Loading branch information
@k8s-ci-robot
k8s-ci-robot authored Oct 31, 2023
2 parents d05e393 + a9478b4 commit 636f1d8
Show file tree
Hide file tree
Showing 9 changed files with 318 additions and 33 deletions.
92 changes: 92 additions & 0 deletions ...reference/setup-tools/kubeadm/generated/kubeadm_certs_renew_super-admin.conf.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
<!--
The file is auto-generated from the Go source code of the component using a generic
[generator](https://github.com/kubernetes-sigs/reference-docs/). To learn how
to generate the reference documentation, please read
[Contributing to the reference documentation](/docs/contribute/generate-ref-docs/).
To update the reference content, please follow the
[Contributing upstream](/docs/contribute/generate-ref-docs/contribute-upstream/)
guide. You can file document formatting bugs against the
[reference-docs](https://github.com/kubernetes-sigs/reference-docs/) project.
-->


Renew the certificate embedded in the kubeconfig file for the super-admin

### Synopsis


Renew the certificate embedded in the kubeconfig file for the super-admin.

Renewals run unconditionally, regardless of certificate expiration date; extra attributes such as SANs will be based on the existing file/certificates, there is no need to resupply them.

Renewal by default tries to use the certificate authority in the local PKI managed by kubeadm; as alternative it is possible to use K8s certificate API for certificate renewal, or as a last option, to generate a CSR request.

After renewal, in order to make changes effective, is required to restart control-plane components and eventually re-distribute the renewed certificate in case the file is used elsewhere.

```
kubeadm certs renew super-admin.conf [flags]
```

### Options

<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>

<tr>
<td colspan="2">--cert-dir string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes/pki"</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The path where to save the certificates</p></td>
</tr>

<tr>
<td colspan="2">--config string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Path to a kubeadm configuration file.</p></td>
</tr>

<tr>
<td colspan="2">-h, --help</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>help for admin.conf</p></td>
</tr>

<tr>
<td colspan="2">--kubeconfig string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes/admin.conf"</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file.</p></td>
</tr>

</tbody>
</table>



### Options inherited from parent commands

<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>

<tr>
<td colspan="2">--rootfs string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>[EXPERIMENTAL] The path to the 'real' host root filesystem.</p></td>
</tr>

</tbody>
</table>



121 changes: 121 additions & 0 deletions ...ence/setup-tools/kubeadm/generated/kubeadm_init_phase_kubeconfig_super-admin.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
<!--
The file is auto-generated from the Go source code of the component using a generic
[generator](https://github.com/kubernetes-sigs/reference-docs/). To learn how
to generate the reference documentation, please read
[Contributing to the reference documentation](/docs/contribute/generate-ref-docs/).
To update the reference content, please follow the
[Contributing upstream](/docs/contribute/generate-ref-docs/contribute-upstream/)
guide. You can file document formatting bugs against the
[reference-docs](https://github.com/kubernetes-sigs/reference-docs/) project.
-->


Generate a kubeconfig file for the super-admin

### Synopsis


Generate a kubeconfig file for the super-admin.

```
kubeadm init phase kubeconfig super-admin [flags]
```

### Options

<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>

<tr>
<td colspan="2">--apiserver-advertise-address string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The IP address the API Server will advertise it's listening on. If not set the default network interface will be used.</p></td>
</tr>

<tr>
<td colspan="2">--apiserver-bind-port int32&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: 6443</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Port for the API Server to bind to.</p></td>
</tr>

<tr>
<td colspan="2">--cert-dir string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes/pki"</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The path where to save and store the certificates.</p></td>
</tr>

<tr>
<td colspan="2">--config string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Path to a kubeadm configuration file.</p></td>
</tr>

<tr>
<td colspan="2">--control-plane-endpoint string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Specify a stable IP address or DNS name for the control plane.</p></td>
</tr>

<tr>
<td colspan="2">--dry-run</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Don't apply any changes; just output what would be done.</p></td>
</tr>

<tr>
<td colspan="2">-h, --help</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>help for admin</p></td>
</tr>

<tr>
<td colspan="2">--kubeconfig-dir string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "/etc/kubernetes"</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>The path where to save the kubeconfig file.</p></td>
</tr>

<tr>
<td colspan="2">--kubernetes-version string&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Default: "stable-1"</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>Choose a specific Kubernetes version for the control plane.</p></td>
</tr>

</tbody>
</table>



### Options inherited from parent commands

<table style="width: 100%; table-layout: fixed;">
<colgroup>
<col span="1" style="width: 10px;" />
<col span="1" />
</colgroup>
<tbody>

<tr>
<td colspan="2">--rootfs string</td>
</tr>
<tr>
<td></td><td style="line-height: 130%; word-wrap: break-word;"><p>[EXPERIMENTAL] The path to the 'real' host root filesystem.</p></td>
</tr>

</tbody>
</table>



22 changes: 16 additions & 6 deletions content/en/docs/reference/setup-tools/kubeadm/implementation-details.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ in a majority of cases, and the most intuitive location; other constants paths a
- `controller-manager.conf`
- `scheduler.conf`
- `admin.conf` for the cluster admin and kubeadm itself
- `super-admin.conf` for the cluster super-admin that can bypass RBAC

- Names of certificates and key files :

Expand Down Expand Up @@ -209,12 +210,21 @@ Kubeadm generates kubeconfig files with identities for control plane components:
This client cert should have the CN `system:kube-scheduler`, as defined by default
[RBAC core components roles](/docs/reference/access-authn-authz/rbac/#core-component-roles)

Additionally, a kubeconfig file for kubeadm itself and the admin is generated and saved into the
`/etc/kubernetes/admin.conf` file. The "admin" here is defined as the actual person(s) that is
administering the cluster and wants to have full control (**root**) over the cluster. The
embedded client certificate for admin should be in the `system:masters` organization, as defined
by default [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles).
It should also include a CN. Kubeadm uses the `kubernetes-admin` CN.
Additionally, a kubeconfig file for kubeadm as an administrative entity is generated and stored
in `/etc/kubernetes/admin.conf`. This file includes a certificate with
`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. `kubeadm:cluster-admins`
is a group managed by kubeadm. It is bound to the `cluster-admin` ClusterRole during `kubeadm init`,
by using the `super-admin.conf` file, which does not require RBAC.
This `admin.conf` file must remain on control plane nodes and not be shared with additional users.

During `kubeadm init` another kubeconfig file is generated and stored in `/etc/kubernetes/super-admin.conf`.
This file includes a certificate with `Subject: O = system:masters, CN = kubernetes-super-admin`.
`system:masters` is a super user group that bypasses RBAC and makes `super-admin.conf` useful in case
of an emergency where a cluster is locked due to RBAC misconfiguration.
The `super-admin.conf` file can be stored in a safe location and not shared with additional users.

See [RBAC user facing role bindings](/docs/reference/access-authn-authz/rbac/#user-facing-roles)
for additional information RBAC and built-in ClusterRoles and groups.

Please note that:

Expand Down
1 change: 1 addition & 0 deletions content/en/docs/reference/setup-tools/kubeadm/kubeadm-certs.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ For more details see [Manual certificate renewal](/docs/tasks/administer-cluster
{{< tab name="etcd-server" include="generated/kubeadm_certs_renew_etcd-server.md" />}}
{{< tab name="front-proxy-client" include="generated/kubeadm_certs_renew_front-proxy-client.md" />}}
{{< tab name="scheduler.conf" include="generated/kubeadm_certs_renew_scheduler.conf.md" />}}
{{< tab name="super-admin.conf" include="generated/kubeadm_certs_renew_super-admin.conf.md" />}}
{{< /tabs >}}

## kubeadm certs certificate-key {#cmd-certs-certificate-key}
Expand Down
1 change: 1 addition & 0 deletions content/en/docs/reference/setup-tools/kubeadm/kubeadm-init-phase.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ You can create all required kubeconfig files by calling the `all` subcommand or
{{< tab name="kubelet" include="generated/kubeadm_init_phase_kubeconfig_kubelet.md" />}}
{{< tab name="controller-manager" include="generated/kubeadm_init_phase_kubeconfig_controller-manager.md" />}}
{{< tab name="scheduler" include="generated/kubeadm_init_phase_kubeconfig_scheduler.md" />}}
{{< tab name="super-admin" include="generated/kubeadm_init_phase_kubeconfig_super-admin.md" />}}
{{< /tabs >}}

## kubeadm init phase control-plane {#cmd-phase-control-plane}
Expand Down
14 changes: 8 additions & 6 deletions content/en/docs/reference/setup-tools/kubeadm/kubeadm-init.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,9 @@ following steps:
arguments, lowercased if necessary.

1. Writes kubeconfig files in `/etc/kubernetes/` for the kubelet, the controller-manager and the
scheduler to use to connect to the API server, each with its own identity, as well as an
additional kubeconfig file for administration named `admin.conf`.
scheduler to use to connect to the API server, each with its own identity. Also
additional kubeconfig files are written, for kubeadm as administrative entity (`admin.conf`)
and for a super admin user that can bypass RBAC (`super-admin.conf`).

1. Generates static Pod manifests for the API server,
controller-manager and scheduler. In case an external etcd is not provided,
Expand Down Expand Up @@ -186,7 +187,7 @@ as a learner and promoted to a voting member only after the etcd data are fully
List of deprecated feature gates:

{{< table caption="kubeadm deprecated feature gates" >}}
Feature | Default
Feature | Default
:-------|:--------
`UpgradeAddonsBeforeControlPlane` | `false`
{{< /table >}}
Expand Down Expand Up @@ -291,7 +292,7 @@ for etcd and CoreDNS.

#### Custom sandbox (pause) images {#custom-pause-image}

To set a custom image for these you need to configure this in your
To set a custom image for these you need to configure this in your
{{< glossary_tooltip text="container runtime" term_id="container-runtime" >}}
to use the image.
Consult the documentation for your container runtime to find out how to change this setting;
Expand Down Expand Up @@ -386,8 +387,9 @@ DNS name or an address of a load balancer.
kubeadm certs certificate-key
```

Once the cluster is up, you can grab the admin credentials from the control-plane node
at `/etc/kubernetes/admin.conf` and use that to talk to the cluster.
Once the cluster is up, you can use the `/etc/kubernetes/admin.conf` file from
a control-plane node to talk to the cluster with administrator credentials or
[Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users).

Note that this style of bootstrap has some relaxed security guarantees because
it does not allow the root CA hash to be validated with
Expand Down
31 changes: 25 additions & 6 deletions content/en/docs/setup/best-practices/certificates.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -184,19 +184,36 @@ you need to provide if you are generating all of your own keys and certificates:

You must manually configure these administrator account and service accounts:

| filename | credential name | Default CN | O (in Subject) |
|-------------------------|----------------------------|-------------------------------------|----------------|
| admin.conf | default-admin | kubernetes-admin | system:masters |
| kubelet.conf | default-auth | system:node:`<nodeName>` (see note) | system:nodes |
| controller-manager.conf | default-controller-manager | system:kube-controller-manager | |
| scheduler.conf | default-scheduler | system:kube-scheduler | |
| filename | credential name | Default CN | O (in Subject) |
|-------------------------|----------------------------|-------------------------------------|------------------------|
| admin.conf | default-admin | kubernetes-admin | `<admin-group>` |
| super-admin.conf | default-super-admin | kubernetes-super-admin | system:masters |
| kubelet.conf | default-auth | system:node:`<nodeName>` (see note) | system:nodes |
| controller-manager.conf | default-controller-manager | system:kube-controller-manager | |
| scheduler.conf | default-scheduler | system:kube-scheduler | |

{{< note >}}
The value of `<nodeName>` for `kubelet.conf` **must** match precisely the value of the node name
provided by the kubelet as it registers with the apiserver. For further details, read the
[Node Authorization](/docs/reference/access-authn-authz/node/).
{{< /note >}}

{{< note >}}
In the above example `<admin-group>` is implementation specific. Some tools sign the
certificate in the default `admin.conf` to be part of the `system:masters` group.
`system:masters` is a break-glass, super user group can bypass the authorization
layer of Kubernetes, such as RBAC. Also some tools do not generate a separate
`super-admin.conf` with a certificate bound to this super user group.

kubeadm generates two separate administrator certificates in kubeconfig files.
One is in `admin.conf` and has `Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`.
`kubeadm:cluster-admins` is a custom group bound to the `cluster-admin` ClusterRole.
This file is generated on all kubeadm managed control plane machines.

Another is in `super-admin.conf` that has `Subject: O = system:masters, CN = kubernetes-super-admin`.
This file is generated only on the node where `kubeadm init` was called.
{{< /note >}}

1. For each config, generate an x509 cert/key pair with the given CN and O.

1. Run `kubectl` as follows for each config:
Expand All @@ -213,6 +230,7 @@ These files are used as follows:
| filename | command | comment |
|-------------------------|-------------------------|-----------------------------------------------------------------------|
| admin.conf | kubectl | Configures administrator user for the cluster |
| super-admin.conf | kubectl | Configures super administrator user for the cluster |
| kubelet.conf | kubelet | One required for each node in the cluster. |
| controller-manager.conf | kube-controller-manager | Must be added to manifest in `manifests/kube-controller-manager.yaml` |
| scheduler.conf | kube-scheduler | Must be added to manifest in `manifests/kube-scheduler.yaml` |
Expand All @@ -221,6 +239,7 @@ The following files illustrate full paths to the files listed in the previous ta

```
/etc/kubernetes/admin.conf
/etc/kubernetes/super-admin.conf
/etc/kubernetes/kubelet.conf
/etc/kubernetes/controller-manager.conf
/etc/kubernetes/scheduler.conf
Expand Down
18 changes: 13 additions & 5 deletions ...nt/en/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -211,11 +211,19 @@ export KUBECONFIG=/etc/kubernetes/admin.conf
```

{{< warning >}}
Kubeadm signs the certificate in the `admin.conf` to have `Subject: O = system:masters, CN = kubernetes-admin`.
`system:masters` is a break-glass, super user group that bypasses the authorization layer (e.g. RBAC).
Do not share the `admin.conf` file with anyone and instead grant users custom permissions by generating
them a kubeconfig file using the `kubeadm kubeconfig user` command. For more details see
[Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users).
The kubeconfig file `admin.conf` that `kubeadm init` generates contains a certificate with
`Subject: O = kubeadm:cluster-admins, CN = kubernetes-admin`. The group `kubeadm:cluster-admins`
is bound to the built-in `cluster-admin` ClusterRole.
Do not share the `admin.conf` file with anyone.

`kubeadm init` generates another kubeconfig file `super-admin.conf` that contains a certificate with
`Subject: O = system:masters, CN = kubernetes-super-admin`.
`system:masters` is a break-glass, super user group that bypasses the authorization layer (for example RBAC).
Do not share the `super-admin.conf` file with anyone. It is recommended to move the file to a safe location.

See
[Generating kubeconfig files for additional users](/docs/tasks/administer-cluster/kubeadm/kubeadm-certs#kubeconfig-additional-users)
on how to use `kubeadm kubeconfig user` to generate kubeconfig files for additional users.
{{< /warning >}}

Make a record of the `kubeadm join` command that `kubeadm init` outputs. You
Expand Down
Loading

0 comments on commit 636f1d8

Please sign in to comment.