diff --git a/source/images/k8s/cert-manager-cluster.svg b/source/images/k8s/cert-manager-cluster.svg new file mode 100644 index 000000000..6c58eb1da --- /dev/null +++ b/source/images/k8s/cert-manager-cluster.svg @@ -0,0 +1,373 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/source/images/k8s/cert-manager-namespaces.svg b/source/images/k8s/cert-manager-namespaces.svg new file mode 100644 index 000000000..251e5bfb5 --- /dev/null +++ b/source/images/k8s/cert-manager-namespaces.svg @@ -0,0 +1,530 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/source/includes/k8s/file-transfer-protocol-k8s.rst b/source/includes/k8s/file-transfer-protocol-k8s.rst index 408db992a..6096eebc5 100644 --- a/source/includes/k8s/file-transfer-protocol-k8s.rst +++ b/source/includes/k8s/file-transfer-protocol-k8s.rst @@ -164,7 +164,7 @@ If SFTP is enabled, the output resembles the following: enableSFTP: true -.. _minio-certificate-key-file-sftp-k8s +.. _minio-certificate-key-file-sftp-k8s: Connect to MinIO Using SFTP with a Certificate Key File ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/source/index.rst b/source/index.rst index e3d376e4d..4b9706933 100644 --- a/source/index.rst +++ b/source/index.rst @@ -106,6 +106,7 @@ For more about connecting to ``play``, see :ref:`MinIO Console play Login `__. +To learn more about cert-manager Issuers, refer to the `Issuer documentation `__. -Below is a logical view of a Kubernetes cluster: +Below is a logical view of a Kubernetes cluster with four namespaces: -![Cert-manager-namespaces.png](images%2FCert-manager-namespaces.png) +- ``minio-operator`` +- ``tenant-1`` +- ``tenant-2`` +- ``other-namespace`` -This cluster contains 4 namespaces: +.. image:: /images/k8s/cert-manager-namespaces.svg + :width: 600px + :alt: A graphic depiction of a kubernetes cluster with four namespaces. Each namespace in its own white box. One is labeled minio-operator with contents two sts content items. A second is labeled tenant-1 with a MinIO logo and four drives of a pool. A third is labeled tenant-2 with similar contents to the tenant-1 box. The fourth is labeled other namespace with contents of other pods. + :align: center -- minio-operator -- tenant-1 -- tenant-2 -- other-namespace +This guide shows you how to set up a different Certificate Authority (CA) in each namespace. +Each namespace's CA references the global ``Cluster Issuer``. -This guide shows you how to set up different Certificate Authorities (CA) in each namespace, all of them referencing a global `Cluster Issuer`. -![Cert-manager Issuers.png](images%2FCert-manager%20Issuers.png) +The following graphic depicts how various namespaces make use of either an ``Issuer`` or ``ClusterIssuer`` type. -Cert Manager is installed in the `cert-manager` namespace. +- cert-manager is installed in the ``cert-manager`` namespace, which does not have either an ``issuer`` or a ``ClusterIssuer``. +- The ``default`` namespace receives the global ``Cluster Issuer``. +- Each tenant's namespace receives a local ``Issuer``. +- The ``minio-operator`` namespace receives a local ``Issuer``. + More about services that require TLS certificates in the ``minio-operator`` namespace are covered :ref:`below `. -The global `Cluster Issuer` is created in the default namespace. +.. image:: /images/k8s/cert-manager-cluster.svg + :width: 600px + :alt: A graphic depiction of a kubernetes cluster with five separate namespaces represented by different boxes. One is labeled minio-operator with a text box that says "minio-operator: issuer". A second is labeled tenant-1 with text box that says "tenant-1: issuer". A third is labeled default with a text box that says "root:ClusterIssuer". A fourth is labeled tenant-2 with a text box that says "tenant-2: issuer". The fifth is labeled cert-manager with a text box that says "minio-operator: issuer". + :align: center -A local `Issuer` is created in each tenant namespace. +.. note:: -An `Issuer` is also created in the `minio-operator` namespace. More about services that require TLS certificates -in the `minio-operator` namespace are covered below in [MinIO Operator services with cert-manager](#minio-operator-services-with-cert-manager).. + This guide uses a self-signed ``Cluster Issuer``. + You can also use `other Issuers supported by cert-manager `__. + The main difference is that you must provide the ``Issuer`` CA certificate to MinIO, instead of the CA's mentioned in this guide. -> [!NOTE] -> This guide uses a self-signed `Cluster Issuer`. You can also use [other Issuers supported by Cert Manager](https://cert-manager.io/docs/configuration/issuers/). -> The main difference is that you must provide the `Issuer` CA certificate to minio, instead of the CA's mentioned in this guide. -## Getting Started +Prerequisites +------------- -### Prerequisites +- A `supported version of Kubernetes `__. + + While cert-manager supports `earlier K8s versions `__, the MinIO Operator requires a currently supported version. +- `kustomize `__ installed +- ``kubectl`` access to your ``k8s`` cluster -- Kubernetes version `+v1.21`. While cert-manager - supports [earlier K8s versions](https://cert-manager.io/docs/installation/supported-releases/), MinIO Operator requires 1.21 or later. -- [kustomize](https://kustomize.io/) installed -- `kubectl` access to your `k8s` cluster +Getting Started +--------------- -### Setup Cert-manager +Setup cert-manager +~~~~~~~~~~~~~~~~~~ -Install [cert-manager](https://cert-manager.io/docs/releases/release-notes/release-notes-1.12/) 1.12.X LTS is preferred, -or install latest, for example using kubectl. +Install cert-manager. +`Release 1.12.X LTS `__ is preferred, but you may install latest. -```bash -kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.9/cert-manager.yaml -``` +The following command installs version 1.12.13 using ``kubectl``. -For more details on the Cert Manager refer to https://cert-manager.io/docs/installation/. +.. code-block:: shell + :class: copyable + + kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.12.13/cert-manager.yaml -### Create Cluster Self-signed root Issuer +For more details on installing cert-manager, see their `installation instructions `__. -The `Cluster Issuer` is the cluster level Issuer all other certificates are derived from. Request Cert Manager to -generate this by creating a `ClusterIssuer` resource: +Create a self-signed root Issuer for the cluster +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -```yaml -# selfsigned-root-clusterissuer.yaml -apiVersion: cert-manager.io/v1 -kind: ClusterIssuer -metadata: - name: selfsigned-root -spec: - selfSigned: {} -``` +The ``Cluster Issuer`` is the top level Issuer from which all other certificates in the cluster derive. -```shell -kubectl apply -f selfsigned-root-clusterissuer.yaml -``` +1. Request cert-manager to generate this by creating a ``ClusterIssuer`` resource. -# MinIO Operator services with cert-manager + Create a file called ``selfsigned-root-clusterissuer.yaml`` with the following contents: -MinIO Operator manages the TLS certificate issuing for the services hosted in the `minio-operator` namespace. That is the Secure Token Service `sts`. + .. code-block:: yaml + :class: copyable + + # selfsigned-root-clusterissuer.yaml + apiVersion: cert-manager.io/v1 + kind: ClusterIssuer + metadata: + name: selfsigned-root + spec: + selfSigned: {} -This section describes how to generate the `sts` TLS certificate with Cert Manager. -These certificates must be issued before installing Operator. -Be sure to follow step [Create Cluster Self-signed root Issuer](#create-cluster-self-signed-root-issuer) mentioned above. +2. Apply the resource to the cluster: -## Secure Token Service (STS) + .. code-block:: shell + :class: copyable -MinIO STS is a service included with MinIO Operator that provides Native IAM Authentication for Kubernetes. In essence -this service allows you to control access to your MinIO tenant from your kubernetes applications without having to explicitly create credentials -for each application. For more information on the Service see the MinIO docs at https://min.io/docs/minio/kubernetes/upstream/developers/sts-for-operator.html. -There is also an [STS](https://github.com/minio/operator/blob/master/docs/STS.md) guide in the docs and example client code in -https://github.com/minio/operator/tree/master/examples/kustomization/sts-example. + kubectl apply -f selfsigned-root-clusterissuer.yaml -For the purpose of this guide, STS Service can be considered a webserver presented with a TLS certificate for https traffic. -This guide covers how to **disable** the automatic generation of the certificate in MinIO Operator and issue the certificate using -Cert Manager instead. +.. _minio-operator-services-with-cert-manager: -### Create minio-operator namespace CA Issuer +MinIO Operator services with cert-manager +----------------------------------------- -To create the Certificate Authority (CA) certificate used to issue certificates for services in the `minio-operator` -namespace, first create the minio-operator namespace +MinIO Operator manages the TLS certificate issuing for the services hosted in the ``minio-operator`` namespace. +That is the :ref:`Secure Token Service (sts) `. -```shell -kubectl create ns minio-operator -``` +This section describes how to generate the ``sts`` TLS certificate with cert-manager. -Request a Certificate with `spec.isCA: true` specified. This is our CA for the `minio-operator` namespace. +- These certificates **must** be issued *before* installing Operator. +- The cluster's self-signed root ``ClusterIssuer`` certificate must already exist, as described above. -```yaml -# operator-ca-tls-secret.yaml -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: minio-operator-ca-certificate - namespace: minio-operator -spec: - isCA: true - commonName: operator - secretName: operator-ca-tls - duration: 70128h # 8y - privateKey: - algorithm: ECDSA - size: 256 - issuerRef: - name: selfsigned-root - kind: ClusterIssuer - group: cert-manager.io -``` - -```shell -kubectl apply -f operator-ca-tls-secret.yaml -``` -A new secret with the name `operator-ca-tls` is created in the `minio-operator` namespace, this is the CA issuing TLS certificates for the services in the `minio-operator` namespace. - -> [!IMPORTANT] -> Make sure to trust this certificate in your applications that need to interact with the `sts` service. - - -Now create the `Issuer`: - -```yaml -# operator-ca-issuer.yaml -apiVersion: cert-manager.io/v1 -kind: Issuer -metadata: - name: minio-operator-ca-issuer - namespace: minio-operator -spec: - ca: - secretName: operator-ca-tls -``` - -```shell -kubectl apply -f operator-ca-issuer.yaml -``` - -### Create TLS certificate for STS service - -Request Cert Manager to issue a new certificate containing following DNS domains: - -```shell -sts -sts.minio-operator.svc. -sts.minio-operator.svc. -``` - -> [!IMPORTANT] -> Replace `` with the actual values for your MinIO tenant. -> `cluster domain` is the internal root DNS domain assigned in your Kubernetes cluster. Typically this is `cluster.local`, check on your coredns -> configuration for the correct value for your Kubernetes cluster. For example, using `kubectl get configmap coredns -n kube-system -o jsonpath="{.data}"`. -> The way the root DNS domain is managed can vary depending on the Kubernetes distribution (Openshift, Rancher, EKS, etc.) - -Create a `Certificate` for the domains mentioned above: - -```yaml -# sts-tls-certificate.yaml -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: sts-certmanager-cert - namespace: minio-operator -spec: - dnsNames: - - sts - - sts.minio-operator.svc - - sts.minio-operator.svc.cluster.local - secretName: sts-tls - issuerRef: - name: minio-operator-ca-issuer -``` - -```shell -kubectl apply -f sts-tls-certificate.yaml -``` -This creates a secret called `sts-tls` in the `minio-operator` namespace. - -> [!IMPORTANT] -> The secret name is not optional. Make sure the secret name is `sts-tls` by setting `spec.secretName: sts-tls` as in the example above. - -### Install Operator with Auto TLS disabled for STS - -When installing the Operator deployment, make sure to set `OPERATOR_STS_AUTO_TLS_ENABLED: off` env variable in the `minio-operator` container. This prevents -MinIO Operator from issuing the certificate for STS and instead wait for you to provide the TLS certificate issued by Cert Manager. - -> [!WARNING] -> Missing to provide the secret `sts-tls` containing the TLS certificate or providing an invalid key-pair in the secret will -> prevent the STS service from start. - -A way to make sure that the env variables are properly set is using kustomization to patch the `minio-operator` deployment: - -```yaml -# minio-operator/kustomization.yaml -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization - -resources: -- github.com/minio/operator/resources - -patches: -- patch: |- - apiVersion: apps/v1 - kind: Deployment - metadata: - name: minio-operator - namespace: minio-operator - spec: - template: - spec: - containers: - - name: minio-operator - env: - - name: OPERATOR_STS_AUTO_TLS_ENABLED - value: "off" - - name: OPERATOR_STS_ENABLED - value: "on" -``` - -```shell -kubectl apply -k minio-operator -``` - -# MinIO tenant TLS certificates with cert-manager - -## Create tenant-1 namespace CA Issuer - -To create the Certificate Authority (CA) certificate in the namespace `tenant-1`, first create the namespace - -```shell -kubectl create ns tenant-1 -``` - -Next, request a Certificate with `spec.isCA: true` specified: - -```yaml -# tenant-1-ca-certificate.yaml -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: tenant-1-ca-certificate - namespace: tenant-1 -spec: - isCA: true - commonName: tenant-1-ca - secretName: tenant-1-ca-tls - duration: 70128h # 8y - privateKey: - algorithm: ECDSA - size: 256 - issuerRef: - name: selfsigned-root - kind: ClusterIssuer - group: cert-manager.io -``` - -```shell -kubectl apply -f tenant-1-ca-certificate.yaml -``` - - -Then create the `Issuer`: - -```yaml -# tenant-1-ca-issuer.yaml -apiVersion: cert-manager.io/v1 -kind: Issuer -metadata: - name: tenant-1-ca-issuer - namespace: tenant-1 -spec: - ca: - secretName: tenant-1-ca-tls -``` - -```shell -kubectl apply -f tenant-1-ca-issuer.yaml -``` - -## Deploy the tenant - -### Create a certificate for Tenant - -Request Cert Manager issue a new TLS server certificate for MinIO that includes the following DNS domains: - -```shell -minio. -minio..svc -minio..svc. -*.-hl..svc. -*..svc. -*..minio..svc.' -``` - -> [!IMPORTANT] -> Replace `` with the actual values for your MinIO tenant. -> * `` is the internal root DNS domain assigned in your Kubernetes cluster. Typically this is `cluster.local`, check on your coredns -> configuration for the correct value for your Kubernetes cluster. For example, using `kubectl get configmap coredns -n kube-system -o jsonpath="{.data}"`. -> The way the root DNS domain is managed can vary depending on the Kubernetes distribution (Openshift, Rancher, EKS, etc.) -> * `tenant-name` is the name provided to your tenant in the `metadata.name` of the Tenant YAML. For this example it is `myminio`. -> * `namespace` is the namespace where the tenant is created, the `metadata.namespace` notes that in the Tenant YAML. For this example it is `tenant-1`. - -Create a `Certificate` for the domains mentioned above: - -```yaml -# tenant-1-minio-certificate.yaml -apiVersion: cert-manager.io/v1 -kind: Certificate -metadata: - name: tenant-certmanager-cert - namespace: tenant-1 -spec: - dnsNames: - - "minio.tenant-1" - - "minio.tenant-1.svc" - - 'minio.tenant-1.svc.cluster.local' - - '*.minio.tenant-1.svc.cluster.local' - - '*.myminio-hl.tenant-1.svc.cluster.local' - - '*.myminio.minio.tenant-1.svc.cluster.local' - secretName: myminio-tls - issuerRef: - name: tenant-1-ca-issuer -``` - -> [!TIP] -> For this example, the Tenant name is `myminio`. We recommend naming the secret in the field `spec.secretName` as `-tls`, following the naming -> convention MinIO Operator uses when creates certificates with Autocert enabled (`spec.requestAutoCert: true`). - -```shell -kubectl apply -f tenant-1-minio-certificate.yaml -``` - -### Configure the tenant to use the certificate created by cert-manager +Secure Token Service (STS) +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO STS is a service included with MinIO Operator that provides Native IAM Authentication for Kubernetes. +This service allows you to control access to your MinIO tenant from your kubernetes applications without having to explicitly create credentials for each application. +For more information, see the :ref:`Secure Token Service documentation `. + +Think of the STS Service as a webserver presented with a TLS certificate for ``https`` traffic. +This guide covers how to **disable** the automatic generation of the certificate in MinIO Operator and issue the certificate using cert-manager instead. + +Create a CA Issuer for the ``minio-operator`` namespace ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +The ``minio-operator`` namespace needs to have its own certificate authority (CA), derived from the cluster's ``ClusterIssuer`` certificate create earlier. + +1. If it does not exist, create the ``minio-operator`` namespace + + .. code-block:: shell + :class: copyable + + kubectl create ns minio-operator + +2. Request a new Certificate with ``spec.isCA: true`` specified. + + This is our :abbr:`CA (Certificate Authority)` for the `minio-operator` namespace. + + Create a file called ``operator-ca-tls-secret.yaml`` with the following contents: + + .. code-block:: yaml + :class: copyable + :emphasize-lines: 7,8 + + # operator-ca-tls-secret.yaml + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: minio-operator-ca-certificate + namespace: minio-operator + spec: + isCA: true + commonName: operator + secretName: operator-ca-tls + duration: 70128h # 8y + privateKey: + algorithm: ECDSA + size: 256 + issuerRef: + name: selfsigned-root + kind: ClusterIssuer + group: cert-manager.io + +3. Apply the resource to the cluster + + .. code-block:: shell + :class: copyable + + kubectl apply -f operator-ca-tls-secret.yaml + +Kubernetes creates a new secret with the name ``operator-ca-tls`` in the ``minio-operator`` namespace. +This certificate serves as the :abbr:`CA (Certificate Authority)` that issues TLS certificates only for the services in the ``minio-`operator`` namespace. + +.. important:: + + Make sure to trust this certificate in any applications that need to interact with the ``sts`` service. + + +Use the secret to create the `Issuer` ++++++++++++++++++++++++++++++++++++++ + +Use the secret created above to add an ``Issuer`` resource for the ``minio-operator`` namespace. + +1. Create a file called ``operator-ca-issuer.yaml`` with the following contents: + + .. code-block:: yaml + + # operator-ca-issuer.yaml + apiVersion: cert-manager.io/v1 + kind: Issuer + metadata: + name: minio-operator-ca-issuer + namespace: minio-operator + spec: + ca: + secretName: operator-ca-tls + + +2. Apply the resource to the cluster + + .. code-block:: shell + + kubectl apply -f operator-ca-issuer.yaml + +Create TLS certificate for STS service +++++++++++++++++++++++++++++++++++++++ + +Now that the ``Issuer`` exists in the ``minio-operator`` namespace, cert-manager can add a certificate. + +The certificate from cert-manager must be valid for the following DNS domains: + +- ``sts`` +- ``sts.minio-operator.svc.`` +- ``sts.minio-operator.svc.`` + + .. important:: + + Replace ```` with the actual values for your MinIO tenant. + ``cluster domain`` is the internal root DNS domain assigned in your Kubernetes cluster. + Typically, this is ``cluster.local``, but confirm the value by checking your coredns configuration for the correct value for your Kubernetes cluster. + + For example: + + .. code-block:: shell + :class: copyable + + kubectl get configmap coredns -n kube-system -o jsonpath="{.data}" + + Different Kubernetes providers manage the root domain differently. + Check with your Kubernetes provider for more information. + +1. Create a ``Certificate`` for the domains mentioned above: + + Create a file named ``sts-tls-certificate.yaml`` with the following contents: + + .. code-block:: yaml + :class: copyable + :emphasize-lines: 7,12 + + # sts-tls-certificate.yaml + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: sts-certmanager-cert + namespace: minio-operator + spec: + dnsNames: + - sts + - sts.minio-operator.svc + - sts.minio-operator.svc.cluster.local + secretName: sts-tls + issuerRef: + name: minio-operator-ca-issuer + + .. important:: + + The ``spec.secretName`` is not optional! + + The secret name **must be** ``sts-tls``. + Confirm this by setting ``spec.secretName: sts-tls`` as highlighted above. + + +2. Apply the resource to the cluster: + + .. code-block:: shell + :class: copyable + + kubectl apply -f sts-tls-certificate.yaml + +This creates a secret called ``sts-tls`` in the ``minio-operator`` namespace. + +.. warning:: + + Failing to provide the ``sts-tls`` secret containing the TLS certificate or providing an invalid key-value pair in the secret will prevent the STS service from starting. + +Install Operator with Auto TLS disabled for STS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When installing the Operator deployment, set the ``OPERATOR_STS_AUTO_TLS_ENABLED`` environment variable to ``off`` in the ``minio-operator`` container. + +Disabling this environment variable prevents the MinIO Operator from issuing the certificate for STS. +Instead, Operator waits for the TLS certificate issued by cert-manager. + +There are several options for defining an environment variable. +The steps below define the variable with kustomize. + +1. Create a kustomization patch file called ``kustomization.yaml`` with the below contents: + + .. code-block:: yaml + :class: copyable + + # minio-operator/kustomization.yaml + apiVersion: kustomize.config.k8s.io/v1beta1 + kind: Kustomization + + resources: + - github.com/minio/operator/resources + + patches: + - patch: |- + apiVersion: apps/v1 + kind: Deployment + metadata: + name: minio-operator + namespace: minio-operator + spec: + template: + spec: + containers: + - name: minio-operator + env: + - name: OPERATOR_STS_AUTO_TLS_ENABLED + value: "off" + - name: OPERATOR_STS_ENABLED + value: "on" + +2. Apply the kustomization resource to the cluster: + + .. code-block:: shell + :class: copyable + + kubectl apply -k minio-operator + +Manage tenant TLS certificates with cert-manager +------------------------------------------------ + +The following procedures create and apply the resources necessary to use cert-manager for the TLS certificates within a tenant. + +The procedures use ``tenant-1`` as the name of the tenant. +Replace the string ``tenant-1`` throughout the procedures to reflect the name of your tenant. + +Create the tenant namespace CA Issuer +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Before deploying a new tenant, create a Certificate Authority and Issuer for the tenant's namespace. + +1. If necessary, create the tenant's namespace. + + .. code-block:: shell + :class: copyable + + kubectl create ns tenant-1 + +2. Request a Certificate for a new Certificate Authority. + + Create a file called ``tenant-1-ca-certificate.yaml`` with the following contents: + + .. code-block:: yaml + :class: copyable + :emphasize-lines: 7,8 + + # tenant-1-ca-certificate.yaml + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: tenant-1-ca-certificate + namespace: tenant-1 + spec: + isCA: true + commonName: tenant-1-ca + secretName: tenant-1-ca-tls + duration: 70128h # 8y + privateKey: + algorithm: ECDSA + size: 256 + issuerRef: + name: selfsigned-root + kind: ClusterIssuer + group: cert-manager.io + + .. important:: + + The ``spec.isCA`` field must be set to ``true`` to create this certificate as a certificate authority. + See the emphasized lines above. + +3. Apply the request file to the cluster. + + .. code-block:: shell + :class: copyable + + kubectl apply -f tenant-1-ca-certificate.yaml + +4. Generate a resource definition for an ``Issuer``. + + Create a file called ``tenant-1-ca-issuer.yaml`` with the following contents: + + .. code-block:: yaml + :class: copyable + + # tenant-1-ca-issuer.yaml + apiVersion: cert-manager.io/v1 + kind: Issuer + metadata: + name: tenant-1-ca-issuer + namespace: tenant-1 + spec: + ca: + secretName: tenant-1-ca-tls + +5. Apply the ``Issuer`` resource definition to the cluster. + + .. code-block:: shell + :class: copyable + + kubectl apply -f tenant-1-ca-issuer.yaml + +Deploy the tenant +~~~~~~~~~~~~~~~~~ + +With the Certificate Authority and ``Issuer`` in place for the tenant's namespace, you can now deploy the object store tenant. + +Create a certificate for the tenant ++++++++++++++++++++++++++++++++++++ + +Request that cert-manager issue a new TLS server certificate for MinIO. +The certificate must be valid for the following DNS domains: + +- ``minio.`` +- ``minio..svc`` +- ``minio..svc.`` +- ``*.-hl..svc.`` +- ``*..svc.`` +- ``*..minio..svc.'`` + +.. important:: + + Replace the filler strings (````) with values for your tenant: + + - ```` is the internal root DNS domain assigned in your Kubernetes cluster. + Typically, this is ``cluster.local``, but confirm the value by checking your coredns configuration for the correct value for your Kubernetes cluster. + + For example: + + .. code-block:: shell + :class: copyable + + kubectl get configmap coredns -n kube-system -o jsonpath="{.data}" + + Different Kubernetes providers manage the root domain differently. + Check with your Kubernetes provider for more information. + + - ``tenant-name`` is the name provided to your tenant in the ``metadata.name`` of the Tenant YAML. + For this example it is ``myminio``. + + - ``namespace`` is the namespace where the tenant is created, the ``metadata.namespace`` notes that in the Tenant YAML. + For this example it is ``tenant-1``. + +1. Request a ``Certificate`` for the domains mentioned above + + Create a file called ``tenant-1-minio-certificate.yaml`` with the following contents: + + .. code-block:: yaml + :class: copyable + + # tenant-1-minio-certificate.yaml + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: tenant-certmanager-cert + namespace: tenant-1 + spec: + dnsNames: + - "minio.tenant-1" + - "minio.tenant-1.svc" + - 'minio.tenant-1.svc.cluster.local' + - '*.minio.tenant-1.svc.cluster.local' + - '*.myminio-hl.tenant-1.svc.cluster.local' + - '*.myminio.minio.tenant-1.svc.cluster.local' + secretName: myminio-tls + issuerRef: + name: tenant-1-ca-issuer + + .. tip:: + + For this example, the Tenant name is ``myminio``. + We recommend naming the secret in the field ``spec.secretName`` as ``-tls``, following the naming convention the MinIO Operator uses when creating certificates without cert-manager. + +2. Apply the certificate resource to the cluster. + + .. code-block:: shell + :class: copyable + + kubectl apply -f tenant-1-minio-certificate.yaml + +Configure the tenant to use the certificate created by cert-manager ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ In the tenant spec, do the following: -* Disable Autocert `spec.requestAutoCert: false`. This instructs Operator to not attempt to issue certificates and instead rely on Cert Manager to provide them in a secret. -* Reference the Secret containing the TLS certificate from the previous step in `spec.externalCertSecret`. - -```yaml -apiVersion: minio.min.io/v2 -kind: Tenant -metadata: - name: myminio - namespace: tenant-1 -spec: -... - ## Disable default tls certificates. - requestAutoCert: false - ## Use certificates generated by cert-manager. - externalCertSecret: - - name: myminio-tls - type: cert-manager.io/v1 -... -``` - -## Trust tenant-1 CA in MinIO Operator - -MinIO Operator can trust as many CA certificates as provided. To do this, create a secret with the prefix `operator-ca-tls-` -followed by a unique identifier in the `minio-operator` namespace. - -MinIO Operator mounts and trusts all certificates issued by the provided CA's. This is required because -MinIO Operator performs health checks using the */minio/health/cluster* endpoint. - -If Operator is not correctly configured to trust the MinIO Certificate (or its CA), you will see an error message like the following in the Operator Pod logs: - -```error -Failed to get cluster health: Get "https://minio.tenant-1.svc.cluster.local/minio/health/cluster": -x509: certificate signed by unknown authority -``` -For more details about health checks, see https://min.io/docs/minio/Kubernetes/upstream/operations/monitoring/healthcheck-probe.html#cluster-write-quorum. - -### Create `operator-ca-tls-tenant-1` secret - -Copy the Cert Manager generated CA public key (ca.crt) into the `minio-operator` namespace. This allows Operator to trust -the cert-manager issued CA and the derived certificates. - -Create a `ca.crt` file containing the CA: -```sh -kubectl get secrets -n tenant-1 tenant-1-ca-tls -o=jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt -``` - -Create the secret: -```sh -kubectl create secret generic operator-ca-tls-tenant-1 --from-file=ca.crt -n minio-operator -``` -> [!TIP] -> In this example we choose a secret name of `operator-ca-tls-tenant-1`. Note the tenant namespace -> `tenant-1` is used as suffix for easy identification of which namespace the CA is coming from. \ No newline at end of file +- Disable Autocert by setting the ``spec.requestAutoCert`` field to ``false``. + + This instructs the MinIO Operator to not attempt to issue certificates and instead rely on cert-manager to provide them in a secret. +- Reference the Secret containing the TLS certificate from the previous procedure in `spec.externalCertSecret`. + + +1. Modify the tenant YAML ``spec`` section to reflect the above + + .. code-block:: yaml + :emphasize-lines: 6,9,11 + + apiVersion: minio.min.io/v2 + kind: Tenant + metadata: + name: myminio + namespace: tenant-1 + spec: + ... + ## Disable default tls certificates. + requestAutoCert: false + ## Use certificates generated by cert-manager. + externalCertSecret: + - name: myminio-tls + type: cert-manager.io/v1 + ... + +Trust the tenant's CA in MinIO Operator +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +MinIO Operator can trust as many CA certificates as provided. + +To do this, create a secret with the prefix ``operator-ca-tls-`` followed by a unique identifier in the `minio-operator` namespace. + +MinIO Operator mounts and trusts **all** certificates issued by the provided Certificate Authorities. +This is required because the MinIO Operator performs health checks using the ``/minio/health/cluster`` endpoint. + +If Operator is not correctly configured to trust the MinIO tenant's Certificate (or its CA), you will see an error message like the following in the Operator Pod logs: + +.. code-block:: shell + + Failed to get cluster health: Get "https://minio.tenant-1.svc.cluster.local/minio/health/cluster": + x509: certificate signed by unknown authority + +For more details about health checks, refer to :ref:`Healthcheck API `. + +Create ``operator-ca-tls-tenant-1`` secret +++++++++++++++++++++++++++++++++++++++++++ + +Copy the tenant's cert-manager generated CA public key (``ca.crt``) into the `minio-operator` namespace. +This allows Operator to trust the cert-manager issued CA and all certificates derived from it. + +1. Create a `ca.crt` file containing the CA: + + .. code-block:: shell + :class: copyable + + kubectl get secrets -n tenant-1 tenant-1-ca-tls -o=jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt + +2. Create the secret: + + .. code-block:: shell + :class: copyable + + kubectl create secret generic operator-ca-tls-tenant-1 --from-file=ca.crt -n minio-operator + +.. tip:: + + In this example we chose a secret name of ``operator-ca-tls-tenant-1``. + Note the tenant namespace ``tenant-1`` is used as suffix for easy identification of which namespace the CA is coming from. + Use the name of your tenant namespace for easier linking secrets to the related resources. \ No newline at end of file diff --git a/source/operations/monitoring/healthcheck-probe.rst b/source/operations/monitoring/healthcheck-probe.rst index 60a8ebfe9..db59d8ba0 100644 --- a/source/operations/monitoring/healthcheck-probe.rst +++ b/source/operations/monitoring/healthcheck-probe.rst @@ -39,6 +39,8 @@ a Prometheus :ref:`alert ` using the ``minio_cluster_nodes_offline_total`` metric to detect whether one or more MinIO nodes are offline. +.. _minio-cluster-write-quorum: + Cluster Write Quorum -------------------- diff --git a/source/operations/network-encryption.rst b/source/operations/network-encryption.rst index 9a1bfe06a..d526b5be9 100644 --- a/source/operations/network-encryption.rst +++ b/source/operations/network-encryption.rst @@ -64,6 +64,12 @@ Enabling TLS If you have a custom Subject Alternative Name (SAN) certificate that is *not* also a wildcard cert, the TLS certificate SAN **must** apply to the hostname for its parent node. Without a wildcard, the SAN must match exactly to be able to connect to the tenant. + Certificate Management with cert-manager + ---------------------------------------- + + Rather than then MinIO Operator managing certificates, you can configure the deployment to use `cert-manager `__. + For instructions for deploying the MinIO Operator and tenants using cert-manager, refer to the :ref:`cert-manager page `. + .. cond:: linux