Skip to content

Commit

Permalink
chore(helm-chart): Rework without implementing transitive secret
Browse files Browse the repository at this point in the history
  • Loading branch information
@mkilchhofer
mkilchhofer committed Jul 18, 2023
1 parent 1d51b6d commit ee37851
Show file tree
Hide file tree
Showing 9 changed files with 68 additions and 308 deletions.
24 changes: 23 additions & 1 deletion .github/workflows/lint-test-chart.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,9 +75,31 @@ jobs:
helm install cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--wait \
--set installCRDs=true \
--set extraArgs='{--enable-certificate-owner-ref}'
- name: Prepare existing secret test scenario
if: fromJSON(steps.changes.outputs.changed)
run: |
openssl req -x509 -newkey rsa:2048 -sha256 -days 365 \
-nodes -keyout ${{ runner.temp }}/tls.key -out ${{ runner.temp }}/tls.crt \
-subj "/CN=metrics-server" \
-addext "subjectAltName=DNS:metrics-server,DNS:metrics-server.kube-system.svc"
kubectl -n kube-system create secret generic metrics-server-existing \
--from-file=${{ runner.temp }}/tls.key \
--from-file=${{ runner.temp }}/tls.crt
cat <<EOF >> charts/metrics-server/ci/tls-existingSecret-values.yaml
apiService:
insecureSkipTLSVerify: false
caBundle: |
$(cat ${{ runner.temp }}/tls.crt | sed -e "s/^/ /g")
EOF
rm ${{ runner.temp }}/tls.key ${{ runner.temp }}/tls.crt
- name: Run chart-testing install
if: fromJSON(steps.changes.outputs.changed)
run: ct install
run: ct install --namespace kube-system
67 changes: 15 additions & 52 deletions charts/metrics-server/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ The following table lists the configurable parameters of the _Metrics Server_ ch
| `topologySpreadConstraints` | Pod Topology Spread Constraints. | `[]` |
| `deploymentAnnotations` | Annotations to add to the deployment. | `{}` |
| `schedulerName` | scheduler to set to the deployment. | `""` |
| `tls.type` | TLS option to use. Either use `metrics-server` for self-signed certificates, `secret`, `existingSecret`, `cert-manager` or `helm` | `"metrics-server"` |
| `tls.certManager.clusterDomain` | Kubernetes cluster domain. Used to configure Subject Alt Names for the certificate | `"cluster.local"` |
| `tls.type` | TLS option to use. Either use `metrics-server` for self-signed certificates, `helm`, `cert-manager` or `existingSecret`. | `"metrics-server"` |
| `tls.clusterDomain` | Kubernetes cluster domain. Used to configure Subject Alt Names for the certificate when using `tls.type` `helm` or `cert-manager`. | `"cluster.local"` |
| `tls.certManager.addInjectorAnnotations` | Automatically add the cert-manager.io/inject-ca-from annotation to the APIService resource. | `true` |
| `tls.certManager.existingIssuer.enabled` | Use an existing cert-manager issuer | `false` |
| `tls.certManager.existingIssuer.kind` | Kind of the existing cert-manager issuer | `"Issuer"` |
Expand All @@ -97,15 +97,10 @@ The following table lists the configurable parameters of the _Metrics Server_ ch
| `tls.certManager.annotations` | Add extra annotations to the Certificate resource | `{}` |
| `tls.certManager.labels` | Add extra labels to the Certificate resource | `{}` |
| `tls.existingSecret.name` | Name of the existing Secret to use for TLS | `""` |
| `tls.secret.annotations` | Add extra annotations to the Secret resource | `{}` |
| `tls.secret.labels` | Add extra labels to the Secret resource | `{}` |
| `tls.secret.crt` | The server certificate to use for metrics-server. Use PEM format | `""` |
| `tls.secret.key` | The private key of the certificate to use for metrics-server. Use PEM format. | `""` |
| `extraObjects` | List of extra manifests to deploy. Will be passed through `tpl` to support templating | `[]` |

## Hardening metrics-server

By default, metrics-server is using a self-signed certificate which is generated during startup. The APIservice is registered with `.spec.insecureSkipTLSVerify` set to `true` as you can see here:
By default, metrics-server is using a self-signed certificate which is generated during startup. The `APIservice` resource is registered with `.spec.insecureSkipTLSVerify` set to `true` as you can see here:

```yaml
apiVersion: apiregistration.k8s.io/v1
Expand All @@ -120,12 +115,13 @@ spec:
#..
```

To harden metrics-server, you have these 4 options described in the following section.
To harden metrics-server, you have these options described in the following section.

### Option 1: Let helm generate a self-signed certificate

This option is probably the easiest solution for you. We delegate helm the generation process for self-signed certificates.
This option is probably the easiest solution for you. We delegate the process to generate a self-signed certificate to helm.
As helm generates them during deploy time, helm can also inject the `apiService.caBundle` for you.

**The only disadvantage of using this method is that it is not GitOps friendly** (e.g. Argo CD). If you are using one of these
GitOps tools with drift detection, it will always detect changes. However if you are deploying the helm chart via Terraform
for example (or maybe even Flux), this method is perfectly fine.
Expand Down Expand Up @@ -154,59 +150,26 @@ tls:
There are other optional parameters, if you want to customize the behavior of the certificate even more.
### Option 3: Provide certificate data
You can use an arbitrary PKI solution and generate a certificate for metrics-server. You need the following data in PEM format:
- The server certificate which is issued with 3 Subject Alt Names:
- `metrics-server.<namespace>`
- `metrics-server.<namespace>.svc`
- `metrics-server.<namespace>.svc.cluster.local`
- The server key file
- The CA certificate *(optional, you can also provide the server certificate to `apiService.caBundle`)*

To use this method, please setup your values file like this:

```yaml
apiService:
insecureSkipTLSVerify: false
caBundle: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
tls:
type: secret
secret:
crt: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
key: |
-----BEGIN RSA PRIVATE KEY-----
...
-----END RSA PRIVATE KEY-----
```

### Option 4: Use existing Secret
### Option 3: Use existing Secret
This option allows you to reuse an existing Secret. This Secrets can have an arbitrary origin, e.g.
- Created via kubectl / Terraform / etc.
- Synced from a secrets management solution like AWS SecretsManager, HashiCorp Vault, etc.
- Synced from a secret management solution like AWS Secrets Manager, HashiCorp Vault, etc.
You still need to pass the CA certificate to ensure proper configuration of the `APIservice` resource,
but the sensitive information (the private key) can be read via Secret. Same as in the previous option "2",
`apiService.caBundle` can also be the server certificate.
When using this type of TLS option, the keys `tls.key` and the `tls.crt` key must be provided in the data field of the
existing Secret.

You need to pass the certificate of the issuing CA (or the certificate itself) via `apiService.caBundle` to ensure
proper configuration of the `APIservice` resource. Otherwise you cannot set `apiService.insecureSkipTLSVerify` to
`false`.

To use this method, please setup your values file like this:

```yaml
apiService:
insecureSkipTLSVerify: false
caBundle:
caBundle: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
Expand Down
85 changes: 0 additions & 85 deletions charts/metrics-server/ci/tls-custom-values.yaml
View file

This file was deleted.

Loading

0 comments on commit ee37851

Please sign in to comment.