Kuberos running in another Kubernetes cluster

[Miouge1]

Hi,

I run a single Kuberos instance for several Kubernetes clusters:

• production cluster (has kuberos)
• staging cluster (no kuberos, but uses the same OIDC config as production cluster)

Both clusters have letsencrypt CA, so I don't prefer to not include any certificate-authority-data.

The thing is that by default Kuberos tries to be smart and adds the /var/run/secrets/kubernetes.io/serviceaccount as a certificate-authority-data but that's not really what I'm after.

As a work around I have added the letsencrypt ca in certificate-authority-data is there a better way to address this? Maybe a flag --guess-ca?

[negz]

This sounds like a bit of a tricky one. As you say, there's two ways to load CA data at the moment: either by reading them from the kubeconfig template file, or (in single cluster deployments) loading them from the service account.

I have to admit I'm a little unfamiliar with Let's Encrypt in practice. Could you explain a little more around why you don't want to use the certificate-authority-data field of the Kuberos template?

[Miouge1]

The thing is that letsencrypt certs are already trusted by the system (ie it's not self signed, example: https://letsencrypt.org/) so you don't need to give the certificate-authority-data as it's already somewhere in /etc/ssl/.

[Bekt]

We're on a internal network and connect to our cluster with insecure-skip-tls-verify: true. When this option is specified, certificate-authority-data is not allowed. It would be nice not to generate certificate-authority-data by default.

error: specifying a root certificates file with the insecure flag is not allowed

[Miouge1]

@negz what do you think about a boolean flag to control that?

[ripta]

@Miouge1 - What happens when you don't mount in a service account by modifying kuberos's pod spec to have automountServiceAccountToken: false (doc)?

Doing so removes the default service account from being mounted into kuberos at all, and the original intention of #36 was to have kuberos leave certificate-authority-data alone.

[negz]

Hi all,

Just wanted to apologise that I haven't been very active in this conversation. It sounds like a problem worth solving, but I want to think a little more about how we solve it. Unfortunately I haven't had a lot of time for Kuberos these last few weeks, but I hope my schedule will clear up soon.

[Bekt]

@ripta thanks a lot for the context! That solved our issue. Adding that to the pod spec no longer auto-generates certificate-authority-data.