Document approaches for infrastructure providers to consider for securing sensitive bootstrap data

[ncdc]

User Story

As a user/operator, I would like to avoid exposing sensitive data (such as certificate private keys) in hypervisor bootstrap data to minimize attack surfaces.

Detailed Description

CAPI generates sensitive cluster data (such as private keys) for the apiserver, etcd, etc. These are stored as secrets in Kubernetes. The kubeadm bootstrapper copies the contents of the secrets into bootstrap data in the KubeadmConfig resource, which is then copied into the Machine resource. Infrastructure providers such as CAPA and CAPV (and presumably others that use cloud-init style bootstrap data) use this bootstrap data as the user data for the VM/hypervisor. In many/most instances, if a user has read-only access to the VM in the provider's API (such as AWS's ec2:DescribeInstances permission), this likely grants them access to the user data, and therefore access to the sensitive data.

Ideally, the VM itself has permission to read the sensitive data, but a user with cloud provider API access to the VM must not be able to do so.

The method used to secure this sensitive data is likely specific to each infrastructure provider. We should document what types of mechanisms to look for (such as a secure data store) so the developers of an infrastructure provider can implement this functionality.

/kind documentation
/priority important-soon
/milestone v0.3.0

cc @akutz @detiber @randomvariable

[randomvariable]

/assign

[detiber]

@ncdc I would also probably add that beyond just the secrets injected during init, the bootstrap tokens and encryption key (for control plane) that we inject for join also fall under this category as well.

[ncdc]

Right, I meant to mention the bootstrap tokens too. What encryption key are you referring to?

[detiber]

Ah, I didn't realize we were still injecting the CA and service account keypairs instead of leveraging the ability to transfer the secrets using kubeadm.

[randomvariable]

We need to do the injection in any case for the initial control plane node.

[vincepri]

/milestone Next

[vincepri]

/help

[alexeldeib]

this and #1846 are very similar/dupes

[alexeldeib]

@vincepri @fabriziopandini Started a doc and expanded on the existing PR. The doc is fairly short right now, I mostly wanted to highlight the API types and how the implementation would change from #1860

#3038

https://docs.google.com/document/d/1R_E3uu8Uvfe4pOcTrfQYUolJCK8mUS58Kq4A8zmqTr8/edit?usp=sharing

[alexeldeib]

/assign

[alexeldeib]

/remove help

[alexeldeib]

/remove-help

[randomvariable]

Important to note that the history of this also included advising implementers to use provider specific features where necessary, for example kubernetes-sigs/cluster-api-provider-aws#1490 and kubernetes-sigs/cluster-api-provider-vsphere#582

[alexeldeib]

Ah, I see there are two side to this issue:

• Bootstrap provider needs secure sources for data (what i'm tackling)
• Infrastructure providers may need a secure place to put that data (the issues you linked)

Perhaps i'll reopen one of the other issues I closed as dupes to reflect the work more accurately, as those are better aligned to the task. But i'm also about to open a PR to fix it, so maybe not a huge deal to disambiguate shrugs

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fabriziopandini]

/remove-lifecycle stale

[vincepri]

/lifecycle frozen

[randomvariable]

related to kubernetes-sigs/cluster-api-provider-aws#1875

[vincepri]

/close

Closing in favor of #3762

[k8s-ci-robot]

@vincepri: Closing this issue.

In response to this:

/close

Closing in favor of #3762

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.