Feature Request: Secret versioning and auto-update

[derrickburns]

One issue with the current approach is that it is impossible to know what version of a secret is currently deployed to a cluster nor whether the current secret is the one stored in the backing store.

I propose adding an opaque versionID field to the ExternalSecrets CRD (like ContainerSolutions controller) AND (most importantly) a watcher that updates Git (i.e. GitOps) when the backend detects a new version of the secret.

In this way, the secret is kept up to date AND one can tell which version of the secret is stored in any cluster that is watching that GitHub repo.

[Sytten]

I also had this requirement from my team, it also hard to tell if the SM secret changes made it to the kube secret before deploying a new version of your app.

[jeffpearce]

Hi @derrickburns and @Sytten . Can you propose how we might change the CRD to allow this as an option?

[Sytten]

We are kind of limited by the small feature set exposed for secrets but I would suggest the following:

• Being able to specify a version ID for each SM secret used in the ExternalSecret (defaults to latest). That way we know what secret should be in the k8s secret at any given time
• Add a version as an annotation of the k8s secret so we can see & automate the check for secret rotation before deploying a service. I am still unsure how that version should pan out.
• Fire an event when a secret is modified, deleted, created. That is more information we can use to automate the secret rotation before a service starts.

I think this would be a good starting point for more visibility though I am still unsure if this is enough to have a really robust system.

[derrickburns]

I would not suggest changing the CRD, but rather change the installlation options for the service. See fluxcd.

You need to pass the Git repo url and generate or accept a key that can be uploaded to Git to store updates to the repo.

[derrickburns]

On second thought, you do need to track the version of the currently installed secret and the version of the secret as stored in Git. You also need a configuration parameter that signals whether the secret is frozen or can be updated. If the secret has been deleted from AWS but the ExternalSercret still exists, you need to indicate in the Git version in Git that the secret is in the DELETED/MISSING state. So, you need at least 2 bits for the state.

In total:

• InstalledVersion
• AvailableVersion
• State

[Sytten]

I am still unsure if modifying git from kubernetes is really a good idea, sounds like a logistical nightmare to me and potential leaks of keys that key modify source code! Plus it is not very helpful for the user to know the version of the ExternalSecret, only the Secret is important. I think having the observability in the cluster is enough for a first step. It would already be a massive improvement.

[derrickburns]

Check out the [flux project](https://github.com/fluxcd/flux). It is now a CNCF project.  It is very popular and does what I am suggesting.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[Sytten]

Not stale

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[Sytten]

Not stale

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[Sytten]

Not stale

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

[github-actions]

This issue was closed because it has been stalled for 30 days with no activity.

[rohanmathure]

Not stale

[Flydiverny]

@rohanmathure see

• KES is deprecated, migrate to ESO! #864