Is it Possible to Encrypt All/Part of a values.yaml that you are feeding to a Helm Chart?

[jseiser]

We are coming from helmfile, https://github.com/roboll/helmfile#environment-secrets , which allows you to encrypt parts of the values that you are feeding into helm charts.

I have not been able to quite figure this out in Flux.

Any help in the right direction would be great.

Thanks,

[stefanprodan]

You can use HelmRelease.spec.valuesFrom and point helm-controller to a Kubernetes Secret that holds the secret values.yaml. Then in Git, you'll encrypt the secret with SOPS. When kustomize-controller applies the HelmRelease and the Secret manifests, it will first decrypt the secret, then helm-controller will use it for setting the values.

Docs:

• https://fluxcd.io/flux/guides/mozilla-sops/
• https://fluxcd.io/flux/guides/helmreleases/#refer-to-values-in-configmap-and-secret-resources

[jseiser]

@stefanprodan

Are there any working example of this?

I have bounced around between slack and the documentation but havent been able to find and end-to-end working example.

This was my last attempt: https://gist.github.com/jseiser/1131558bb121ca6ff099cf85cc0e3e85

The secret is created. The data is there. Its getting decrypted as well, because helm returns values.

[ec2-user@ip-10-17-1-39 ~]$ helm get values kube-prometheus-stack -n monitoring
USER-SUPPLIED VALUES:
data: |-
  defaultRules:
    create: true
    rules:

Same output, but from my other cluster where this chart is deployed by Terraform.

[ec2-user@ip-10-16-3-202 ~]$ helm get values kube-prometheus-stack -n monitoring
USER-SUPPLIED VALUES:
alertmanager:
  alertmanagerSpec:

[jseiser]

OK,

I am still not able to find documentation on how to properly use 'secretGenerator'. The problem is def. that the secret does not get formatted correctly, but again, i have no idea how to get it formatted correctly using the secretGenerator.

I was able to copy this: https://github.com/fluxcd/helm-controller/blob/main/config/testdata/valuesfrom/secret.yaml

And encrypt the secret myself, using sops. Then pass THAT into the helmrelease and it works, but its less than ideal since when you update the secret, it does not trigger the helm release to update.

[stefanprodan]

The problem is def. that the secret does not get formatted correctly

Values is yaml, so you need to pass the input format to sops if you’ve removed the file extension, see https://fluxcd.io/flux/components/kustomize/kustomization/#kustomize-secretgenerator

[jseiser]

@stefanprodan

So i have been through that documentation, and we went through it in slack, and no one was able to actually get it to work.

actual file

❯ cat infrastructure/kube-prometheus-stack/prom-stack-values
defaultRules:
  create: true
  rules:
    alertmanager: true
    etcd: true
    configReloaders: true
    general: true

with a .sops.yaml that looks like this, which is what is covered in the flux sops documentation.

❯ cat .sops.yaml
creation_rules:
  - path_regex: .*.yaml
    encrypted_regex: ^(data|stringData)$

❯ sops -e --input-type=yaml infrastructure/kube-prometheus-stack/prom-stack-values > values.encrypted
error loading config: no matching creation rules found

So making sops.yaml look like

  - filename_regex: values.encrypted$
    age: >-

❯ sops -e --input-type=yaml infrastructure/kube-prometheus-stack/prom-stack-values > values.encrypted

The file gets all messed up.

❯ cat values.encrypted
{
	"defaultRules": {
		"create": "ENC[AES256_GCM,data:ZDcN8g==,iv:rsTh/VQjZZu7bQlVLhZyEQmt67CNq5P2/d3+7QxGFc0=,tag:d7JWae2rZA4R921IZozA7Q==,type:bool]",
		"rules": {
			"alertmanager": "ENC[AES256_GCM,data:q1OkWw==,iv:tGPZ5g+v4g/0UHqdKV0t2TkurCuxmJgxMmxxJr+Eo0A=,tag:YXyoK6BJp54lIwVxPGbQnw==,type:bool]",
			"etcd": "ENC[AES256_GCM,data:VXWyLw==,iv:je6rtAnzmAAwCadU3NGyHAc5JcuB+/5fdVTofAq+a+A=,tag:jfei3DIScKYyoyqwq5SZKw==,type:bool]",

Is there any end-to-end working example of Flux + secretGenerator ?

Thanks.

[stefanprodan]

Please try sops -e --input-type=yaml --output-type=yaml infrastructure/kube-prometheus-stack/prom-stack-values > values.encrypted

[jseiser]

Will Do. Mulțumesc

[jseiser]

Doing it that way, just encrypted part of the yaml.

❯ cat values.encrypted
defaultRules:
    create: ENC[AES256_GCM,data:fbZBuQ==,iv:WK5AJ/Eo9VicIjqEYfZ4fuOa9wfbVfl4qsLp9hrm2kM=,tag:7ISjtYeCSmPk+ADVW1MEBw==,type:bool]
    rules:
        alertmanager: ENC[AES256_GCM,data:M0RDgA==,iv:Ejzsr0ZWwYzmYNOUcJojMBJ8CDH7V/AN/xQI6EUaTxY=,tag:0zFJOkFkk5doFKblvLjwXQ==,type:bool]
        etcd: ENC[AES256_GCM,data:fdTZtg==,iv:0oDiUfgGVDvzCUw1Zr0YtB2wtmtG7Nyd6lb9TVfn5kg=,tag:0vkCF/OjVw+IETaNteP0oQ==,type:bool]
        configReloaders: ENC[AES256_GCM,data:6MDcKA==,iv:z5zasC5B8W2ePXnjzLfKG5xTFgcfzQ/o3P9X1s5mrZg=,tag:mon2W+ZY34GwujhOcO/bQw==,type:bool]

[stefanprodan]

That's how SOPS works, why would the keys be secret?

[jseiser]

Why would I know the answer to that mate. Coming from something like helmfile, sops encrypts the entire file, and helm-secret decrypts it.

Ill try to deploy this one.

[jseiser]

@stefanprodan

This appeared to work.

https://gist.github.com/jseiser/eed3b4ffdd1ff3def604c4c78c989c1f

[stefanprodan]

Yeah the issue was that the format had to be set for both input and output, most users don't run into this as they have values.yaml on disk and they override it with the encrypted version, having .yaml SOPS automatically detects the format. I'll amend the generator docs to add --output-type.

[kingdonb]

I wrote this PR to try to cover this with docs:

fluxcd/website#1165

Perhaps my PR can be shorter with your change to the sops generator docs (there is too much text here, but there are also a lot of things to trip over.)

[harvetech]

Sops encrypt the complete values.yaml file, which is definitely not needed, and only the part where the secrets are should be encrypted. I also understand It's hard to find which parts should be encrypted and which not but encrypting the complete file also does not feel right.

[makkes]

You are free to combine values from different sources as explained in the docs. That way, you'll have all non-sensitive data in the HelmRelease spec or a ConfigMap and all sensitive data in a separate, encrypted Secret.