Cloud KMS secret data source

[rochdev]

It would be nice to have a google_kms_secret data source similar to aws_kms_secret. This would allow managing secrets with Cloud KMS.

[danawillow]

Hey @rochdev, Google's Cloud KMS doesn't store secrets, it encrypts them and allows them to be stored elsewhere. Would a storage bucket / bucket object data source work for your use case? If you could give us a bit more detail about your use case and how the various resources you're planning on using would be interacting, this could help us figure out how to approach this :)

(also in case you haven't seen, #441 is tracking KMS resources)

[rochdev]

The use case would be to encrypt secrets with Cloud KMS, which could then be used as a data source in the Terraform files.

Basically it would be: encrypt a secret using Cloud KMS, put that secret as the payload of the data source directly in the .tf file, and when using it the data source would decrypt is automatically using KMS.

See the aws_kms_secret documentation for an example using the AWS equivalent.

#441 is for the Terraform resources to actually create and manage Cloud KMS resources (which I am also looking forward to :), but this feature request is for a data source.

I understand Cloud KMS was not built for this purpose (secrets are not necessarily keys), but since it's able to encrypt small payloads I think it would still be great to have this feature.

[rochdev]

Just as an example of how we have achieved this functionality right now with the external provider:

example.tf

data "external" "secret" {
  program = ["${path.module}/example.sh", "${var.encrypted_secret}"]
}

// then use ${lookup(data.external.secret.result, "secret")}

example.sh

secret=$(echo -n "$1" | base64 --decode | gcloud kms decrypt \
  --ciphertext-file=- \
  --plaintext-file=- \
  --key=key \
  --keyring=keyring \
  --location=global
)

jq -nMcr --arg secret "$secret" '{"secret":$secret}'

What would be nice instead would be something along these lines:

data "google_kms_secret" "secret" {
  key         = "key"
  keyring     = "keyring"
  location    = "global"
  ciphertext  = "${var.encrypted_secret}"
}

// then use ${data.google_kms_secret.secret.plaintext}

${var.encrypted_secret} is a secret encrypted with Cloud KMS and then base64 encoded.

This would allow encrypted secrets to be stored in GitHub and decrypted with Cloud KMS on the fly for use in other resources.

[danawillow]

@emilymye for insight

[tragiclifestories]

@mrparkers - did you get anywhere with this? Have just found a use case for it 😉

EDIT: or indeed anyone else ...

[mrparkers]

@tragiclifestories I have it mostly done, I'm just waiting for #692 to be merged so I can use the CryptoKey resource in the acceptance test.

[tragiclifestories]

@mrparkers now's your chance ...

[mrparkers]

Nice! I'll try and create a PR within a few hours.

[migibert]

Hi,

I have a usecase where I need to encrypt terraform computed data using KMS and store the result in GCS.

For now, I used a workaround similar to @rochdev using an external data source but I'd love to use a data source like the one you proposed!

resource "google_service_account" "account" {
  account_id   = "account"
  display_name = "account"
}

resource "google_service_account_key" "account" {
  service_account_id = "${google_service_account.account.id}"
  public_key_type    = "TYPE_X509_PEM_FILE"
  private_key_type   = "TYPE_GOOGLE_CREDENTIALS_FILE"
}

resource "google_kms_key_ring" "ring" {
  name     = "ring"
  location = "global"
}

resource "google_kms_crypto_key" "key" {
  name            = "key"
  key_ring        = "${google_kms_key_ring.ring.id}"
}

data "google_kms_secret" "secret" {
  crypto_key    = "${google_kms_crypto_key.key.id}"
  plaintext     = "${google_service_account_key.account.private_key}"
}

resource "google_storage_bucket" "bucket" {
  name     = "bucket"
  location = "EU"
}

resource "google_storage_bucket_object" "account-key" {
  name             = "account-key.json.encrypted.base64"
  content          = "${google_kms_secret.ciphertext}",
  content_type     = "application/octet-stream",
  bucket           = "${google_storage_bucket.bucket.name}"
}

I understand the security issue mentioned by @rosbo about storing a secret in a tfstate file, but as the problem seems more global and not confined to the google cloud provider, I don't see it like a show stopper. Ciphering or not the tfstate file is the developer responsibility and we should just show him a warning to let him understand the consequences of his choice.

[tragiclifestories]

(For context, @rosbo's comments referred to above are on the PR: #741)

While I agree with you (see my PR comments), I think it is still important to make it easy for people to do the right thing, or at least easier than it is to do the wrong thing. On most TF backends, encryption is a 'some assembly required' matter.

TBH though I think your point is good that it's a 'global' problem and some joined-up thinking is probably necessary from the Terraform team as to how (or if) users should handle secrets in their tfstate.

[rosbo]

We have decided to go ahead and merge this new data source. See discussions in PR #741. Thank you all for contributing to the discussion :)

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!