Unable to read Vault KVv2 secrets metadata #1396
Comments
|
I have a suspicion the error occurs in consul-template/dependency/vault_read.go Lines 147 to 149 in 3250aa0 consul-template/dependency/vault_common.go Lines 362 to 375 in 3250aa0 imo, the code should check specifically for existing |
|
I found a workaround. (and probably another bug?) This Consul template fetches metadata from Vault KV v2.
The metadata endpoint is useful for fetching available secrets versions. It allows logic like iterating through a list of versions: The exact template doesn't work though ( |
|
Hey @terorie, thanks for the report. I agree with your assessment that the addPrefixToVKVPath not handling metadata is the issue. Looks like this stopped working with changes to fix issues where it mistakenly saw random 'foodata' entries as matching 'data' and not fixing their path. This broke 'metadata' as it was a good case of that bug, but there was no test asserting that. The fix needs to include both a way to recognize 'metadata' as a valid path and to be sure to add a test for it. |
|
Great find @terorie, this was an overlook on my part and not considering the vault usage of |
|
After digging into this a bit more, I noticed reading metadata was also not working in |
|
Thank you all so much for looking into this! @findkim I'm going to test the changes included in #1399 and report my findings. I'll also test out iterating versioned secrets with this using the aforementioned template. I might have another small bug there, but this is probably just wrong configuration on my end. |
|
I could confirm that #1399 fixes iterating over KVv2 secrets versions. Vault data: vault kv put secret/test_secret abc=def
vault kv get -version 1 secret/test_secret
vault kv put secret/test_secret abc=def
vault kv get -version 2 secret/test_secretConsul Template config: vault {
address = "http://localhost:8200"
token = "token"
}
template {
destination = "./version-1-test.txt"
contents = <<EOF
{{- with secret "secret/data/test_secret?version=1" -}}
{{ .Data }}
{{- end }}
EOF
}
template {
destination = "./version-iteration-test.txt"
contents = <<EOF
{{- with secret "secret/metadata/test_secret" -}}
{{- range $key, $value := .Data.versions -}}
{{- with secret (printf "secret/data/test_secret?version=%s" $key) }}
{{ .Data }}
{{- end }}
{{- end }}
{{- end }}
EOF
}Output: version-iteration-test.txt Output: version-1-test.txt |
eikenb
added
hashicat-update-required
Consul Template version
Vault Agent 1.4.2
Configuration
vault-k8sannotationsGenerated Vault Agent config
{ "auto_auth": { "method": { "type": "kubernetes", "mount_path": "auth/kubernetes", "config": { "role": "api-deployment" } }, "sink": [ { "type": "file", "config": { "path": "/home/vault/.vault-token" } } ] }, "exit_after_auth": false, "pid_file": "/home/vault/.pid", "vault": { "address": "<omitted>" }, "template": [ { "destination": "/vault/secrets/minitoken-mac-secret", "contents": "{{- with secret \"secret/metadata/minitoken/mac_secret\" -}}\n{{ . }}\n{{- end }}\n", "left_delimiter": "{{", "right_delimiter": "}}" } ] }Consul Template
{{- with secret "secret/metadata/minitoken/mac_secret" -}} {{ . }} {{- end }}Sanity Check Template that works
{{- range secrets "secret/metadata/minitoken/" -}} Found secret: {{ . }} {{- end }}Logs
Output of sanity check template
Command
Vault Agent logs
Expected behavior
The
secretcall should be able to read Vault KV v2 metadata directly from thesecret/metadata/<path>value.Actual behavior
It was reported that
no secret existswhen asking for metadata.This is probably due to
vault.read()injecting path segment/data:secret/data/metadata/<path>.A sanity check listing the secrets in
secret/metadatashows that the secret that we read exists (see Configuration section).Steps to reproduce
secret/metadataendpoint referring to secret via template.The text was updated successfully, but these errors were encountered: