Secrets Manager Datasource Doesn't Appear to Work

[sjiveson]

./gomplate --version
gomplate version 4.1.0

aws --version
aws-cli/1.24.10 Python/3.6.8 Linux/4.18.0-553.8.1.el8_10.x86_64 botocore/1.26.10

cat /etc/redhat-release
Red Hat Enterprise Linux release 8.10 (Ootpa)

Hi,

I'm attempting to retrieve secrets using AWS Secrets Manager (on an EC2 instance), however, I'm getting this error:

ec2-user]# ./gomplate --verbose -d 'foo=aws+sm:' -i '{{ ds "foo" "goemplate" }}'
13:32:18 DBG starting gomplate
13:32:18 DBG config is:
---
datasources:
  foo:
    header: {}
    url: 'aws+sm:'
in: '{{ ds "f...'
 version=4.1.0 build=cc2584028866967a39b096265d5b9af4516c734f
13:32:18 DBG completed rendering templatesRendered=0 errors=1 duration=84.088165ms
13:32:18 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"foo\" \"goemplate\">: error calling ds: couldn't read datasource 'foo' (aws+sm:///goemplate): stat (url: \"aws+sm:///\", name: \"goemplate\"): stat goemplate: file does not exist"

Looking at my CloudTrail logs, I'm seeing this:

...
errorCode	ResourceNotFoundException
errorMessage	Secrets Manager can't find the specified secret.
eventCategory	Management
eventID	NNee72f2-f237-4467-81b4-fe0d81f8cNNN
eventName	GetSecretValue
eventSource	secretsmanager.amazonaws.com
eventTime	2024-07-23T13:23:37Z
eventType	AwsApiCall
eventVersion	1.09
managementEvent	1
readOnly	1
requestParameters.secretId	/goemplate
...

As you can see on that last line, a forward slash is being included at the start of the secretId when it shouldn't be. This also occurs if I specify, for instance, goemplate/something or goemplate/something/something.

Any assistance appreciated and thanks for a great tool.

[hairyhenderson]

I think this may a duplicate to #2180 - as a workaround, try setting the region?

[sjiveson]

It's not that unfortunately, the error message when I hadn't was very clear which was nice:

15:46:40 ERR  err="renderTemplate: failed to render template one.txt.tmpl: template: one.txt.tmpl:1:32: executing \"one.txt.tmpl\" at <ds \"testsource\" \"/goemplate/awssm/test\">: error calling ds: couldn't read datasource 'testsource' (aws+sm:///goemplate/awssm/test): stat (url: \"aws+sm:///\", name: \"goemplate/awssm/test\"): stat test: getSecretValue: operation error Secrets Manager: GetSecretValue, failed to resolve service endpoint, endpoint rule error, Invalid Configuration: Missing Region"

[hairyhenderson]

Ah - this reminds me of some strangeness I had to deal with in the old v3 implementation. Looks like some of that got lost in the v4/go-fsimple shuffle.

[github-actions]

This issue is stale because it has been open for 60 days with no
activity. If it is no longer relevant or necessary, please close it.
Given no action, it will be closed in 14 days.

If it's still relevant, one of the following will remove the stale
marking:

• A maintainer can add this issue to a milestone to indicate that
  it's been accepted and will be worked on
• A maintainer can remove the stale label
• Anyone can post an update or other comment

[sjiveson]

I'm not capable but a fix would still be much appreciated

[github-actions]

This issue is stale because it has been open for 60 days with no
activity. If it is no longer relevant or necessary, please close it.
Given no action, it will be closed in 14 days.

If it's still relevant, one of the following will remove the stale
marking:

• A maintainer can add this issue to a milestone to indicate that
  it's been accepted and will be worked on
• A maintainer can remove the stale label
• Anyone can post an update or other comment

[sjiveson]

And again. I'm not capable but a fix would still be much appreciated

[hairyhenderson]

@sjiveson I'm sorry I haven't been able to solve this yet. My time available to work on gomplate is limited. I'll try to find some time over the next few days.

[hairyhenderson]

@sjiveson I've actually started digging into this a bit - just to double-check - goemplate isn't a typo right?

And, have you tried referring to the key directly?

$ gomplate --verbose -d 'foo=aws+sm:goemplate' -i '{{ ds "foo" }}'

It's somewhat unusual to specify the whole aws+sm scheme as the datasource, though should not be invalid.

I think what's going on is the URL parsing has no special-case for the aws+sm scheme anymore - the non-prefixed argument is being interpreted as a regular path rooted in /, whereas with aws+sm the old behaviour was to build an opaque URL from that (with no leading /).

[sjiveson]

Hey. Thank you for responding - I know the feeling well, hence my own delay in responding 😏 .

It was a typo with the secret name but I didn't bother changing it in Secrets Manager and it is the correct name of the secret.

Referring to the key directly as you suggested - v3:

AWS_REGION="eu-west-2" ./gomplatev3 --verbose -d 'foo=aws+sm:goemplate' -i '{{ ds "foo" }}'
17:22:10 DBG starting gomplate
17:22:10 DBG config is:
---
datasources:
  foo:
    header: {}
    url: aws+sm:goemplate
in: '{{ ds "f...'
 build=f525a6ea version=3.11.8
SECRET-VALUE-HERE17:22:11 DBG completed rendering duration=0.119866419 errors=0 templatesRendered=1

With v4:

AWS_REGION="eu-west-2" ./gomplatev4 --verbose -d 'foo=aws+sm:goemplate' -i '{{ ds "foo" }}'
17:22:19 DBG starting gomplate
17:22:19 DBG config is:
---
datasources:
  foo:
    header: {}
    url: aws+sm:goemplate
in: '{{ ds "f...'
 version=4.1.0 build=cc2584028866967a39b096265d5b9af4516c734f
17:22:19 DBG completed rendering templatesRendered=0 errors=1 duration=174.397089ms
17:22:19 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"foo\">: error calling ds: couldn't read datasource 'foo' (aws+sm:goemplate): stat (url: \"aws+sm:///\", name: \"goemplate\"): stat goemplate: file does not exist"

I should be able to respond more quickly now I'm setup to test again.

[hairyhenderson]

Thanks @sjiveson - that's strange. I've set up a few secrets in Secrets Manager and I think this is indeed the path issue I described above:

$ aws secretsmanager create-secret --name one --secret-string "one's value"
...
$ aws secretsmanager create-secret --name /two --secret-string "two's value"
...
$ aws secretsmanager create-secret --name dir/three --secret-string "three's value"
...
$ aws secretsmanager create-secret --name /dir/four --secret-string "four's value"
...

So there should be 4 secrets, accessible as aws+sm:one, aws+sm:///two, aws+sm:dir/three, and aws+sm:///dir/four.

one fails in the same way yours does:

$ gomplate -d secret=aws+sm:one -i '{{ ds "secret" }}'
11:03:16 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"secret\">: error calling ds: couldn't read datasource 'secret' (aws+sm:one): stat (url: \"aws+sm:///\", name: \"one\"): stat one: file does not exist"

/two works:

$ gomplate -d secret=aws+sm:///two -i '{{ ds "secret" }}'
two's value

dir/three also fails:

$ bin/gomplate -d secret=aws+sm:dir/three -i '{{ ds "secret" }}'
11:04:05 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"secret\">: error calling ds: couldn't read datasource 'secret' (aws+sm:dir/three): stat (url: \"aws+sm:///\", name: \"dir/three\"): stat three: file does not exist"

/dir/four succeeds:

$ bin/gomplate -d secret=aws+sm:///dir/four -i '{{ ds "secret" }}'
four's value

[hairyhenderson]

I thought this was perhaps a go-fsimpl bug, however I'm not convinced now. The test program (fscli) works as expected:

$ bin/fscli_darwin-amd64 -base-url aws+sm: cat one 
one's value
$ bin/fscli_darwin-amd64 -base-url aws+sm:/ cat two
two's value

So, the bug is gomplate-specific.

[hairyhenderson]

Ok I think I have a fix at #2284

[hairyhenderson]

Just checked this again and it seems I missed one use-case:

$ gomplate --verbose -d 'sm=aws+sm:' -i '{{ ds "sm" "mysecret" }}'
23:26:51 DBG starting gomplate
23:26:51 DBG config is:
---
datasources:
  sm:
    header: {}
    url: 'aws+sm:'
in: '{{ ds "s...'
 version=4.2.1-91-g7eb7829d build=7eb7829d
23:26:51 DBG completed rendering templatesRendered=0 errors=1 duration=103.26749ms
23:26:51 ERR  err="renderTemplate: failed to render template <arg>: template: <arg>:1:3: executing \"<arg>\" at <ds \"sm\" \"mysecret\">: error calling ds: couldn't read datasource 'sm' (aws+sm:///mysecret): stat (url: \"aws+sm:///\", name: \"mysecret\"): stat mysecret: file does not exist"

Reopening to fix that...

[sjiveson]

Many thanks for your efforts - that and your time are much appreciated. Cheers