aws ec2 auth: allow role to have multiple bindings

[mfischer-zd]

As a user, I want to be able to authenticate my instance to a particular role via AWS EC2 authentication if that instance is a member of any of a set of account IDs, AMI IDs, IAM instance profiles, or IAM roles.

As a practical matter I think this means making the bound_* attributes lists instead of single values, and checking the instance for a match against the lists.

[mfischer-zd]

Here's the situation, in case you're wondering where my request is coming from:

We use CloudFormation to provision our IAM resources (roles and instance profiles) in addition to the EC2 instances that are associated with them. The CloudFormation-generated names follow the pattern <stack_name>-<logical_id>-<random-id>, for example, my-stack-SomeIamRole-YXAUT1ZYS8NX.

(In the above example, SomeIamRole is an arbitrary logical ID defined in the template. CloudFormation doesn't enforce names that reflect their types, although it's a best practice. Amazon is free to change the automatic naming scheme in the future, so it's dangerous to rely too much upon the autogenerated names.)

Currently, an instance that wishes to authenticate to Vault via EC2 authentication must present a role that is pre-configured by the administrator. (If no role is specified, the instance ID is used.) I'd like to provision a Vault auth role for the Vault client that reflects the permission set it will be granted, but I can't do that because I am currently permitted only a single IAM role or profile binding to match. So if I have multiple stacks, or multiple IAM roles or profiles in a stack, I will have to create multiple Vault roles. And if there are multiple Vault auth roles that exist only for instance profile matching purposes, it'll be difficult for the client to know which one to provide in the authentication request. This presents a difficult handshaking dilemma.

I tried to work around this problem by creating an IAM role that has no IAM permissions and is used only by Vault for matching purposes, but it turns out that one cannot associate more than one IAM role to an instance profile (even though the object is an array). So that's a dead end.

Other ideas welcome.

[jefferai]

ICYMI, the IAM role ARN/instance profile ARN actually perform prefix matching. Does this help? I'm not sure I exactly follow the issue -- if it's related to having to match the ARN, or something else. Per AWS docs it seemed like only a single profile existed for an instance and only a single iam role could be referenced in the instance profile.

[mfischer-zd]

ICYMI, the IAM role ARN/instance profile ARN actually perform prefix matching. Does this help?

I'm the one who asked for the feature, and thanks for implementing it, but sadly, it doesn't help for this particular purpose.

Per AWS docs it seemed like only a single profile existed for an instance and only a single iam role could be referenced in the instance profile.

This is correct.

I'm not sure I exactly follow the issue -- if it's related to having to match the ARN, or something else.

The issue is, how can my client choose which Vault role to present to Vault when attempting to authenticate, if I have to have one Vault role per IAM role/profile prefix? When I create a pile of stacks, I end up with a pile of IAM roles and prefixes, and trying to handshake becomes a big mess. On the other hand, if I can prepopulate a list with a set of prefixes in Vault, it becomes somewhat easier to manage.

[jefferai]

So you're saying that your client has multiple IAM roles (and if so, how does this happen, based on what I know of instance profiles and iam roles)? Or you're saying that you're having trouble mapping more than one client to a single Vault role (and if so, I'm still not grokking why).

[mfischer-zd]

So you're saying that your client has multiple IAM roles (and if so, how does this happen, based on what I know of instance profiles and iam roles)?

No, I'm saying my stack has multiple IAM roles; or I have multiple stacks, each with a different IAM role.

Or you're saying that you're having trouble mapping more than one client to a single Vault role (and if so, I'm still not grokking why).

Yes; because each client has a different IAM role associated with it as a consequence of being in a different stack or autoscaling group.

[joelthompson]

Or you're saying that you're having trouble mapping more than one client to a single Vault role (and if so, I'm still not grokking why).

Yes; because each client has a different IAM role associated with it as a consequence of being in a different stack or autoscaling group.

It's not just that each client has a different IAM role associated with it -- it's that each client has a different, and not necessarily deterministic IAM role or profile (given the way CFN works), which makes the matching not something to depend upon.

Also, FWIW, I submitted a PR for a different authentication backend, #1962, which I think would solve this use case, albeit in a somewhat roundabout way:

• You have a static IAM role, i.e., the "IAM role that has no IAM permissions and is used only by Vault for matching purposes"
• You launch your instances into their non-deterministically named IAM instance profile which maps to a non-deterministically named IAM role.
• That role has permissions to do an sts:AssumeRole on the first role
• The credentials retrieved from the sts:AssumeRole would then be used to sign a GetCallerIdentity request and passed to Vault, authenticating the client to Vault.

It's a little roundabout in that it requires you to have the auxilliary IAM role which you assume first, but it largely matches what you were trying to do and has the further advantage of not needing to update the Vault role every time you launch a new stack/delete an old stack (which I think any scheme using the AWS-EC2 authentication backend would have to do with what you're asking for).

[mfischer-zd]

@joelthompson: That is very clever! Thank you for the example.

Closing in favor of #1962, which seems like a superior solution.

[joelthompson]

Thanks @mfischer-zd :) To be fair, I implemented the same 1:many mapping from IAM principals to Vault roles (i.e., any given Vault role must be mapped to only a single IAM principal, though many different Vault roles could map to the same IAM principal) chiefly because aws-ec2 did the same thing. The role-assumption trick adds a layer of indirection that can avoid the need to break that assumption, but I'm not convinced other use cases won't come up that will need us to change that mapping.

[mfischer-zd]

Actually, I'm going to reopen this just because it's still a good idea, and is easier to implement than role assumption.

[joelthompson]

Hey @mfischer-zd -- I've been thinking a little bit more about this use case.

I feel that this isn't necessarily "easier to implement" for your use case than role assumption. I would argue that it's differently complex. Here, you would still need some sort of external orchestration engine that you trust to take the output from CFN and update the Vault roles (both on stack creation and deletion) to map to the desired instance profiles/IAM roles.

As a workaround, if you're going to need some sort of external process outside of CFN to update Vault, you could just create a new Vault role for every instance profile, with the same name as the instance profile but mapping to your desired set of policies. This would give your instances the role name they need to look up (by querying what their instance profile name is).

Another thought I had about a potential workaround, and this is going to depend very heavily on your use case, but if you're OK putting these instances from different stacks in the same IAM role (which is a plausible assumption since you want them to have the same Vault role), you could do something like:

• Have a "base" CFN stack that creates your IAM roles (and possibly instance profiles)
• Configure your Vault role bound to the IAM role or EC2 instance profile created in the base stack
• Have your other CFN stacks either create new instance profiles and map to the IAM role created in the base stack, or put your new instances all in the profile that was created in the base stack.

AWS now supports cross-stack references which makes this more appealing. However, this makes a heavy assumption about your use case, and so I don't know if it's appropriate for you or not.

Hope one of these workarounds helps for the time being!

[mfischer-zd]

@joelthompson Thanks again for the detailed and useful analysis. We, too, had considered the "base" stack idea as an option.

For my particular use case, I agree that #1962 is the best solution. However, there are other likely scenarios in which an effective and simpler solution would be to allow different IAM role/profile/AMI selectors (bindings) to map to a single Vault auth role, so that's why I reopened this issue.