Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create data source for aws iam roles #18585

Merged
merged 3 commits into from
Aug 18, 2021

Conversation

jlamande
Copy link

@jlamande jlamande commented Apr 6, 2021

The purpose of this new data source is to provide a way get ARNs and Names of IAM Roles that are created outside of the current Terraform state.

E.g., in an AWS SSO powered environment, IAM Roles are automatically provisioned by AWS SSO in the different AWS accounts of the organization and their names is only following a pattern. The exact name is unpredictable because it contains a random/hash number.

When provisioning resources based on these roles (as Kubernetes aws-auth map) this data source is a must-have.

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #14470
Closes #14173

Output from acceptance testing:

$ make testacc TESTARGS='-run=TestAccAWSDataSourceIAMRoles_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSDataSourceIAMRoles_ -timeout 180m
=== RUN   TestAccAWSDataSourceIAMRoles_basic
=== PAUSE TestAccAWSDataSourceIAMRoles_basic
=== RUN   TestAccAWSDataSourceIAMRoles_filterRoleName
=== PAUSE TestAccAWSDataSourceIAMRoles_filterRoleName
=== RUN   TestAccAWSDataSourceIAMRoles_pathPrefix
=== PAUSE TestAccAWSDataSourceIAMRoles_pathPrefix
=== RUN   TestAccAWSDataSourceIAMRoles_pathPrefixAndFilterRoleName
=== PAUSE TestAccAWSDataSourceIAMRoles_pathPrefixAndFilterRoleName
=== CONT  TestAccAWSDataSourceIAMRoles_basic
=== CONT  TestAccAWSDataSourceIAMRoles_pathPrefixAndFilterRoleName
=== CONT  TestAccAWSDataSourceIAMRoles_pathPrefix
=== CONT  TestAccAWSDataSourceIAMRoles_filterRoleName
--- PASS: TestAccAWSDataSourceIAMRoles_basic (71.14s)
--- PASS: TestAccAWSDataSourceIAMRoles_filterRoleName (76.23s)
--- PASS: TestAccAWSDataSourceIAMRoles_pathPrefix (76.35s)
--- PASS: TestAccAWSDataSourceIAMRoles_pathPrefixAndFilterRoleName (76.37s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	78.687s

@jlamande jlamande requested a review from a team as a code owner April 6, 2021 09:00
@ghost ghost added size/XL Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. provider Pertains to the provider itself, rather than any interaction with AWS. service/iam Issues and PRs that pertain to the iam service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Apr 6, 2021
@github-actions github-actions bot added the needs-triage Waiting for first response or review from a maintainer. label Apr 6, 2021
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @jlamande 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@jlamande jlamande force-pushed the f-data_source-aws_iam_roles branch from ba859b5 to e0f1757 Compare April 6, 2021 16:29
@Marcvd316
Copy link

Marcvd316 commented Apr 6, 2021

Awesome work! Just commenting to add a link to a related issue : #14470

@anGie44 anGie44 added new-data-source Introduces a new data source. and removed needs-triage Waiting for first response or review from a maintainer. labels Apr 6, 2021
@jlamande
Copy link
Author

Hi @anGie44
Any chance to have some feedback ?

@jlamande
Copy link
Author

jlamande commented Jun 9, 2021

Hi @anGie44,
Any chance to have some feedback ?

Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @jlamande , thank you for this PR! It's off to a great start. A couple comments to address given we'll want to support a different argument in place of filter (as it's not natively supported by the API method), but things should fall into place once that is addressed. I've suggested the use of name_regex but if you feel name_prefix or another alternative may be better suited for this data source, feel free to discuss here!

aws/data_source_aws_iam_roles.go Outdated Show resolved Hide resolved
aws/data_source_aws_iam_roles.go Outdated Show resolved Hide resolved
@anGie44 anGie44 added the waiting-response Maintainers are waiting on response from community or contributor. label Jul 28, 2021
@tomelliff
Copy link
Contributor

This is potentially a follow up feature request but it might be useful to also return the ARNs wtih the path stripped. Or at least documentation should show how to get at the ARN without the path in it.

EKS role mapping against AWS SSO roles for some reason requires the path to be removed from the ARN of the role. This is documented in https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html and https://docs.aws.amazon.com/eks/latest/userguide/troubleshooting_iam.html#security-iam-troubleshoot-ConfigMap

@anGie44 anGie44 removed the waiting-response Maintainers are waiting on response from community or contributor. label Aug 18, 2021
@mattoneillphx
Copy link

This PR will be incredibly useful and provides a workaround for #20552, thank you!

Any idea on when this may get released?

@anGie44 anGie44 force-pushed the f-data_source-aws_iam_roles branch 3 times, most recently from a13129c to f7c17b8 Compare August 18, 2021 03:46
@anGie44 anGie44 force-pushed the f-data_source-aws_iam_roles branch from f7c17b8 to c4a413c Compare August 18, 2021 04:10
Comment on lines +58 to +73
### Role ARNs with paths removed

For services like Amazon EKS that do not permit a path in the role ARN when used in a cluster's configuration map

```terraform
data "aws_iam_roles" "roles" {
path_prefix = "/aws-reserved/sso.amazonaws.com/"
}

output "arns" {
value = [
for parts in [for arn in data.aws_iam_roles.roles.arns : split("/", arn)] :
format("%s/%s", parts[0], element(parts, length(parts) - 1))
]
}
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tomelliff thanks for pointing out this type of usage! hope this helps :)

Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @jlamande for introducing this new data source 🎉 To help this get into an upcoming release, I've committed the requested changes.

Output of acceptance tests (commercial):

--- PASS: TestAccAWSIAMRolesDataSource_nonExistentPathPrefix (11.37s)
--- PASS: TestAccAWSIAMRolesDataSource_nameRegex (19.79s)
--- PASS: TestAccAWSIAMRolesDataSource_basic (20.72s)
--- PASS: TestAccAWSIAMRolesDataSource_pathPrefix (21.56s)
--- PASS: TestAccAWSIAMRolesDataSource_nameRegexAndPathPrefix (31.89s)

@anGie44 anGie44 added this to the v3.55.0 milestone Aug 18, 2021
@anGie44 anGie44 merged commit 8224439 into hashicorp:main Aug 18, 2021
@jlamande
Copy link
Author

Hi @anGie44
sorry I wasnt connected during summer.
Great to see that the PR advanced and has been merged.
Just a question : did you squash the commits ?

@anGie44
Copy link
Contributor

anGie44 commented Aug 19, 2021

No worries @jlamande !
No, commits were all kept-as is (your initial commit, commit that merged main into this branch, my commit with changes)

@github-actions
Copy link

This functionality has been released in v3.55.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. new-data-source Introduces a new data source. provider Pertains to the provider itself, rather than any interaction with AWS. service/iam Issues and PRs that pertain to the iam service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

New Data Source : aws_iam_roles Fetching SSO IAM role ARNs dynamically
5 participants