A role to create S3 bucket with Dynamodb table for Terraform Backend

.github/workflows/integration.yml

@@ -88,14 +88,18 @@ jobs:
         with:
           python-version: ${{ env.python_version }}
 
-      # install ansible
+      # Install ansible
       - name: Install ansible-core (${{ env.ansible_version }})
         run: >-
           python3 -m pip install
           https://github.com/ansible/ansible/archive/${{ env.ansible_version }}.tar.gz
           --disable-pip-version-check
         shell: bash
 
+      # Install Terraform binary
+      - name: Instal Terraform binary
+        uses: hashicorp/setup-terraform@v3
+
       - name: Pre install collections dependencies first so the collection install does not
         run: ansible-galaxy collection install --pre '-r${{ env.source }}/tests/integration/requirements.yml' -p /home/runner/collections/
 

README.md

@@ -1,7 +1,102 @@
 # cloud.terraform_ops roles/playbooks to create/configure Cloud providers' resources for AAP Terraform integration
 
+This repository hosts the `cloud.terraform_ops` Ansible Collection.
+
+The collection includes a variety of Ansible roles and playbooks, to help automate the management of cloud providers' resources for AAP Terraform integration.
+
+<!--start requires_ansible-->
+## Ansible version compatibility
+
+This collection has been tested against following Ansible versions: **>=2.14.0**.
+
+## Included content
+
+Click on the name of a role or playbook to view that content's documentation:
+
+<!--start collection content-->
+### Roles
+Name | Description
+--- | ---
+[cloud.terraform_ops.aws_s3backend](https://github.com/ansible-collections/cloud.terraform_ops/blob/main/roles/aws_s3backend/README.md)|A role to create the necessary AWS infrastructure for an S3 remote backend for Terraform.
+
+
+## Installation and Usage
+
+### Requirements
+
+The [amazon.aws](https://github.com/ansible-collections/amazon.aws) and [community.aws](https://github.com/ansible-collections/amazon.aws) collections MUST be installed in order for this collection to work.
+
+
+### Installation
+
+To consume this Validated Content from Automation Hub, please ensure that you add the following lines to your ansible.cfg file.
+
+```
+[galaxy]
+server_list = automation_hub
+
+[galaxy_server.automation_hub]
+url=https://cloud.redhat.com/api/automation-hub/
+auth_url=https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token
+token=<SuperSecretToken>
+```
+The token can be obtained from the [Automation Hub Web UI](https://console.redhat.com/ansible/automation-hub/token).
+
+Once the above steps are done, you can run the following command to install the collection.
+
+```
+ansible-galaxy collection install cloud.terraform_ops
+```
+
+### Using this collection
+
+Once installed, you can reference the cloud.terraform_ops collection content by its fully qualified collection name (FQCN), for example:
+
+```yaml
+  - hosts: all
+
+    roles:
+      - role: cloud.terraform_ops.aws_s3backend
+        aws_s3backend_bucket_name: sample_bucket_for_tf_backend
+```
+
+### See Also
+
+* [Ansible Using collections](https://docs.ansible.com/ansible/latest/user_guide/collections_using.html) for more details.
+
+## Contributing to this collection
+
+We welcome community contributions to this collection. If you find problems, please open an issue or create a PR against this collection repository.
+
+### Testing and Development
+
+The project uses `ansible-lint` and `black`.
+Assuming this repository is checked out in the proper structure,
+e.g. `collections_root/ansible_collections/cloud/terraform_ops/`, run:
+
+```shell
+  tox -e linters -vv
+```
+
+Sanity and unit tests are run as normal:
+
+```shell
+  ansible-test sanity
+```
+
+If you want to run AWS cloud integration tests, ensure you log in :
+
+```shell
+# using the "default" profile on AWS
+  aws configure set aws_access_key_id     my-access-key
+  aws configure set aws_secret_access_key my-secret-key
+  aws configure set region                eu-north-1
+
+  ansible-test integration [target]
+```
+
 ## License
 
 GNU General Public License v3.0 or later
 
-See [LICENSE](https://github.com/ansible-collections/cloud.aws_ops/blob/main/LICENSE) to see the full text.
+See [LICENSE](https://github.com/ansible-collections/cloud.terraform_ops/blob/main/LICENSE) to see the full text.

galaxy.yml

@@ -21,7 +21,7 @@ tags:
 dependencies:
   amazon.aws: '>=5.1.0'
   community.aws: '>=5.0.0'
-version: 0.1.0-dev0
+version: 0.1.0
 build_ignore:
   - .DS_Store
   - '*.tar.gz'

roles/aws_s3backend/README.md

@@ -0,0 +1,42 @@
+# aws_s3backend
+
+A role to ensure that the necessary AWS infrastructure is present/absent for an S3 remote backend for Terraform.
+When creating, the S3 bucket will be created with the required permissions for Terraform. See U(https://developer.hashicorp.com/terraform/language/settings/backends/s3#s3-bucket-permissions).
+The role also allow for optionally creating a DynamoDB table with the required permissions for state locking and with a partition key named LockID with type of String.
+The role is able to either accept an existing IAM role to be granted the above permissions or create a new one.
+
+## Requirements
+
+AWS User Account with permission to create S3 bucket, DynamoDB table and IAM policy.
+
+## Role Variables
+
+- **aws_s3backend_operation**: Whether to create or delete the Backend resources (S3 bucket and DynamoDB table). Choices: 'create', 'delete'. Default: 'create'.
+- **aws_s3backend_bucket_name**: The name of the S3 bucket to create/delete. **Required**
+- **aws_s3backend_dynamodb_table_name**: The name of the DynamoDB table to create/delete for state locking. The table will be created with a partition key named LockID with type of String.
+- **aws_s3backend_iam_type**: The type of IAM resource to grant access to. Choices: 'user', 'group', 'role'.
+- **aws_s3backend_iam_name**: The name of the IAM resource (user, group or role)user name, group name or role name of IAM resource you wish to grant access to S3 and DynamoDB. Note that, if the specified resource does not exist, it will be created. Required when I(aws_s3backend_iam_type) is provided.
+- **aws_s3backend_terraform_state_path**: Object path granted to the specified user/role/group.
+- **aws_s3backend_delete_iam_resource**: On deletion, specifies whether the IAM resource (user, role or group) should be deleted along with the other resources (S3 bucket and DynamoDB table).
+
+## Example Playbook
+
+    - hosts: localhost
+      roles:
+        - role: cloud.terraform_ops.aws_s3backend
+          aws_s3backend_operation: create
+          aws_s3backend_bucket_name: test_terraform_s3
+          aws_s3backend_dynamodb_table_name: db_state_lock
+          aws_s3backend_iam_type: user
+          aws_s3backend_iam_name: ansible
+          aws_s3backend_terraform_state_path: /test/terraform.tfstate
+
+## License
+
+GNU General Public License v3.0 or later
+
+See [LICENCE](https://github.com/ansible-collections/cloud.terraform_ops/blob/main/LICENSE) to see the full text.
+
+## Author Information
+
+- Ansible Cloud Content Team

roles/aws_s3backend/defaults/main.yml

@@ -0,0 +1,4 @@
+---
+aws_s3backend_operation: "create"
+aws_s3backend_policy_name: "terraform-policy-for-{{ aws_s3backend_bucket_name }}"
+aws_s3backend_delete_iam_resource: false

roles/aws_s3backend/meta/argument_specs.yml

@@ -0,0 +1,47 @@
+---
+argument_specs:
+  main:
+    version_added: 1.0.0
+    short_description: A role to create/delete the AWS infrastructure for an S3 remote backend for Terraform.
+    description:
+      - A role to ensure that the necessary AWS infrastructure is present/absent for an S3 remote backend for Terraform.
+      - When creating, the S3 bucket will be created with the required permissions for Terraform. See U(https://developer.hashicorp.com/terraform/language/settings/backends/s3#s3-bucket-permissions).
+      - The role also allow for optionally creating a DynamoDB table with the required permissions for state locking and with a partition key named LockID with type of String.
+    options:
+      aws_s3backend_operation:
+        description:
+          - Whether to create or delete the Backend resources (S3 bucket and DynamoDB table).
+        choices:
+          - create
+          - delete
+        default: create
+      aws_s3backend_bucket_name:
+        description:
+          - The name of the S3 bucket to create/delete.
+        required: true
+      aws_s3backend_dynamodb_table_name:
+        description:
+          - The name of the DynamoDB table to create/delete for state locking.
+          - The table will be created with a partition key named LockID with type of String.
+        type: str
+      aws_s3backend_delete_iam_resource:
+        description:
+          - Whether to delete or not the IAM resource along with the S3 and DynamoDB table while I(aws_s3backend_operation=delete).
+          - Ignored when I(aws_s3backend_operation=create).
+        type: bool
+        default: false
+      aws_s3backend_iam_type:
+        description:
+          - The type of IAM resource to grant access to.
+        choices: [ "user", "group", "role"]
+        type: str
+      aws_s3backend_iam_name:
+        description:
+          - The user name, group name or role name of IAM resource you wish to grant access to S3 and DynamoDB.
+          - The specified user/role/group name will be created if it does not exist.
+          - Required When I(aws_s3backend_iam_type) is provided.
+        type: str
+      aws_s3backend_terraform_state_path:
+        description:
+          - Object path granted to the specified user/role/group.
+          - When not specified, user/role/group will be allowed to create state file anywhere into the bucket.

roles/aws_s3backend/tasks/create.yml

@@ -0,0 +1,27 @@
+---
+- name: Create S3 bucket with versioning enabled
+  amazon.aws.s3_bucket:
+    name: "{{ aws_s3backend_bucket_name }}"
+    state: present
+    versioning: true
+
+- name: Create dynamodb table
+  community.aws.dynamodb_table:
+    state: present
+    name: "{{ aws_s3backend_dynamodb_table_name }}"
+    hash_key_name: "LockID"
+    hash_key_type: "STRING"
+    wait: true
+    table_class: 'STANDARD'
+  when: aws_s3backend_dynamodb_table_name is defined
+
+- name: Create/Assign policy to the IAM user/role/group
+  when: aws_s3backend_iam_type is defined
+  block:
+    - name: Create/Assign policy to the IAM role
+      ansible.builtin.include_tasks: role.yml
+      when: aws_s3backend_iam_type == "role"
+
+    - name: Create/Assign policy to the IAM user/group
+      ansible.builtin.include_tasks: user_or_group.yml
+      when: aws_s3backend_iam_type in ("user", "group")

roles/aws_s3backend/tasks/delete.yml

@@ -0,0 +1,45 @@
+---
+- name: Delete S3 bucket
+  amazon.aws.s3_bucket:
+    name: "{{ aws_s3backend_bucket_name }}"
+    state: absent
+    force: true
+
+- name: Delete dynamodb table
+  community.aws.dynamodb_table:
+    state: absent
+    name: "{{ aws_s3backend_dynamodb_table_name }}"
+    wait: true
+  when: aws_s3backend_dynamodb_table_name is defined
+
+- name: Delete the policy attached to the user/role/group
+  amazon.aws.iam_policy:
+    iam_type: "{{ aws_s3backend_iam_type }}"
+    iam_name: "{{ aws_s3backend_iam_name }}"
+    policy_name: "{{ aws_s3backend_policy_name }}"
+    state: absent
+  when: aws_s3backend_iam_type is defined
+
+- name: Delete IAM user
+  amazon.aws.iam_user:
+    name: "{{ aws_s3backend_iam_name }}"
+    state: absent
+  when:
+    - aws_s3backend_delete_iam_resource | bool
+    - aws_s3backend_iam_type | default('') == "user"
+
+- name: Delete IAM role
+  amazon.aws.iam_role:
+    name: "{{ aws_s3backend_iam_name }}"
+    state: absent
+  when:
+    - aws_s3backend_delete_iam_resource | bool
+    - aws_s3backend_iam_type | default('') == "role"
+
+- name: Delete IAM group
+  amazon.aws.iam_group:
+    name: "{{ aws_s3backend_iam_name }}"
+    state: absent
+  when:
+    - aws_s3backend_delete_iam_resource | bool
+    - aws_s3backend_iam_type | default('') == "group"

roles/aws_s3backend/tasks/main.yml

@@ -0,0 +1,10 @@
+---
+- name: Ensure 'aws_s3backend_iam_name' is provided when 'aws_s3backend_iam_type' is provided
+  ansible.builtin.fail:
+    msg: "'aws_s3backend_iam_name' is required when 'aws_s3backend_iam_type' is provided."
+  when:
+    - aws_s3backend_iam_name is undefined
+    - aws_s3backend_iam_type is defined
+
+- name: Create/Configure/Delete AWS infrastructure for S3 Remote backend
+  ansible.builtin.include_tasks: "{{ aws_s3backend_operation }}.yml"

roles/aws_s3backend/tasks/role.yml

@@ -0,0 +1,19 @@
+---
+- name: Gather IAM Role info
+  amazon.aws.iam_role_info:
+    name: "{{ aws_s3backend_iam_name }}"
+  register: existing_roles
+
+- name: Create Role with policy
+  amazon.aws.iam_role:
+    name: "{{ aws_s3backend_iam_name }}"
+    state: present
+  when: existing_roles.iam_roles | length == 0
+
+- name: Apply policy to Role
+  amazon.aws.iam_policy:
+    iam_type: "role"
+    iam_name: "{{ aws_s3backend_iam_name }}"
+    policy_name: "{{ aws_s3backend_policy_name }}"
+    policy_json: "{{ lookup('template', 'policy.json.j2') }}"
+    state: present

roles/aws_s3backend/tasks/user_or_group.yml

@@ -0,0 +1,29 @@
+---
+- name: Gather IAM User/Group info
+  amazon.aws.iam_user_info:
+    name: "{{ (aws_s3backend_iam_type == 'user') | ternary(aws_s3backend_iam_name, omit) }}"
+    group: "{{ (aws_s3backend_iam_type == 'group') | ternary(aws_s3backend_iam_name, omit) }}"
+  register: users
+
+- name: Create IAM group
+  amazon.aws.iam_group:
+    name: "{{ aws_s3backend_iam_name }}"
+    state: present
+  when:
+    - aws_s3backend_iam_type == "group"
+    - users.iam_users | length == 0
+
+- name: Create IAM user
+  amazon.aws.iam_user:
+    name: "{{ aws_s3backend_iam_name }}"
+    state: present
+  when:
+    - aws_s3backend_iam_type == "user"
+    - users.iam_users | length == 0
+
+- name: Assign policy to the IAM user/group
+  amazon.aws.iam_policy:
+    iam_type: "{{ aws_s3backend_iam_type }}"
+    iam_name: "{{ aws_s3backend_iam_name }}"
+    policy_name: "{{ aws_s3backend_policy_name }}"
+    policy_json: "{{ lookup('template', 'policy.json.j2') }}"

roles/aws_s3backend/templates/policy.json.j2

@@ -0,0 +1,31 @@
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Action": "s3:ListBucket",
+      "Resource": "arn:aws:s3:::{{ aws_s3backend_bucket_name }}"
+    },
+    {
+      "Effect": "Allow",
+      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
+      {% if aws_s3backend_terraform_state_path is defined %}
+      "Resource": "arn:aws:s3:::{{ aws_s3backend_bucket_name }}/{{ aws_s3backend_terraform_state_path }}"
+      {% else %}
+      "Resource": "arn:aws:s3:::{{ aws_s3backend_bucket_name }}/*"
+      {% endif %}
+    },
+    {% if aws_s3backend_dynamodb_table_name is defined %}
+    {
+      "Effect": "Allow",
+      "Action": [
+        "dynamodb:DescribeTable",
+        "dynamodb:GetItem",
+        "dynamodb:PutItem",
+        "dynamodb:DeleteItem"
+      ],
+      "Resource": "arn:aws:dynamodb:*:*:table/{{ aws_s3backend_dynamodb_table_name }}"
+    }
+    {% endif %}
+  ]
+}