AzureRM Terraform Backend

.github/workflows/integration.yml

@@ -101,7 +101,7 @@ jobs:
         uses: hashicorp/setup-terraform@v3
         with:
           terraform_wrapper: false
-          terraform_version: "1.6.3"
+          terraform_version: ">1.7.0"
 
       - name: Pre install collections dependencies first so the collection install does not
         run: ansible-galaxy collection install --pre '-r${{ env.source }}/tests/integration/requirements.yml' -p /home/runner/collections/
@@ -119,6 +119,12 @@ jobs:
           collection_path: ${{ steps.install.outputs.collection_path }}
           ansible_core_ci_key: ${{ secrets.ANSIBLE_CORE_CI_KEY }}
 
+      - name: Create AzureRM session credentials
+        uses: ansible-network/github_actions/.github/actions/ansible_azure_test_provider@main
+        with:
+          collection_path: ${{ steps.install.outputs.collection_path }}
+          ansible_core_ci_key: ${{ secrets.ANSIBLE_CORE_CI_KEY }}
+
       # we use raw git to create a repository in the tests
       # this fails if the committer doesn't have a name and an email set
       - name: Set up git

README.md

@@ -24,6 +24,7 @@ Name | Description
 --- | ---
 [cloud.terraform_ops.aws_s3backend](https://github.com/redhat-cop/cloud.terraform_ops/blob/main/roles/aws_s3backend/README.md)|A role to create the necessary AWS infrastructure for an S3 remote backend for Terraform.
 [cloud.terraform_ops.gcs_backend](https://github.com/redhat-cop/cloud.terraform_ops/blob/main/roles/gcs_backend/README.md)|A role to create the necessary Google Cloud infrastructure for a Google Cloud Storage (GCS) remote backend for Terraform.
+[cloud.terraform_ops.azurerm_backend](https://github.com/redhat-cop/cloud.terraform_ops/tree/main/roles/azurerm_backend/README.md)|A role to create/delete the necessary Azure infrastructure for an Azurerm remote backend for Terraform.
 
 ## Installation and Usage
 
@@ -33,6 +34,8 @@ Name | Description
 
 - The [google.cloud](https://github.com/ansible-collections/google.cloud) collection MUST be installed to use the role [cloud.terraform_ops.gcs_backend](https://github.com/redhat-cop/cloud.terraform_ops/blob/main/roles/gcs_backend/README.md).
 
+- The [azure.azcollection](https://github.com/ansible-collections/azure) collection MUST be installed to use the role [cloud.terraform_ops.azurerm_backend](https://github.com/redhat-cop/cloud.terraform_ops/tree/main/roles/azurerm_backend/README.md).
+
 
 ### Installation
 

roles/azurerm_backend/README.md

@@ -0,0 +1,43 @@
+# azurerm_backend
+
+A role to create/delete the necessary Azure infrastructure for an Azurerm remote backend for Terraform.
+The role ensures that the specified resource group, storage account and container are present/absent. When a service principal is specified, the role will assign the [Storage Blob data contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) to the service principal.
+
+## Requirements
+
+Azure Account with permission to create Resource group, Storage account and container, and eventually assign role to service principal.
+
+## Role Variables
+
+Name | Description | Type | Default | Required
+--- | --- | --- | --- | ---
+azurerm_backend_operation|Whether to create or delete the infrastructure for the azurerm Terraform backend. Choices: 'create', 'delete'|string|'create'|N/A
+azurerm_backend_resource_group_name|The name of the resource group to create/delete|string|N/A|Yes
+azurerm_backend_location|Azure location for the resource group|string|N/A|When creating a new resource group.
+azurerm_backend_storage_account_name|The name of the storage account name to be created. When not specified and if none is existing from the resource group, the role will create a new one with generated name|string|N/A|No
+azurerm_backend_storage_account_type|Type of storage account. Valid values are: 'Premium_LRS', 'Standard_GRS', 'Standard_LRS', 'Standard_RAGRS', 'Standard_ZRS', 'Premium_ZRS', 'Standard_RAGZRS', 'Standard_GZRS'.|string|Standard_LRS|When creating a storage account.
+azurerm_backend_container_name|The Name of the Storage Container to create within the Storage Account.|string|N/A|When __azurerm_backend_operation=create__.
+azurerm_backend_service_principal_id|The service principal Id used by Terraform to push Terraform state to the container. The role will assign the [Storage Blob data contributor](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) role for the specified container to this service principal.|string|N/A|No
+
+## Example Playbook
+
+    - hosts: localhost
+      roles:
+        - role: cloud.terraform_ops.azurerm_backend
+          azurerm_backend_operation: create
+          azurerm_backend_resource_group_name: "StorageAccount-ResourceGroup"
+          azurerm_backend_location: eastus
+          azurerm_backend_storage_account_name: ansible123
+          azurerm_backend_storage_account_type: Premium_LRS
+          azurerm_backend_container_name: tfstate
+          azurerm_backend_service_principal_id: abcdef12-123a-456b-789c-12345abcde6e
+
+## License
+
+GNU General Public License v3.0 or later
+
+See [LICENCE](https://github.com/redhat-cop/cloud.terraform_ops/blob/main/LICENSE) to see the full text.
+
+## Author Information
+
+- Ansible Cloud Content Team

roles/azurerm_backend/defaults/main.yml

@@ -0,0 +1,3 @@
+---
+azurerm_backend_operation: create
+azurerm_backend_storage_account_type: Standard_LRS

roles/azurerm_backend/meta/argument_specs.yml

@@ -0,0 +1,53 @@
+---
+argument_specs:
+  main:
+    version_added: 1.0.0
+    short_description: A role to create/delete the necessary Azure infrastructure for an Azurerm remote backend for Terraform.
+    description:
+      - A role to ensure that the necessary Azure infrastructure is present/absent for an Azurerm remote backend for Terraform.
+      - When O(azurerm_backend_operation=create), the role ensures that a resource group, a storage account and a container are present and optionally
+        assigns the 'Storage Blob Data Contributor' role for the specified container to the service principal provided by the user.
+      - When O(azurerm_backend_operation=delete), the role will force deletion of the resource group.
+    options:
+      azurerm_backend_operation:
+        description:
+          - Whether to create or delete the infrastructure for the azurerm Terraform backend.
+          - When set to V(delete), the resource group will be deleted with all attached resources.
+        choices:
+          - create
+          - delete
+        default: create
+      azurerm_backend_resource_group_name:
+        description:
+          - The name of the resource group to create/delete.
+        required: true
+      azurerm_backend_location:
+        description:
+          - Azure location for the resource group. Required when creating a new resource group.
+      azurerm_backend_storage_account_name:
+        description:
+          - The name of the storage account name to be created.
+          - When not specified and if none is existing from the resource group, the role will create
+            a new one with generated name.
+      azurerm_backend_storage_account_type:
+        description:
+          - Type of storage account. Required when creating a storage account.
+        type: str
+        choices:
+          - Premium_LRS
+          - Standard_GRS
+          - Standard_LRS
+          - Standard_RAGRS
+          - Standard_ZRS
+          - Premium_ZRS
+          - Standard_RAGZRS
+          - Standard_GZRS
+        default: Standard_LRS
+      azurerm_backend_container_name:
+        description:
+          - The Name of the Storage Container to create within the Storage Account.
+          - Required when O(azurerm_backend_operation=create).
+      azurerm_backend_service_principal_id:
+        description:
+          - The service principal Id used by Terraform to push Terraform state to the container.
+          - The role will assign the Storage Blob Data Contributor U(https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) role for the specified container to this service principal.

roles/azurerm_backend/tasks/create.yml

@@ -0,0 +1,87 @@
+---
+- name: Ensure the container name has been defined
+  ansible.builtin.fail:
+    msg: "The container name must be defined using 'azurerm_backend_container_name'"
+  when: azurerm_backend_container_name is undefined
+
+- name: Check if resource group exists
+  azure.azcollection.azure_rm_resourcegroup_info:
+    name: "{{ azurerm_backend_resource_group_name }}"
+  register: rg_info
+
+- name: Set variable for resource group id
+  ansible.builtin.set_fact:
+    azurerm_backend_resource_group_id: "{{ rg_info.resourcegroups.0.id }}"
+  when: rg_info.resourcegroups | length > 0
+
+- name: Create resource group
+  when: rg_info.resourcegroups | length == 0
+  block:
+    - name: Ensure the resource group location is defined
+      ansible.builtin.fail:
+        msg: "The resource group location should be defined using 'azurerm_backend_location'"
+      when: azurerm_backend_location is undefined
+
+    - name: Create resource group
+      azure.azcollection.azure_rm_resourcegroup:
+        name: "{{ azurerm_backend_resource_group_name }}"
+        location: "{{ azurerm_backend_location }}"
+      register: create_resource_group
+
+    - name: Set variable for resource group id
+      ansible.builtin.set_fact:
+        azurerm_backend_resource_group_id: "{{ create_resource_group.state.id }}"
+
+- name: List existing storage accounts
+  azure.azcollection.azure_rm_storageaccount_info:
+    resource_group: "{{ azurerm_backend_resource_group_name }}"
+    name: "{{ azurerm_backend_storage_account_name | default(omit) }}"
+  register: storage_accounts
+
+- name: Set Storage account name variable if storage account present
+  ansible.builtin.set_fact:
+    azurerm_backend_storage_account_name: "{{ storage_accounts.storageaccounts.0.name }}"
+  when:
+    - azurerm_backend_storage_account_name is undefined
+    - storage_accounts.storageaccounts | length > 0
+
+- name: Generate name for the storage account name
+  when:
+    - azurerm_backend_storage_account_name is undefined
+    - storage_accounts.storageaccounts | length == 0
+  block:
+    - name: Create Storage account
+      azure.azcollection.azure_rm_storageaccount:
+        resource_group: "{{ azurerm_backend_resource_group_name }}"
+        name: "{{ lookup('ansible.builtin.password', '/dev/null', chars=['ascii_lowercase', 'digits'], length=12) }}"
+        type: "{{ azurerm_backend_storage_account_type }}"
+      register: create_storage
+      retries: 300
+      delay: 1
+      until: create_storage is successful
+
+    - name: Set Storage account name variable
+      ansible.builtin.set_fact:
+        azurerm_backend_storage_account_name: "{{ create_storage.state.name }}"
+
+- name: Create Storage account
+  azure.azcollection.azure_rm_storageaccount:
+    resource_group: "{{ azurerm_backend_resource_group_name }}"
+    name: "{{ azurerm_backend_storage_account_name }}"
+    type: "{{ azurerm_backend_storage_account_type }}"
+  when:
+    - azurerm_backend_storage_account_name is defined
+    - storage_accounts.storageaccounts | length == 0
+
+- name: Create Container
+  azure.azcollection.azure_rm_storageblob:
+    resource_group: "{{ azurerm_backend_resource_group_name }}"
+    storage_account_name: "{{ azurerm_backend_storage_account_name }}"
+    container: "{{ azurerm_backend_container_name }}"
+
+- name: Assign the Storage Blob Data Contributor (ba92f5b4-2d11-453d-a403-e96b0029c9fe) role for the specified container to the Service principal
+  azure.azcollection.azure_rm_roleassignment:
+    assignee_object_id: "{{ azurerm_backend_service_principal_id }}"
+    scope: "{{ azurerm_backend_resource_group_id }}/providers/Microsoft.Storage/storageAccounts/{{ azurerm_backend_storage_account_name }}/blobServices/default/containers/{{ azurerm_backend_container_name }}"
+    role_definition_id: "/subscriptions/{{ azurerm_backend_resource_group_id.split('/')[2] }}/providers/Microsoft.Authorization/roleDefinitions/ba92f5b4-2d11-453d-a403-e96b0029c9fe"
+  when: azurerm_backend_service_principal_id is defined

roles/azurerm_backend/tasks/delete.yml

@@ -0,0 +1,6 @@
+---
+- name: Delete resource group
+  azure.azcollection.azure_rm_resourcegroup:
+    name: "{{ azurerm_backend_resource_group_name }}"
+    state: absent
+    force_delete_nonempty: true

roles/azurerm_backend/tasks/main.yml

@@ -0,0 +1,3 @@
+---
+- name: "Create/Delete Azure infrastructure for Terraform backend."
+  ansible.builtin.include_tasks: "{{ azurerm_backend_operation }}.yml"

test-requirements.txt

@@ -1,6 +1,61 @@
-botocore>=1.25.0
-boto3>=1.22.0
-
 # Required for gcs_backend tests
 google-auth
-google-cloud-storage
+google-cloud-storage
+
+# Required for aws tests
+botocore>=1.29.0
+boto3>=1.26.0
+
+# Requires Azure tests
+packaging
+requests[security]
+xmltodict
+msgraph-sdk==1.0.0
+azure-cli-core==2.34.0
+azure-common==1.1.11
+azure-identity==1.14.0
+azure-mgmt-authorization==2.0.0
+azure-mgmt-apimanagement==3.0.0
+azure-mgmt-batch==16.2.0
+azure-mgmt-cdn==11.0.0
+azure-mgmt-compute==26.1.0
+azure-mgmt-containerinstance==9.0.0
+azure-mgmt-core==1.3.0
+azure-mgmt-containerregistry==9.1.0
+azure-containerregistry==1.1.0
+azure-mgmt-containerservice==20.0.0
+azure-mgmt-datalake-store==1.0.0
+azure-mgmt-datafactory==2.0.0
+azure-mgmt-dns==8.0.0
+azure-mgmt-marketplaceordering==1.1.0
+azure-mgmt-monitor==3.0.0
+azure-mgmt-managedservices==6.0.0
+azure-mgmt-managementgroups==1.0.0
+azure-mgmt-network==19.1.0
+azure-mgmt-nspkg==2.0.0
+azure-mgmt-privatedns==1.0.0
+azure-mgmt-redis==13.0.0
+azure-mgmt-resource==21.1.0
+azure-mgmt-rdbms==10.0.0
+azure-mgmt-search==8.0.0
+azure-mgmt-servicebus==7.1.0
+azure-mgmt-sql==3.0.1
+azure-mgmt-storage==19.0.0
+azure-mgmt-trafficmanager==1.0.0b1
+azure-mgmt-web==6.1.0
+azure-nspkg==2.0.0
+azure-storage-blob==12.11.0
+azure-core==1.28.0
+azure-keyvault==4.2.0
+azure-mgmt-keyvault==10.0.0
+azure-mgmt-cosmosdb==6.4.0
+azure-mgmt-hdinsight==9.0.0
+azure-mgmt-devtestlabs==9.0.0
+azure-mgmt-loganalytics==12.0.0
+azure-mgmt-automation==1.0.0
+azure-mgmt-iothub==2.2.0
+azure-iot-hub==2.6.1
+azure-mgmt-recoveryservices==2.0.0
+azure-mgmt-recoveryservicesbackup==3.0.0
+azure-mgmt-notificationhubs==7.0.0
+azure-mgmt-eventhub==10.1.0

tests/integration/constraints.txt

@@ -1,6 +1,2 @@
-# When updating the minimal requirements please also update
-# - tests/unit/constraints.txt
-# - tests/integration/constraints.txt
-# - tests/integration/targets/setup_botocore_pip
 botocore>=1.29.0
 boto3>=1.26.0

tests/integration/requirements.txt

@@ -1,7 +1,61 @@
-# Our code is based on the AWS SDKs
-boto3
-botocore
-
 # Required for gcs_backend tests
 google-auth
-google-cloud-storage
+google-cloud-storage
+
+# Required for aws tests
+botocore>=1.29.0
+boto3>=1.26.0
+
+# Requires Azure tests
+packaging
+requests[security]
+xmltodict
+msgraph-sdk==1.0.0
+azure-cli-core==2.34.0
+azure-common==1.1.11
+azure-identity==1.14.0
+azure-mgmt-authorization==2.0.0
+azure-mgmt-apimanagement==3.0.0
+azure-mgmt-batch==16.2.0
+azure-mgmt-cdn==11.0.0
+azure-mgmt-compute==26.1.0
+azure-mgmt-containerinstance==9.0.0
+azure-mgmt-core==1.3.0
+azure-mgmt-containerregistry==9.1.0
+azure-containerregistry==1.1.0
+azure-mgmt-containerservice==20.0.0
+azure-mgmt-datalake-store==1.0.0
+azure-mgmt-datafactory==2.0.0
+azure-mgmt-dns==8.0.0
+azure-mgmt-marketplaceordering==1.1.0
+azure-mgmt-monitor==3.0.0
+azure-mgmt-managedservices==6.0.0
+azure-mgmt-managementgroups==1.0.0
+azure-mgmt-network==19.1.0
+azure-mgmt-nspkg==2.0.0
+azure-mgmt-privatedns==1.0.0
+azure-mgmt-redis==13.0.0
+azure-mgmt-resource==21.1.0
+azure-mgmt-rdbms==10.0.0
+azure-mgmt-search==8.0.0
+azure-mgmt-servicebus==7.1.0
+azure-mgmt-sql==3.0.1
+azure-mgmt-storage==19.0.0
+azure-mgmt-trafficmanager==1.0.0b1
+azure-mgmt-web==6.1.0
+azure-nspkg==2.0.0
+azure-storage-blob==12.11.0
+azure-core==1.28.0
+azure-keyvault==4.2.0
+azure-mgmt-keyvault==10.0.0
+azure-mgmt-cosmosdb==6.4.0
+azure-mgmt-hdinsight==9.0.0
+azure-mgmt-devtestlabs==9.0.0
+azure-mgmt-loganalytics==12.0.0
+azure-mgmt-automation==1.0.0
+azure-mgmt-iothub==2.2.0
+azure-iot-hub==2.6.1
+azure-mgmt-recoveryservices==2.0.0
+azure-mgmt-recoveryservicesbackup==3.0.0
+azure-mgmt-notificationhubs==7.0.0
+azure-mgmt-eventhub==10.1.0

tests/integration/requirements.yml

@@ -12,3 +12,6 @@ collections:
   - name: https://github.com/ansible-collections/google.cloud.git
     type: git
     version: master
+  - name: https://github.com/ansible-collections/azure.git
+    type: git
+    version: dev

tests/integration/targets/test_azurerm_backend/aliases

@@ -0,0 +1,3 @@
+cloud/azure
+role/azurerm_backend
+time=1m

tests/integration/targets/test_azurerm_backend/defaults/main.yml

@@ -0,0 +1,3 @@
+---
+azurerm_backend_container_name: "tfstate"
+azurerm_backend_tfstate_file: "testing.terraform.tfstate"