rds_instance - add snapshot tests, update docs, refactor tests (#1081)

rds_instance - add snapshot tests, update docs, refactor tests

Depends-On: ansible-collections/amazon.aws#776
Depends-On: #1105
SUMMARY

add snapshot tests to test restoring db from snapshot and fix bugs associated
fix some typos in documentation and remove duplicate parameter (added as alias so no breaking change)
remove unused IAM role in tests and add some missing cleanups

ISSUE TYPE

Bugfix Pull Request
Feature Pull Request

COMPONENT NAME
rds_instance
ADDITIONAL INFORMATION
this module had both db_snapshot_identifier and snapshot_identifier as separate params, with the latter being required to restore from snapshot, resulting in some parameter missing errors. moving snapshot_identifier as an alias of db_snapshot_identifier fixes this issue.

Reviewed-by: Alina Buzachis <None>
Reviewed-by: Joseph Torcasso <None>
Reviewed-by: Mark Chappell <None>
Reviewed-by: Sloane Hertel <None>

changelogs/fragments/1081-rds_instance-snapshot-tests-and-restore-db.yml

@@ -0,0 +1,4 @@
+minor_changes:
+  - rds_instance - add snapshot tests to test suite to test restoring db from snapshot (https://github.com/ansible-collections/community.aws/pull/1081).
+bugfixes:
+  - rds_instance - fix bugs associated with restoring db instance from snapshot (https://github.com/ansible-collections/community.aws/pull/1081).

plugins/modules/rds_cluster.py

@@ -758,7 +758,7 @@ def get_rds_method_attribute_name(cluster):
             method_name = 'modify_db_cluster'
             method_options_name = 'get_modify_options'
         elif creation_source == 'snapshot':
-            method_name = 'restore_db_cluster_from_db_snapshot'
+            method_name = 'restore_db_cluster_from_snapshot'
             method_options_name = 'get_restore_snapshot_options'
         elif creation_source == 's3':
             method_name = 'restore_db_cluster_from_s3'

plugins/modules/rds_instance.py

@@ -38,7 +38,7 @@
         type: str
     force_update_password:
         description:
-          - Set to True to update your cluster password with I(master_user_password). Since comparing passwords to determine
+          - Set to C(True) to update your instance password with I(master_user_password). Since comparing passwords to determine
             if it needs to be updated is not possible this is set to False by default to allow idempotence.
         type: bool
         default: False
@@ -52,12 +52,12 @@
         default: True
     read_replica:
         description:
-          - Set to False to promote a read replica cluster or true to create one. When creating a read replica C(creation_source) should
+          - Set to C(False) to promote a read replica instance or true to create one. When creating a read replica C(creation_source) should
             be set to 'instance' or not provided. C(source_db_instance_identifier) must be provided with this option.
         type: bool
     wait:
         description:
-          - Whether to wait for the cluster to be available, stopped, or deleted. At a later time a wait_timeout option may be added.
+          - Whether to wait for the instance to be available, stopped, or deleted. At a later time a I(wait_timeout) option may be added.
             Following each API call to create/modify/delete the instance a waiter is used with a 60 second delay 30 times until the instance reaches
             the expected state (available/stopped/deleted). The total task time may also be influenced by AWSRetry which helps stabilize if the
             instance is in an invalid state to operate on to begin with (such as if you try to stop it when it is in the process of rebooting).
@@ -76,7 +76,7 @@
         type: bool
     apply_immediately:
         description:
-          - A value that specifies whether modifying a cluster with I(new_db_instance_identifier) and I(master_user_password)
+          - A value that specifies whether modifying an instance with I(new_db_instance_identifier) and I(master_user_password)
             should be applied as soon as possible, regardless of the I(preferred_maintenance_window) setting. If false, changes
             are applied during the next maintenance window.
         type: bool
@@ -87,8 +87,8 @@
         type: bool
     availability_zone:
         description:
-          - A list of EC2 Availability Zones that instances in the DB cluster can be created in.
-            May be used when creating a cluster or when restoring from S3 or a snapshot. Mutually exclusive with I(multi_az).
+          - A list of EC2 Availability Zones that the DB instance can be created in.
+            May be used when creating an instance or when restoring from S3 or a snapshot. Mutually exclusive with I(multi_az).
         aliases:
           - az
           - zone
@@ -97,15 +97,15 @@
         description:
           - The number of days for which automated backups are retained.
           - When set to C(0), automated backups will be disabled. (Not applicable if the DB instance is a source to read replicas)
-          - May be used when creating a new cluster, when restoring from S3, or when modifying a cluster.
+          - May be used when creating a new instance, when restoring from S3, or when modifying an instance.
         type: int
     ca_certificate_identifier:
         description:
           - The identifier of the CA certificate for the DB instance.
         type: str
     character_set_name:
         description:
-          - The character set to associate with the DB cluster.
+          - The character set to associate with the DB instance.
         type: str
     copy_tags_to_snapshot:
         description:
@@ -152,8 +152,11 @@
         elements: str
     db_snapshot_identifier:
         description:
-          - The identifier for the DB snapshot to restore from if using I(creation_source=snapshot).
+          - The identifier or ARN of the DB snapshot to restore from when using I(creation_source=snapshot).
         type: str
+        aliases:
+          - snapshot_identifier
+          - snapshot_id
     db_subnet_group_name:
         description:
           - The DB subnet group name to use for the DB instance.
@@ -185,7 +188,7 @@
     enable_iam_database_authentication:
         description:
           - Enable mapping of AWS Identity and Access Management (IAM) accounts to database accounts.
-            If this option is omitted when creating the cluster, Amazon RDS sets this to False.
+            If this option is omitted when creating the instance, Amazon RDS sets this to False.
         type: bool
     enable_performance_insights:
         description:
@@ -256,7 +259,7 @@
         type: str
     master_username:
         description:
-          - The name of the master user for the DB cluster. Must be 1-16 letters or numbers and begin with a letter.
+          - The name of the master user for the DB instance. Must be 1-16 letters or numbers and begin with a letter.
         aliases:
           - username
         type: str
@@ -279,7 +282,7 @@
         type: bool
     new_db_instance_identifier:
         description:
-          - The new DB cluster (lowercase) identifier for the DB cluster when renaming a DB instance. The identifier must contain
+          - The new DB instance (lowercase) identifier for the DB instance when renaming a DB instance. The identifier must contain
             from 1 to 63 letters, numbers, or hyphens and the first character must be a letter and may not end in a hyphen or
             contain consecutive hyphens. Use I(apply_immediately) to rename immediately, otherwise it is updated during the
             next maintenance window.
@@ -369,14 +372,10 @@
         type: str
     skip_final_snapshot:
         description:
-          - Whether a final DB cluster snapshot is created before the DB cluster is deleted. If this is false I(final_db_snapshot_identifier)
+          - Whether a final DB instance snapshot is created before the DB instance is deleted. If this is false I(final_db_snapshot_identifier)
             must be provided.
         type: bool
         default: false
-    snapshot_identifier:
-        description:
-          - The ARN of the DB snapshot to restore from when using I(creation_source=snapshot).
-        type: str
     source_db_instance_identifier:
         description:
           - The identifier or ARN of the source DB instance from which to restore when creating a read replica or spinning up a point-in-time
@@ -410,7 +409,7 @@
         type: str
     tags:
         description:
-          - A dictionary of key value pairs to assign the DB cluster.
+          - A dictionary of key value pairs to assign the DB instance.
         type: dict
     tde_credential_arn:
         description:
@@ -439,7 +438,7 @@
           - restore_from_latest
     vpc_security_group_ids:
         description:
-          - A list of EC2 VPC security groups to associate with the DB cluster.
+          - A list of EC2 VPC security groups to associate with the DB instance.
         type: list
         elements: str
     purge_security_groups:
@@ -528,13 +527,25 @@
   community.aws.rds_instance:
     id: "my-instance-id"
     state: present
-    engine: postgres
-    engine_version: 14.2
-    username: "{{ username }}"
-    password: "{{ password }}"
-    db_instance_class: db.m6g.large
-    allocated_storage: "{{ allocated_storage }}"
     purge_iam_roles: yes
+
+# Restore DB instance from snapshot
+- name: Create a snapshot and wait until completion
+  community.aws.rds_instance_snapshot:
+    instance_id: 'my-instance-id'
+    snapshot_id: 'my-new-snapshot'
+    state: present
+    wait: yes
+  register: snapshot
+
+- name: Restore DB from snapshot
+  community.aws.rds_instance:
+    id: 'my-restored-db'
+    creation_source: snapshot
+    snapshot_identifier: 'my-new-snapshot'
+    engine: mariadb
+    state: present
+  register: restored_db
 '''
 
 RETURN = r'''
@@ -1267,7 +1278,7 @@ def main():
         db_name=dict(),
         db_parameter_group_name=dict(),
         db_security_groups=dict(type='list', elements='str'),
-        db_snapshot_identifier=dict(),
+        db_snapshot_identifier=dict(type='str', aliases=['snapshot_identifier', 'snapshot_id']),
         db_subnet_group_name=dict(aliases=['subnet_group']),
         deletion_protection=dict(type='bool'),
         domain=dict(),
@@ -1304,7 +1315,6 @@ def main():
         s3_ingestion_role_arn=dict(),
         s3_prefix=dict(),
         skip_final_snapshot=dict(type='bool', default=False),
-        snapshot_identifier=dict(),
         source_db_instance_identifier=dict(),
         source_engine=dict(choices=['mysql']),
         source_engine_version=dict(),
@@ -1325,13 +1335,13 @@ def main():
         ('engine', 'aurora-mysql', ('db_cluster_identifier',)),
         ('engine', 'aurora-postresql', ('db_cluster_identifier',)),
         ('storage_type', 'io1', ('iops', 'allocated_storage')),
-        ('creation_source', 'snapshot', ('snapshot_identifier', 'engine')),
+        ('creation_source', 'snapshot', ('db_snapshot_identifier', 'engine')),
         ('creation_source', 's3', (
             's3_bucket_name', 'engine', 'master_username', 'master_user_password',
             'source_engine', 'source_engine_version', 's3_ingestion_role_arn')),
     ]
     mutually_exclusive = [
-        ('s3_bucket_name', 'source_db_instance_identifier', 'snapshot_identifier'),
+        ('s3_bucket_name', 'source_db_instance_identifier', 'db_snapshot_identifier'),
         ('use_latest_restorable_time', 'restore_time'),
         ('availability_zone', 'multi_az'),
     ]

tests/integration/targets/rds_instance/aliases

@@ -1,3 +1,4 @@
 slow
 
 cloud/aws
+rds_instance_info

tests/integration/targets/rds_instance/inventory

@@ -1,26 +1,20 @@
 # inventory names shortened down to fit resource name length limits
 [tests]
-# processor feature tests
 processor
-# restore instance tests
 restore
-# security groups db tests
 sgroups
-# modify complex tests
 complex
-# other tests
 states
 modify
 tagging
 replica
 upgrade
+snapshot
+aurora
 
 # TODO: uncomment after adding iam:CreatePolicy and iam:DeletePolicy
 # iam_roles
 
-# TODO: uncomment after adding rds_cluster module
-# aurora
-
 
 [all:vars]
 ansible_connection=local

tests/integration/targets/rds_instance/main.yml

@@ -6,6 +6,5 @@
 - hosts: all
   gather_facts: no
   strategy: free
-  serial: 9
   roles:
     - rds_instance

tests/integration/targets/rds_instance/roles/rds_instance/defaults/main.yml

@@ -33,3 +33,6 @@ mariadb_engine_version_2: 10.4.21
 # For iam roles tests
 postgres_db_instance_class: db.m6g.large # smallest psql instance
 postgres_db_engine_version: 14.2
+
+# For snapshot tests
+snapshot_id: "{{ instance_id }}-ss"

tests/integration/targets/rds_instance/roles/rds_instance/tasks/test_aurora.yml

@@ -20,6 +20,8 @@
           cluster_id: "{{ cluster_id }}"
           username: "{{ username }}"
           password: "{{ password }}"
+          tags:
+            CreatedBy: rds_instance integration tests
 
       - name: Create an Aurora instance
         rds_instance:
@@ -38,29 +40,24 @@
             - "result.db_instance_identifier == '{{ instance_id }}'"
             - "result.tags | length == 1"
 
-      - name: Modify tags
+      - name: Create an Aurora instance with both username/password and id - invalid
         rds_instance:
-          id: "{{ instance_id }}"
+          id: "{{ instance_id }}-new"
+          cluster_id: "{{ cluster_id }}"
+          engine: aurora
           state: present
+          username: "{{ username }}"
+          password: "{{ password }}"
+          db_instance_class: "{{ aurora_db_instance_class }}"
           tags:
-            Test: rds_instance
-        register: result
-
-      - assert:
-          that:
-            - result.changed
-            - result.tags | length == 1
-            - "result.tags.Test == 'rds_instance'"
-
-      - name: Test idempotence
-        rds_instance:
-          id: "{{ instance_id }}"
-          state: present
+            CreatedBy: rds_instance integration tests
         register: result
+        ignore_errors: yes
 
       - assert:
           that:
-            - not result.changed
+            - result.failed
+            - "'Set master user password for the DB Cluster' in result.msg"
 
       - name: Attempt to modify password (a cluster-managed attribute)
         rds_instance:
@@ -92,20 +89,6 @@
             - "'Modify database endpoint port number for the DB Cluster using the ModifyDbCluster API' in result.msg"
             - "'Please see rds_cluster' in result.msg"
 
-      - name: Modify Aurora instance identifier
-        rds_instance:
-          id: "{{ instance_id }}"
-          state: present
-          purge_tags: False
-          new_id: "{{ modified_instance_id }}"
-          apply_immediately: True
-        register: result
-
-      - assert:
-          that:
-            - result.changed
-            - "result.db_instance_identifier == '{{ modified_instance_id }}'"
-
     always:
 
       - name: Delete the instance

tests/integration/targets/rds_instance/roles/rds_instance/tasks/test_complex.yml

@@ -139,9 +139,6 @@
         port: 1150
         max_allocated_storage: 150
       register: result
-      retries: 30
-      delay: 10
-      until: result is not failed
       check_mode: yes
 
     - assert:
@@ -165,9 +162,6 @@
         port: 1150
         max_allocated_storage: 150
       register: result
-      retries: 30
-      delay: 10
-      until: result is not failed
 
     - assert:
         that:
@@ -185,3 +179,10 @@
         skip_final_snapshot: True
         wait: false
       ignore_errors: yes
+
+    - name: Remove enhanced monitoring role
+      iam_role:
+        assume_role_policy_document: "{{ lookup('file','files/enhanced_monitoring_assume_policy.json') }}"
+        name: "{{ instance_id }}-role"
+        state: absent
+      ignore_errors: yes