Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix KeyError: SecurityGroups in elasticache module. #410

Merged
merged 16 commits into from
Apr 20, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions changelogs/fragments/410-elasticache-fixes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
bugfixes:
- elasticache - Fix ``KeyError`` issue when updating security group (https://github.com/ansible-collections/community.aws/pull/410).
minor_changes:
- elasticache - Improve docs a little, add intgration tests (https://github.com/ansible-collections/community.aws/pull/410).
5 changes: 3 additions & 2 deletions plugins/modules/elasticache.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@
elements: str
cache_security_groups:
description:
- A list of cache security group names to associate with this cache cluster. Must be an empty list if inside a VPC.
- A list of cache security group names to associate with this cache cluster.
- Don't use if your Cache is inside a VPC. In that case use I(security_group_ids) instead!
type: list
elements: str
zone:
Expand Down Expand Up @@ -393,7 +394,7 @@ def _requires_modification(self):
# check vpc security groups
if self.security_group_ids:
vpc_security_groups = []
security_groups = self.data['SecurityGroups'] or []
security_groups = self.data.get('SecurityGroups', [])
for sg in security_groups:
vpc_security_groups.append(sg['SecurityGroupId'])
if set(vpc_security_groups) != set(self.security_group_ids):
Expand Down
8 changes: 8 additions & 0 deletions tests/integration/targets/elasticache/aliases
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Sometimes hit AWS capacity issues - InsufficientCacheClusterCapacity
# https://docs.aws.amazon.com/AmazonElastiCache/latest/red-ug/ErrorMessages.html#ErrorMessages.INSUFFICIENT_CACHE_CLUSTER_CAPACITY
unstable

cloud/aws
shippable/aws/group1

elasticache_subnet_group
10 changes: 10 additions & 0 deletions tests/integration/targets/elasticache/defaults/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---

vpc_name: "{{ resource_prefix }}-elasticache-test-vpc"
vpc_seed: '{{ resource_prefix }}'
vpc_cidr_prefix: '10.{{ 256 | random(seed=vpc_seed) }}'

elasticache_redis_sg_name: "{{ resource_prefix }}-elasticache-test-redis-sg"
elasticache_redis_test_name: "{{ resource_prefix }}-redis-test"
elasticache_subnet_group_name: "{{ resource_prefix }}-elasticache-test-vpc-subnet-group"
elasticache_redis_port: 6379
134 changes: 134 additions & 0 deletions tests/integration/targets/elasticache/tasks/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
---

- name: Integration testing for the elasticache module
module_defaults:
group/aws:
aws_access_key: '{{ aws_access_key }}'
aws_secret_key: '{{ aws_secret_key }}'
security_token: '{{ security_token | default(omit) }}'
region: '{{ aws_region }}'
stefanhorning marked this conversation as resolved.
Show resolved Hide resolved
collections:
- amazon.aws
block:
stefanhorning marked this conversation as resolved.
Show resolved Hide resolved
# == Dependency setup ==

- name: Create VPC to launch Elasticache instances into
ec2_vpc_net:
name: "{{ vpc_name }}"
cidr_block: "{{ vpc_cidr_prefix }}.0.0/16"
state: present
register: elasticache_vpc

- name: Create subnet 1 in this VPC to launch Elasticache instances into
ec2_vpc_subnet:
vpc_id: "{{ elasticache_vpc.vpc.id }}"
cidr: "{{ vpc_cidr_prefix }}.1.0/24"
state: present
register: elasticache_vpc_subnet_1

- name: Create subnet 2 in this VPC to launch Elasticache instances into
ec2_vpc_subnet:
vpc_id: "{{ elasticache_vpc.vpc.id }}"
cidr: "{{ vpc_cidr_prefix }}.2.0/24"
state: present
register: elasticache_vpc_subnet_2

- name: Create Elasticache Subnet Group (grouping two subnets together)
elasticache_subnet_group:
name: "{{ elasticache_subnet_group_name }}"
description: Subnet group grouping together both VPC subnets for Elasticache Test setup
subnets:
- "{{ elasticache_vpc_subnet_1.subnet.id }}"
- "{{ elasticache_vpc_subnet_2.subnet.id }}"
state: present

# == Actual testing of the elasticache module ==

- name: Create Redis Server on Elasticache in VPC subnets
elasticache:
stefanhorning marked this conversation as resolved.
Show resolved Hide resolved
name: "{{ elasticache_redis_test_name }}"
engine: redis
node_type: cache.t3.micro
cache_port: "{{ elasticache_redis_port }}"
cache_subnet_group: "{{ elasticache_subnet_group_name }}"
num_nodes: 1
state: present
register: elasticache_redis

- name: Assert that task worked
assert:
that:
- elasticache_redis is changed
- elasticache_redis.elasticache.data is defined
- elasticache_redis.elasticache.name == "{{ elasticache_redis_test_name }}"
- elasticache_redis.elasticache.data.CacheSubnetGroupName == "{{ elasticache_subnet_group_name }}"

- name: Add security group for Redis access in Elasticache
ec2_group:
name: "{{ elasticache_redis_sg_name }}"
description: Allow access to Elasticache Redis for testing EC module
vpc_id: "{{ elasticache_vpc.vpc.id }}"
rules:
- proto: tcp
from_port: "{{ elasticache_redis_port }}"
to_port: "{{ elasticache_redis_port }}"
cidr_ip: 10.31.0.0/16
register: elasticache_redis_sg

- name: Update Redis Elasticache config with security group (to if changes to existing setup work)
elasticache:
name: "{{ elasticache_redis.elasticache.name }}"
engine: redis
node_type: cache.t3.micro
num_nodes: 1
cache_port: "{{ elasticache_redis_port }}"
cache_subnet_group: "{{ elasticache_subnet_group_name }}"
security_group_ids: "{{ elasticache_redis_sg.group_id }}"
state: present
register: elasticache_redis_new

- name: Assert that task worked
assert:
that:
- elasticache_redis_new is changed
- elasticache_redis_new.elasticache.data is defined
- elasticache_redis_new.elasticache.data.Engine == "redis"
- elasticache_redis_new.elasticache.data.SecurityGroups.0.SecurityGroupId == elasticache_redis_sg.group_id

always:

# == Cleanup ==

- name: Make sure test Redis is deleted again from Elasticache
elasticache:
name: "{{ elasticache_redis_test_name }}"
engine: redis
state: absent

- name: Make sure Elasticache Subnet group is deleted again
elasticache_subnet_group:
name: "{{ elasticache_subnet_group_name }}"
state: absent

- name: Make sure Redis Security Group is deleted again
ec2_group:
name: "{{ elasticache_redis_sg_name }}"
state: absent

- name: Make sure VPC subnet 1 is deleted again
ec2_vpc_subnet:
vpc_id: "{{ elasticache_vpc.vpc.id }}"
cidr: "{{ vpc_cidr_prefix }}.1.0/24"
state: absent

- name: Make sure VPC subnet 2 is deleted again
ec2_vpc_subnet:
vpc_id: "{{ elasticache_vpc.vpc.id }}"
cidr: "{{ vpc_cidr_prefix }}.2.0/24"
state: absent

- name: Make sure VPC is deleted again (only works if subnets were deleted)
ec2_vpc_net:
name: "{{ vpc_name }}"
cidr_block: "{{ vpc_cidr_prefix }}.0.0/16"
state: absent
stefanhorning marked this conversation as resolved.
Show resolved Hide resolved