sns_topic: Integer parameters within defaultHealthyRetryPolicy are not converted from strings

[gregharvey]

Summary

It's a curious issue that if you need to store variables for a module in a dict then when you pass them over to the module they will always be a string. (See example below.)

There doesn't seem to be any sane way to pass a true integer to the module in this case. Which leaves the only way around this as having the module itself ensure the values passed are cast as integers before they are sent to the AWS API. Given we know the API is fussy and we haven't seen anything like this with other modules, I'm assuming the "usual" behaviour for these AWS modules is to cast integers as there's no way to guarantee Ansible/jinja will send a true integer to the module. This module should do this too.

We've patched this module locally and it now works for us, I'll provide a PR tomorrow with our working changes. PR provided! #716

Issue Type

Bug Report

Component Name

sns_topic

Ansible Version

$ ansible --version
ansible [core 2.11.2] 
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.7.3 (default, Jan 22 2021, 20:04:44) [GCC 8.3.0]
  jinja version = 2.10
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
# /usr/local/lib/python3.7/dist-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0  
ansible.netcommon             2.1.0  
ansible.posix                 1.2.0  
ansible.utils                 2.2.0  
ansible.windows               1.6.0  
arista.eos                    2.1.2  
awx.awx                       19.2.0 
azure.azcollection            1.7.0  
check_point.mgmt              2.0.0  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.0.0  
cisco.asa                     2.0.2  
cisco.intersight              1.0.15 
cisco.ios                     2.2.0  
cisco.iosxr                   2.2.0  
cisco.meraki                  2.4.0  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.3.0  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.0.0  
community.crypto              1.7.0  
community.digitalocean        1.5.1  
community.docker              1.7.0  
community.fortios             1.0.0  
community.general             3.2.0  
community.google              1.0.0  
community.grafana             1.2.1  
community.hashi_vault         1.1.3  
community.hrobot              1.1.1  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.1  
community.mongodb             1.2.1  
community.mysql               2.1.0  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.3.0  
community.proxysql            1.0.0  
community.rabbitmq            1.0.3  
community.routeros            1.1.0  
community.skydive             1.0.0  
community.sops                1.0.6  
community.vmware              1.10.0 
community.windows             1.4.0  
community.zabbix              1.3.0  
containers.podman             1.6.1  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.4.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.9.1  
fortinet.fortimanager         2.0.3  
fortinet.fortios              2.0.2  
frr.frr                       1.0.3  
gluster.gluster               1.0.1  
google.cloud                  1.0.2  
hetzner.hcloud                1.4.3  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.2.0  
junipernetworks.junos         2.2.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.2.0 
netapp.azure                  21.7.0 
netapp.cloudmanager           21.7.0 
netapp.elementsw              21.6.1 
netapp.ontap                  21.7.0 
netapp.um_info                21.6.0 
netapp_eseries.santricity     1.2.12 
netbox.netbox                 3.1.1  
ngine_io.cloudstack           2.1.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.4.0  
openvswitch.openvswitch       2.0.0  
ovirt.ovirt                   1.5.0  
purestorage.flasharray        1.8.0  
purestorage.flashblade        1.6.0  
sensu.sensu_go                1.11.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.17.0 
theforeman.foreman            2.1.0  
vyos.vyos                     2.3.0  
wti.remote                    1.0.1  

AWS SDK versions

$ pip3 show boto boto3 botocore
Name: boto
Version: 2.44.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /usr/lib/python3/dist-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.17.98
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.7/dist-packages
Requires: jmespath, botocore, s3transfer
Required-by: 
---
Name: botocore
Version: 1.21.40
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /usr/local/lib/python3.7/dist-packages
Requires: python-dateutil, jmespath, urllib3
Required-by: s3transfer, boto3, awscli

Configuration

$ ansible-config dump --only-changed

OS / Environment

Debian Buster

Steps to Reproduce

If my playbook looks like this - note particularly the vars, the SNS vars nested neatly into a dict for readability:

---
- hosts: localhost
  tasks:
    - include: sns.yml
      vars:
        aws_sns:
          name: "alarms" # Name of the alarm.
          display_name: "" # Display name for the topic, for when the topic is owned by this AWS account.
          delivery_policy_default_healthy_retry_policy_min_delay_target: 20
          delivery_policy_default_healthy_retry_policy_max_delay_target: 20
          delivery_policy_default_healthy_retry_policy_num_retries: 3
          delivery_policy_default_healthy_retry_policy_backoff_function: "linear"
          delivery_policy_disable_subscription_overrides: false
          subscriptions:
            - endpoint: "[email protected]"
              protocol: "email"

And my role task looks like this in sns.yml:

- name: Create SNS topic.
  community.aws.sns_topic:
    name: "{{ aws_sns.name }}"
    state: present
    display_name: "{{ aws_sns.display_name }}"
    delivery_policy:
      http:
        defaultHealthyRetryPolicy:
          minDelayTarget: "{{ aws_sns.delivery_policy_default_healthy_retry_policy_min_delay_target | int }}" 
          maxDelayTarget: "{{ aws_sns.delivery_policy_default_healthy_retry_policy_max_delay_target | int }}" 
          numRetries: "{{ aws_sns.delivery_policy_default_healthy_retry_policy_num_retries | int }}" 
          backoffFunction: "{{ aws_sns.delivery_policy_default_healthy_retry_policy_backoff_function }}"
        disableSubscriptionOverrides: "{{ aws_sns.delivery_policy_disable_subscription_overrides }}"
    subscriptions:
      "{{ aws_sns.subscriptions }}"

There's nothing wrong with that, we're even casting it as an int with jinja to try and be doubly sure, but it will still fail with this:

"msg": "Couldn't set topic delivery policy: An error occurred (InvalidParameter) when calling the SetTopicAttributes operation: Invalid parameter: DeliveryPolicy: http.defaultHealthyRetryPolicy.minDelayTarget type mismatch, expecting integer, received a string"

Expected Results

I expect the integers passed in the aws_sns dict to be respected as integers and passed as integers to the AWS API.

Actual Results

"msg": "Couldn't set topic delivery policy: An error occurred (InvalidParameter) when calling the SetTopicAttributes operation: Invalid parameter: DeliveryPolicy: http.defaultHealthyRetryPolicy.minDelayTarget type mismatch, expecting integer, received a string"

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[gregharvey]

Forked code can be found here, ahead of a proper PR: codeenigma/ce-provision#288

Specifically, these four lines: https://github.com/codeenigma/ce-provision/pull/288/files#diff-dc75a3db723f90b72bb0286469561ef308b6051ae3825415c45efab1c8c4994cR358

[ansibullbot]

cc @jillr @joelthompson @markuman @nand0p @s-hertel @tremble @wimnat
click here for bot help