community.aws.route53 state:get - return value differs between 2.10.7 and 2.10.6 #523

dseeley · 2021-04-05T09:55:40Z

SUMMARY

With Ansible 2.10.7, the return value (within the set parameter) from the community.aws.route53 module run with state: get does not correspond to expected values defined in documentation. Works as documented in Ansible 2.9.7.

ISSUE TYPE

Bug Report

COMPONENT NAME

community.aws.route53

ANSIBLE VERSION

ansible 2.10.7
  config file = /homedir/modpath/ansible.cfg
  configured module search path = ['/homedir/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 3.8.5 (default, Jan 27 2021, 15:41:15) [GCC 9.3.0]

CONFIGURATION

ANSIBLE_SSH_ARGS(/homedir/modpath/ansible.cfg) = -o 'UserKnownHostsFile=/dev/null' -o 'ControlMaster=auto' -o 'ControlPersist=60s'
ANSIBLE_SSH_CONTROL_PATH_DIR(/homedir/modpath/ansible.cfg) = /tmp/.ansible/cp
ANSIBLE_SSH_RETRIES(/homedir/modpath/ansible.cfg) = 5
DEFAULT_FORCE_HANDLERS(/homedir/modpath/ansible.cfg) = True
DEFAULT_FORKS(/homedir/modpath/ansible.cfg) = 50
DEFAULT_ROLES_PATH(/homedir/modpath/ansible.cfg) = ['/homedir/modpath/roles']
DEFAULT_VAULT_PASSWORD_FILE(/homedir/modpath/ansible.cfg) = /homedir/modpath/.vaultpass-client.py
HOST_KEY_CHECKING(/homedir/modpath/ansible.cfg) = False
INTERPRETER_PYTHON(/homedir/modpath/ansible.cfg) = auto
TRANSFORM_INVALID_GROUP_CHARS(/homedir/modpath/ansible.cfg) = ignore

OS / ENVIRONMENT

Ubuntu 20.04.2 LTS
Python 3.8.5
Ansible installed in new OS with pip

STEPS TO REPRODUCE

    - community.aws.route53:
        state: "get"
        zone: "intensible.com"
        record: "test.intensible.com"
        type: "A"
      register: r__route53_a

    - debug: msg={{r__route53_a}}

EXPECTED RESULTS

    "msg": {
        "changed": false,
        "failed": false,
        "nameservers": [
            "ns-324.awsdns-40.com.",
            "ns-852.awsdns-42.net.",
            "ns-1059.awsdns-04.org.",
            "ns-1680.awsdns-18.co.uk."
        ],
        "set": {
            "alias": false,
            "failover": null,
            "health_check": null,
            "hosted_zone_id": "xxxx",
            "identifier": null,
            "record": "test.intensible.com.",
            "region": null,
            "ttl": "300",
            "type": "A",
            "value": "10.10.10.10",
            "values": [
                "10.10.10.10"
            ],
            "weight": null,
            "zone": "intensible.com."
        }
    }

ACTUAL RESULTS

Loading collection amazon.aws from /usr/local/lib/python3.8/dist-packages/ansible_collections/amazon/aws
Using module file /usr/local/lib/python3.8/dist-packages/ansible_collections/community/aws/plugins/modules/route53.py
Pipelining is enabled.
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: dougal
<127.0.0.1> EXEC /bin/sh -c '/usr/bin/python3 && sleep 0'
ok: [localhost] => {
    "changed": false,
    "invocation": {
        "module_args": {
            "alias": null,
            "alias_evaluate_target_health": false,
            "alias_hosted_zone_id": null,
            "aws_access_key": "xxxx",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "debug_botocore_endpoint_logs": false,
            "ec2_url": null,
            "failover": null,
            "health_check": null,
            "hosted_zone_id": null,
            "identifier": null,
            "overwrite": null,
            "private_zone": false,
            "profile": null,
            "record": "test.intensible.com",
            "region": null,
            "retry_interval": 500,
            "security_token": null,
            "state": "get",
            "ttl": 3600,
            "type": "A",
            "validate_certs": true,
            "value": null,
            "vpc_id": null,
            "wait": false,
            "wait_timeout": 300,
            "weight": null,
            "zone": "intensible.com"
        }
    },
    "nameservers": [
        "ns-324.awsdns-40.com",
        "ns-852.awsdns-42.net",
        "ns-1059.awsdns-04.org",
        "ns-1680.awsdns-18.co.uk"
    ],
    "set": {
        "Name": "test.intensible.com.",
        "ResourceRecords": [
            {
                "Value": "10.10.10.10"
            }
        ],
        "TTL": 300,
        "Type": "A"
    }
}

TASK [debug] ***********************************************************************************************************
task path: /homedir/modpath/test5.yml:42
ok: [localhost] => {
    "msg": {
        "changed": false,
        "failed": false,
        "nameservers": [
            "ns-324.awsdns-40.com",
            "ns-852.awsdns-42.net",
            "ns-1059.awsdns-04.org",
            "ns-1680.awsdns-18.co.uk"
        ],
        "set": {
            "Name": "test.intensible.com.",
            "ResourceRecords": [
                {
                    "Value": "10.10.10.10"
                }
            ],
            "TTL": 300,
            "Type": "A"
        }
    }
}

The text was updated successfully, but these errors were encountered:

ansibullbot · 2021-04-05T10:05:42Z

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

dseeley · 2021-04-05T10:08:48Z

!component =plugins/modules/route53.py

ansibullbot · 2021-04-05T10:16:15Z

Files identified in the description:

[plugins/modules/route53.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/route53.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot · 2021-04-05T10:16:16Z

cc @jillr @jimbydamonk @s-hertel @tremble @wimnat
click here for bot help

tremble · 2021-04-05T11:25:55Z

Apologies for the breakage here. The module was recently updated to use the boto3 library instead of the boto (v2) library. This is likely a side effect of Amazon changing the way they formatted the results between the two libraries.

I would strongly recommend using the route53_info module rather than 'get' on the route53 module. In general 'get', 'list', etc actions for normal (not _info) modules should be considered deprecated. While it's not been formally deprecated with the route53 module, it's just a matter of time.

dseeley · 2021-04-05T13:27:55Z

Thanks for the quick reply.

I would strongly recommend using the route53_info module rather than 'get' on the route53 module

Yep, I considered that too, but it requires an extra call to route53_zone first to get the zone_id. Not a deal-breaker, but slows it down a little (it also doesn't appear to make use of the max_items or type fields, so needs further filtering).

(Also confirmed the old functionality still works in 2.10.6)

felixfontein · 2021-04-05T16:07:43Z

Apologies for the breakage here. The module was recently updated to use the boto3 library instead of the boto (v2) library. This is likely a side effect of Amazon changing the way they formatted the results between the two libraries.

It's not caused by that: the old format was explicitly formatted by the module, see the removed code here: 08d0b9c#diff-39a0039c8f787ef918e13c9e34de717b42890a4baada9448cd80a76242bffee4L468

tamirhad · 2021-04-06T12:22:16Z

max_items and type fields are ignored since 2.10.x release.
working on 2.9.x releases.

tremble · 2021-04-06T12:24:12Z

@tamirhad That looks like it would be a different issue, please raise a separate issue rather than tagging onto this one.

tamirhad · 2021-04-06T17:40:24Z

@tamirhad That looks like it would be a different issue, please raise a separate issue rather than tagging onto this one.

opened new issue about this:
#529

felixfontein · 2021-04-06T18:44:03Z

I'm wondering on that's the best solution for this. Reverting to the old format will break some users which started using the new format, and not reverting is breaking other users who were using the old format.

(This isn't just about state=get, since the new state is also returned in other cases.)

tremble · 2021-04-16T15:08:22Z

#528 should land soon. It's going to return both the 'old' and 'new' formats. We can look at removing the old return values with something like 3.0.0

dseeley · 2021-04-16T16:18:55Z

#528 should land soon. It's going to return both the 'old' and 'new' formats. We can look at removing the old return values with something like 3.0.0

Sounds great - thanks.

felixfontein · 2021-04-16T21:49:47Z

@tremble how about removing state=get in 3.0.0 completely? :)

+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523). + In any case, state=get on the route53 module is not likely to remain supported indefinitely. + Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output.

+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523). + In any case, state=get on the route53 module is not likely to remain supported indefinitely. + Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output. + Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit.

dseeley · 2021-06-14T05:42:55Z

@tremble how about removing state=get in 3.0.0 completely? :)

At the moment using route53_info instead of route53 (state=get) is a problem when Route53 has a large number of records (I believe because it's ignoring max_items, i.e. #529). Tested on a zone with ~800 records - it takes several minutes to return, where state=get returns within a second or two.

+ Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523). + In any case, state=get on the route53 module is not likely to remain supported indefinitely. + Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output. + Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit.

…delete (#92) * Use route53_info to get record sets for delete + Previous method of getting route53 records (using route53 module with state=get), has an issue in Ansible 2.10.7 where the return values have changed due to boto3 migration (ansible-collections/community.aws#523). + In any case, state=get on the route53 module is not likely to remain supported indefinitely. + Using the route53_info module also has some issues with ignoring type and max_items (ansible-collections/community.aws#529), so these are worked around in this fix by post-filtering the output. + Do not call route53_info asynchronously, as it makes too many concurrent requests and blows the AWS Route53 API limit. * Fix for private zone * Fix non-private DNS. Add testsuite support for scaling up/down * Break dependence upon 2.9.6 <= Ansible <= 2.10.6. Route53 deletion enhancement. * Remove '--format json' from 'ansible-galaxy collection list' assertion, as 'format option' is not present in ansible-galaxy 2.10.x

…#523) IGW use module_util functions for tag management SUMMARY Internet Gateway use module_util functions for tag management Cleaning ISSUE TYPE Feature Pull Request COMPONENT NAME ec2_vpc_igw Reviewed-by: Mark Chappell <None> Reviewed-by: None <None>

ansibullbot added bug This issue/PR relates to a bug needs_triage python3 labels Apr 5, 2021

ansibullbot added module module plugins plugin (any type) labels Apr 5, 2021

ansibullbot removed the needs_triage label Apr 5, 2021

dseeley changed the title ~~community.aws.route53 state:get - return value differs between 2.10 and 2.9~~ community.aws.route53 state:get - return value differs between 2.10.7 and 2.10.6 Apr 5, 2021

tremble mentioned this issue Apr 6, 2021

Route53 - Ensure old return values are still returned #528

Merged

ansibullbot added the has_pr label Apr 6, 2021

tamirhad mentioned this issue Apr 6, 2021

community.aws.route53_info query:record_sets - type and max_items ignored on all 2.10 versions #529

Closed

ansible-zuul bot closed this as completed in #528 Apr 16, 2021

dseeley mentioned this issue Apr 17, 2021

Use route53_info to get record sets for delete dseeley/clusterverse#1

Merged

dseeley-sky mentioned this issue Jun 1, 2021

Enable Ansible 4. Optionally use route53_info to get record sets for delete sky-uk/clusterverse#92

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

community.aws.route53 state:get - return value differs between 2.10.7 and 2.10.6 #523

community.aws.route53 state:get - return value differs between 2.10.7 and 2.10.6 #523

dseeley commented Apr 5, 2021

ansibullbot commented Apr 5, 2021

dseeley commented Apr 5, 2021

ansibullbot commented Apr 5, 2021

ansibullbot commented Apr 5, 2021

tremble commented Apr 5, 2021

dseeley commented Apr 5, 2021 •

edited

Loading

felixfontein commented Apr 5, 2021

tamirhad commented Apr 6, 2021 •

edited

Loading

tremble commented Apr 6, 2021

tamirhad commented Apr 6, 2021

felixfontein commented Apr 6, 2021

tremble commented Apr 16, 2021

dseeley commented Apr 16, 2021

felixfontein commented Apr 16, 2021

dseeley commented Jun 14, 2021

community.aws.route53 state:get - return value differs between 2.10.7 and 2.10.6 #523

community.aws.route53 state:get - return value differs between 2.10.7 and 2.10.6 #523

Comments

dseeley commented Apr 5, 2021

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

ansibullbot commented Apr 5, 2021

dseeley commented Apr 5, 2021

ansibullbot commented Apr 5, 2021

ansibullbot commented Apr 5, 2021

tremble commented Apr 5, 2021

dseeley commented Apr 5, 2021 • edited Loading

felixfontein commented Apr 5, 2021

tamirhad commented Apr 6, 2021 • edited Loading

tremble commented Apr 6, 2021

tamirhad commented Apr 6, 2021

felixfontein commented Apr 6, 2021

tremble commented Apr 16, 2021

dseeley commented Apr 16, 2021

felixfontein commented Apr 16, 2021

dseeley commented Jun 14, 2021

dseeley commented Apr 5, 2021 •

edited

Loading

tamirhad commented Apr 6, 2021 •

edited

Loading