All aws ansible modules from the collection return the following error - An error occurred (400) when calling the operations Invalid #1458

kuvivek · 2022-09-12T22:44:30Z

Summary

AWS ansible module "aws_acm_info" returns the following error - "Couldn't obtain certificates: An error occurred (400) when calling the ListCertificates operation: \nInvalidHttpRequestThe HTTP request is invalid. Reason: Unable to parse request7978969e-c8b8-4693-a30e-d19face584f0"

Similarly for the elb_classic_lb_info returns the following error:
fatal: [localhost]: FAILED! => {
"boto3_version": "1.24.71",
"botocore_version": "1.27.71",
"changed": false,
"error": {
"code": "InvalidAction",
"message": "The action DescribeLoadBalancers is not valid for this web service."
},

Issue Type

Bug Report

Component Name

aws_acm_info, elb_classic_lb_info

Ansible Version

$ ansible --version

ansible [core 2.13.4]
config file = None
configured module search path = ['/Users/valassis/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.9/site-packages/ansible
ansible collection location = /Users/valassis/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible
python version = 3.9.13 (main, Aug 7 2022, 01:33:23) [Clang 13.1.6 (clang-1316.0.21.2.5)]
jinja version = 3.1.2
libyaml = True

Collection Versions

$ ansible-galaxy collection list

MacBook-Pro:~ valassis$ ansible-galaxy collection list

/usr/local/lib/python3.9/site-packages/ansible_collections

Collection Version

amazon.aws 3.4.0
ansible.netcommon 3.1.0
ansible.posix 1.4.0
ansible.utils 2.6.1
ansible.windows 1.11.0
arista.eos 5.0.1
awx.awx 21.4.0
azure.azcollection 1.13.0
check_point.mgmt 2.3.0
chocolatey.chocolatey 1.3.0
cisco.aci 2.2.0
cisco.asa 3.1.0
cisco.dnac 6.5.3
cisco.intersight 1.0.19
cisco.ios 3.3.0
cisco.iosxr 3.3.0
cisco.ise 2.5.0
cisco.meraki 2.10.1
cisco.mso 2.0.0
cisco.nso 1.0.3
cisco.nxos 3.1.0
cisco.ucs 1.8.0
cloud.common 2.1.2
cloudscale_ch.cloud 2.2.2
community.aws 3.5.0
community.azure 1.1.0
community.ciscosmb 1.0.5
community.crypto 2.5.0
community.digitalocean 1.21.0
community.dns 2.3.1
community.docker 2.7.1
community.fortios 1.0.0
community.general 5.5.0
community.google 1.0.0
community.grafana 1.5.2
community.hashi_vault 3.2.0
community.hrobot 1.5.2
community.libvirt 1.2.0
community.mongodb 1.4.2
community.mysql 3.4.0
community.network 4.0.1
community.okd 2.2.0
community.postgresql 2.2.0
community.proxysql 1.4.0
community.rabbitmq 1.2.2
community.routeros 2.2.1
community.sap 1.0.0
community.sap_libs 1.2.0
community.skydive 1.0.0
community.sops 1.3.0
community.vmware 2.8.0
community.windows 1.11.0
community.zabbix 1.8.0
containers.podman 1.9.4
cyberark.conjur 1.1.0
cyberark.pas 1.0.14
dellemc.enterprise_sonic 1.1.1
dellemc.openmanage 5.5.0
dellemc.os10 1.1.1
dellemc.os6 1.0.7
dellemc.os9 1.0.4
f5networks.f5_modules 1.19.0
fortinet.fortimanager 2.1.5
fortinet.fortios 2.1.7
frr.frr 2.0.0
gluster.gluster 1.0.2
google.cloud 1.0.2
hetzner.hcloud 1.8.1
hpe.nimble 1.1.4
ibm.qradar 2.0.0
ibm.spectrum_virtualize 1.9.0
infinidat.infinibox 1.3.3
infoblox.nios_modules 1.3.0
inspur.sm 2.0.0
junipernetworks.junos 3.1.0
kubernetes.core 2.3.2
mellanox.onyx 1.0.0
netapp.aws 21.7.0
netapp.azure 21.10.0
netapp.cloudmanager 21.19.0
netapp.elementsw 21.7.0
netapp.ontap 21.22.0
netapp.storagegrid 21.10.0
netapp.um_info 21.8.0
netapp_eseries.santricity 1.3.1
netbox.netbox 3.7.1
ngine_io.cloudstack 2.2.4
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.2
openstack.cloud 1.8.0
openvswitch.openvswitch 2.1.0
ovirt.ovirt 2.2.3
purestorage.flasharray 1.13.0
purestorage.flashblade 1.9.0
purestorage.fusion 1.0.2
sensu.sensu_go 1.13.1
servicenow.servicenow 1.0.6
splunk.es 2.0.0
t_systems_mms.icinga_director 1.31.0
theforeman.foreman 3.4.0
vmware.vmware_rest 2.2.0
vyos.vyos 3.0.1
wti.remote 1.0.4

/Users/valassis/.ansible/collections/ansible_collections

Collection Version

amazon.aws 3.0.0
community.aws 2.1.0
MacBook-Pro:~ valassis$

AWS SDK versions

$ pip show boto boto3 botocore

MacBook-Pro:~ valassis$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.24.71
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:

Name: botocore
Version: 1.27.71
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /usr/local/lib/python3.9/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
MacBook-Pro:~ valassis$

Configuration

$ ansible-config dump --only-changed

MacBook-Pro:~ valassis$ ansible-config dump --only-changed
MacBook-Pro:~ valassis$

OS / Environment

MacOS

Steps to Reproduce

- name: Check if the new domain certificate has already created
  aws_acm_info:
    domain_name: "*.{{domain_name}}"
    region: "{{ region_main }}"
  register: out_main
  failed_when: out_main.certificates == []
  tags:
    - route53

....

Expected Results

{“elbs”: [{“attributes”: {“access_log”: {“enabled”: false}, “connection_draining”: {“enabled”: true, “timeout”: 300}, “connection_settings”: {“idle_timeout”: 60}, “cross_zone_load_balancing”: {“enabled”: true}}, “availability_zones”: [“us-east-1a”, “us-east-1b”, “us-east-1c”, “us-east-1d”, “us-east-1e”], “backend_server_description”: [], “canonical_hosted_zone_name”: “test-lb-XXXXXXXXXXXX.us-east-1.elb.amazonaws.com”, “canonical_hosted_zone_name_id”: “XXXXXXXXXXXXXX”, “created_time”: “2017-08-23T18:25:03.280000+00:00”, “dns_name”: “test-lb-XXXXXXXXXXXX.us-east-1.elb.amazonaws.com”, “health_check”: {“healthy_threshold”: 10, “interval”: 30, “target”: “HTTP:80/index.html”, “timeout”: 5, “unhealthy_threshold”: 2}, “instances”: [], “instances_inservice”: [], “instances_inservice_count”: 0, “instances_outofservice”: [], “instances_outofservice_count”: 0, “instances_unknownservice”: [], “instances_unknownservice_count”: 0, “listener_descriptions”: [{“listener”: {“instance_port”: 80, “instance_protocol”: “HTTP”, “load_balancer_port”: 80, “protocol”: “HTTP”}, “policy_names”: []}], “load_balancer_name”: “test-lb”, “policies”: {“app_cookie_stickiness_policies”: [], “lb_cookie_stickiness_policies”: [], “other_policies”: []}, “scheme”: “internet-facing”, “security_groups”: [“sg-29d13055”], “source_security_group”: {“group_name”: “default”, “owner_alias”: “XXXXXXXXXXXX”}, “subnets”: [“subnet-XXXXXXXX”, “subnet-XXXXXXXX”], “tags”: {}, “vpc_id”: “vpc-c248fda4”}]}

Actual Results

TASK [Get the VPC frontend elb info] **********************************************************************************************************************************************************************************************
task path: /Users/valassis/testplaybook.yaml:54
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: valassis
<127.0.0.1> EXEC /bin/sh -c 'echo ~valassis && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "echo /Users/valassis/.ansible/tmp"&& mkdir "echo /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870" && echo ansible-tmp-1663022299.306713-28922-198337152409870="echo /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870" ) && sleep 0'
Using module file /Users/valassis/.ansible/collections/ansible_collections/community/aws/plugins/modules/elb_classic_lb_info.py
<127.0.0.1> PUT /Users/valassis/.ansible/tmp/ansible-local-288864yyoygyi/tmp5p_jrez3 TO /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870/AnsiballZ_elb_classic_lb_info.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870/ /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870/AnsiballZ_elb_classic_lb_info.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/usr/local/opt/[email protected]/bin/python3.9 /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870/AnsiballZ_elb_classic_lb_info.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /Users/valassis/.ansible/tmp/ansible-tmp-1663022299.306713-28922-198337152409870/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
File "/var/folders/06/0gkrqgn12615qsblsh8bp4640000gp/T/ansible_community.aws.elb_classic_lb_info_payload_xc6d_aoh/ansible_community.aws.elb_classic_lb_info_payload.zip/ansible_collections/community/aws/plugins/modules/elb_classic_lb_info.py", line 227, in main
File "/var/folders/06/0gkrqgn12615qsblsh8bp4640000gp/T/ansible_community.aws.elb_classic_lb_info_payload_xc6d_aoh/ansible_community.aws.elb_classic_lb_info_payload.zip/ansible_collections/community/aws/plugins/modules/elb_classic_lb_info.py", line 165, in list_elbs
File "/var/folders/06/0gkrqgn12615qsblsh8bp4640000gp/T/ansible_community.aws.elb_classic_lb_info_payload_xc6d_aoh/ansible_community.aws.elb_classic_lb_info_payload.zip/ansible_collections/community/aws/plugins/modules/elb_classic_lb_info.py", line 192, in get_lb
File "/var/folders/06/0gkrqgn12615qsblsh8bp4640000gp/T/ansible_community.aws.elb_classic_lb_info_payload_xc6d_aoh/ansible_community.aws.elb_classic_lb_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/core.py", line 334, in deciding_wrapper
return retrying_wrapper(*args, **kwargs)
File "/var/folders/06/0gkrqgn12615qsblsh8bp4640000gp/T/ansible_community.aws.elb_classic_lb_info_payload_xc6d_aoh/ansible_community.aws.elb_classic_lb_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 118, in _retry_wrapper
return _retry_func(
File "/var/folders/06/0gkrqgn12615qsblsh8bp4640000gp/T/ansible_community.aws.elb_classic_lb_info_payload_xc6d_aoh/ansible_community.aws.elb_classic_lb_info_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
return func()
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 514, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 938, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidAction) when calling the DescribeLoadBalancers operation: The action DescribeLoadBalancers is not valid for this web service.
fatal: [localhost]: FAILED! => {
"boto3_version": "1.24.71",
"botocore_version": "1.27.71",
"changed": false,
"error": {
"code": "InvalidAction",
"message": "The action DescribeLoadBalancers is not valid for this web service."
},
"invocation": {
"module_args": {
"aws_access_key": null,
"aws_ca_bundle": null,
"aws_config": null,
"aws_secret_key": null,
"debug_botocore_endpoint_logs": false,
"ec2_url": null,
"names": [
"vpc-frontend"
],
"profile": null,
"region": "eu-west-1",
"security_token": null,
"validate_certs": true
}
},
"msg": "Failed to get load balancer information.: An error occurred (InvalidAction) when calling the DescribeLoadBalancers operation: The action DescribeLoadBalancers is not valid for this web service.",
"request_id": "05043107-0d14-4a2f-901d-22c598719aa3",
"response_metadata": {
"http_headers": {
"cache-control": "no-cache, no-store",
"connection": "close",
"content-type": "text/xml;charset=UTF-8",
"date": "Mon, 12 Sep 2022 22:38:19 GMT",
"server": "AmazonEC2",
"strict-transport-security": "max-age=31536000; includeSubDomains",
"transfer-encoding": "chunked",
"vary": "accept-encoding",
"x-amzn-requestid": "05043107-0d14-4a2f-901d-22c598719aa3"
},
"http_status_code": 400,
"retry_attempts": 0
}
}

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

ansibullbot · 2022-09-12T23:02:02Z

Files identified in the description:

[plugins/modules/elb_classic_lb_info.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/elb_classic_lb_info.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot · 2022-09-12T23:02:03Z

cc @jillr @markuman @mjschultz @nand0p @s-hertel @tremble
click here for bot help

tremble · 2022-09-13T13:33:12Z

Looking at the output:

"invocation": {
  "module_args": {
    "aws_access_key": null,
    "aws_ca_bundle": null,
    "aws_config": null,
    "aws_secret_key": null,
    "debug_botocore_endpoint_logs": false,
    "ec2_url": null,
    "names": [
      "vpc-frontend"
    ],
    "profile": null,
    "region": "eu-west-1",
    "security_token": null,
    "validate_certs": true
  }
}

It's unclear how you're passing the authentication tokens to botocore. I'm guessing you're either using the default botocore credentials (likely from ~/.aws) or you're using environment variables.

The error message however seems to imply that the endpoint URL is being overridden. Either by setting one of the environment variables (EC2_URL or AWS_URL) or by setting endpoint_url in the boto/aws configuration files. I'm guessing that one of these has been hard coded to a specific service endpoint URL that isn't the correct endpoint for the modules you're using.

kuvivek · 2022-09-13T18:35:51Z

Yes, I have EC2_URL being set in the $HOME/.bash_profile and also I have set the aws access keys and aws secret keys in th e $HOME/.aws/credentials file.

tremble · 2022-09-13T18:50:43Z

Setting EC2_URL is probably causing the issue you've seen, please try un-setting it.

kuvivek · 2022-10-11T06:56:18Z

Thanks after unsetting the EC2_URL, it started working.

…

On Tue, Sep 13, 2022 at 8:50 PM Mark Chappell ***@***.***> wrote: Setting EC2_URL is probably causing the issue you've seen, please try un-setting it. — Reply to this email directly, view it on GitHub <#1458 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AF5VG4UNJ62ZVGDMUR7DGETV6DEI3ANCNFSM6AAAAAAQK4WHCI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

Cleanup shared 'EC2' parameters SUMMARY As seen in both #1170 and ansible-collections/community.aws#1458 the "EC2_" prefix for a number of fall-back environment variables results in confusion. "EC2_" implies it's specific to the EC2 API, not all of AWS. Deprecates the ec2_ parameter aliases Deprecates the EC2_ environment variables Deprecates the security_token/aws_security_token/access_token parameter aliases which were based on the original boto parameter names ISSUE TYPE Feature Pull Request COMPONENT NAME plugins/module_utils/botocore.py plugins/module_utils/modules.py ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None>

Cleanup shared 'EC2' parameters SUMMARY As seen in both #1170 and ansible-collections/community.aws#1458 the "EC2_" prefix for a number of fall-back environment variables results in confusion. "EC2_" implies it's specific to the EC2 API, not all of AWS. Deprecates the ec2_ parameter aliases Deprecates the EC2_ environment variables Deprecates the security_token/aws_security_token/access_token parameter aliases which were based on the original boto parameter names ISSUE TYPE Feature Pull Request COMPONENT NAME plugins/module_utils/botocore.py plugins/module_utils/modules.py ADDITIONAL INFORMATION Reviewed-by: Alina Buzachis <None> (cherry picked from commit 06cecac)

[PR #1172/06cecace backport][stable-5] Cleanup shared 'EC2' parameters This is a backport of PR #1172 as merged into main (06cecac). SUMMARY As seen in both #1170 and ansible-collections/community.aws#1458 the "EC2_" prefix for a number of fall-back environment variables results in confusion. "EC2_" implies it's specific to the EC2 API, not all of AWS. Deprecates the ec2_ parameter aliases Deprecates the EC2_ environment variables Deprecates the security_token/aws_security_token/access_token parameter aliases which were based on the original boto parameter names ISSUE TYPE Feature Pull Request COMPONENT NAME plugins/module_utils/botocore.py plugins/module_utils/modules.py ADDITIONAL INFORMATION Reviewed-by: Mark Chappell <None>

ansibullbot added bug This issue/PR relates to a bug module module needs_triage plugins plugin (any type) python3 traceback labels Sep 12, 2022

tremble added the needs_info This issue requires further information. Please answer any outstanding questions label Sep 13, 2022

ansibullbot removed the needs_triage label Sep 13, 2022

ansibullbot removed the needs_info This issue requires further information. Please answer any outstanding questions label Sep 13, 2022

tremble closed this as completed Sep 21, 2022

tremble mentioned this issue Oct 14, 2022

Cleanup shared 'EC2' parameters ansible-collections/amazon.aws#1172

Merged

patchback bot mentioned this issue Oct 17, 2022

[PR #1172/06cecace backport][stable-5] Cleanup shared 'EC2' parameters ansible-collections/amazon.aws#1173

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

All aws ansible modules from the collection return the following error - An error occurred (400) when calling the operations Invalid #1458

All aws ansible modules from the collection return the following error - An error occurred (400) when calling the operations Invalid #1458

kuvivek commented Sep 12, 2022

ansibullbot commented Sep 12, 2022

ansibullbot commented Sep 12, 2022

tremble commented Sep 13, 2022

kuvivek commented Sep 13, 2022

tremble commented Sep 13, 2022

kuvivek commented Oct 11, 2022 via email

All aws ansible modules from the collection return the following error - An error occurred (400) when calling the operations Invalid #1458

All aws ansible modules from the collection return the following error - An error occurred (400) when calling the operations Invalid #1458

Comments

kuvivek commented Sep 12, 2022

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

/usr/local/lib/python3.9/site-packages/ansible_collections

/Users/valassis/.ansible/collections/ansible_collections

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

ansibullbot commented Sep 12, 2022

ansibullbot commented Sep 12, 2022

tremble commented Sep 13, 2022

kuvivek commented Sep 13, 2022

tremble commented Sep 13, 2022

kuvivek commented Oct 11, 2022 via email