community.aws.elb_target_group ignoring targets.AvailabilityZone 'all' param #1736

octavian2204 · 2023-03-04T16:34:04Z

Summary

Having ALB in vpc-1 and targets in vpc-2 I am getting the error (despite that the targets.AvailabilityZone is set to 'all'):
"Couldn't register targets: An error occurred (ValidationError) when calling the RegisterTargets operation: You must specify a valid Availability Zone or select 'all' for all enabled Availability zones, since the IP address '192.168.112.221' is outside the VPC"

The ALB has Cross-zone load balancing enabled (On).

Despite that the module is failing, the TG is being created with no targets. If register the target manually (from AWS console) the module will report OK:
ok: [lvt00] => {
"changed": false,
...
}

Issue Type

Bug Report

Component Name

elb_target_group

Ansible Version

$ ansible --version

ansible [core 2.13.4]
  config file = None
  configured module search path = ['/Users/octav/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/octav/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.10.8 (main, Oct 21 2022, 22:22:30) [Clang 14.0.0 (clang-1400.0.29.202)]
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /Users/octav/ansible/collections/ansible_collections
Collection           Version
-------------------- -------
amazon.aws           5.2.0
ansible.posix        1.5.1
ansible.windows      1.13.0
community.aws        5.2.0
community.docker     3.4.1
community.postgresql 2.3.2

AWS SDK versions

$ pip3 show boto boto3 botocore

WARNING: Package(s) not found: boto
Name: boto3
Version: 1.24.69
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /opt/homebrew/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.27.69
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /opt/homebrew/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

CALLBACKS_ENABLED(/Users/octav/ansible/workdir/ansible.cfg) = ['yaml', 'profile_roles']
COLLECTIONS_PATHS(/Users/octav/ansible/workdir/ansible.cfg) = ['/Users/octav/ansible/collections']
DEFAULT_HOST_LIST(/Users/octav/ansible/workdir/ansible.cfg) = ['/Users/octav/ansible/workdir/inventory']
DEFAULT_LOAD_CALLBACK_PLUGINS(/Users/octav/ansible/workdir/ansible.cfg) = True
DEFAULT_ROLES_PATH(/Users/octav/ansible/workdir/ansible.cfg) = ['/Users/octav/ansible/roles']
HOST_KEY_CHECKING(/Users/octav/ansible/workdir/ansible.cfg) = False
INTERPRETER_PYTHON(/Users/octav/ansible/workdir/ansible.cfg) = auto_silent
INVENTORY_ENABLED(/Users/octav/ansible/workdir/ansible.cfg) = ['ini', 'host_list', 'script', 'auto', 'yaml', 'toml', 'tower']
RETRY_FILES_ENABLED(/Users/octav/ansible/workdir/ansible.cfg) = False

OS / Environment

ProductName: macOS
ProductVersion: 13.1
BuildVersion: 22C65

Steps to Reproduce

- name: "Create TG"
  community.aws.elb_target_group:
    access_key: "{{ aws_access_key_id }}"
    secret_key: "{{ aws_secret_access_key }}"
    name: "name"
    protocol: http
    port: 81
    vpc_id: 'vpc-069XXXXXXX2c' # VPC-1
    region: eu-west-1
    health_check_protocol: http
    health_check_path: /
    health_check_port: 81
    successful_response_codes: "200-299"
    health_check_interval: 15
    health_check_timeout: 5
    healthy_threshold_count: 4
    unhealthy_threshold_count: 3
    state: present
    target_type: ip
    targets:
      - Id: "{{ ansible_facts.default_ipv4.address }}"
        Port: 81
        AvailabilityZone: all
    tags: "{{ default_tags | combine({'Name':  'name'}) }}"

Expected Results

changed: [lvt00] => {
    "changed": true,
    "deregistration_delay_timeout_seconds": "300",
    "health_check_enabled": true,
    "health_check_interval_seconds": 15,
    "health_check_path": "/",
    "health_check_port": "81",
    "health_check_protocol": "HTTP",
    "health_check_timeout_seconds": 5,
    "healthy_threshold_count": 4,
    "invocation": {
        "module_args": {
            "access_key": "XXXXXXXXXXXXXXXXXX",
            "aws_ca_bundle": null,
            "aws_config": null,
            "debug_botocore_endpoint_logs": false,
            "deregistration_connection_termination": false,
            "deregistration_delay_timeout": null,
            "endpoint_url": null,
            "health_check_interval": 15,
            "health_check_path": "/",
            "health_check_port": "81",
            "health_check_protocol": "http",
            "health_check_timeout": 5,
            "healthy_threshold_count": 4,
            "load_balancing_algorithm_type": null,
            "modify_targets": true,
            "name": "name",
            "port": 81,
            "preserve_client_ip_enabled": null,
            "profile": null,
            "protocol": "http",
            "protocol_version": null,
            "proxy_protocol_v2_enabled": null,
            "purge_tags": true,
            "region": "eu-west-1",
            "secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "session_token": null,
            "state": "present",
            "stickiness_app_cookie_duration": null,
            "stickiness_app_cookie_name": null,
            "stickiness_enabled": null,
            "stickiness_lb_cookie_duration": null,
            "stickiness_type": null,
            "successful_response_codes": "200-299",
            "tags": {
                "Env": "Env_PROD",
                "Ioper": "Ioper_T1",
                "Name": "name"
            },
            "target_type": "ip",
            "targets": [
                {
                    "AvailabilityZone": "all",
                    "Id": "192.168.112.221",
                    "Port": 81
                }
            ],
            "unhealthy_threshold_count": 3,
            "validate_certs": true,
            "vpc_id": "vpc-069XXXXXXX2c",
            "wait": false,
            "wait_timeout": 200
        }
    },
    "ip_address_type": "ipv4",
    "load_balancer_arns": [
        "arn:aws:elasticloadbalancing:eu-west-1:58XXXXXXXX:loadbalancer/app/alb-ext/9e5cXXXXXXX25"
    ],
    "load_balancing_algorithm_type": "round_robin",
    "load_balancing_cross_zone_enabled": "use_load_balancer_configuration",
    "matcher": {
        "http_code": "200-299"
    },
    "port": 81,
    "protocol": "HTTP",
    "protocol_version": "HTTP1",
    "slow_start_duration_seconds": "0",
    "stickiness_app_cookie_cookie_name": "",
    "stickiness_app_cookie_duration_seconds": "86400",
    "stickiness_enabled": "false",
    "stickiness_lb_cookie_duration_seconds": "86400",
    "stickiness_type": "lb_cookie",
    "tags": {
        "Env": "Env_PROD",
        "Ioper": "Ioper_T1",
        "Name": "name"
    },
    "target_group_arn": "arn:aws:elasticloadbalancing:eu-west-1:XXXXXXX:targetgroup/bane/8e2b4XXXXXXXc",
    "target_group_health_dns_failover_minimum_healthy_targets_count": "1",
    "target_group_health_dns_failover_minimum_healthy_targets_percentage": "off",
    "target_group_health_unhealthy_state_routing_minimum_healthy_targets_count": "1",
    "target_group_health_unhealthy_state_routing_minimum_healthy_targets_percentage": "off",
    "target_group_name": "name",
    "target_type": "ip",
    "unhealthy_threshold_count": 3,
    "vpc_id": "vpc-069XXXXXXX2c"
}

Actual Results

The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_community.aws.elb_target_group_payload_fvdhi88t/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/community/aws/plugins/modules/elb_target_group.py", line 738, in create_or_update_target_group
  File "/tmp/ansible_community.aws.elb_target_group_payload_fvdhi88t/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/modules.py", line 354, in deciding_wrapper
    return retrying_wrapper(*args, **kwargs)
  File "/tmp/ansible_community.aws.elb_target_group_payload_fvdhi88t/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
    return _retry_func(
  File "/tmp/ansible_community.aws.elb_target_group_payload_fvdhi88t/ansible_community.aws.elb_target_group_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 69, in _retry_func
    return func()
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 530, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.9/site-packages/botocore/client.py", line 960, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the RegisterTargets operation: You must specify a valid Availability Zone or select 'all' for all enabled Availability zones, since the IP address '192.168.112.221' is outside the VPC
fatal: [lvt00]: FAILED! => {
    "boto3_version": "1.26.73",
    "botocore_version": "1.29.73",
    "changed": false,
    "error": {
        "code": "ValidationError",
        "message": "You must specify a valid Availability Zone or select 'all' for all enabled Availability zones, since the IP address '192.168.112.221' is outside the VPC",
        "type": "Sender"
    },
    "invocation": {
        "module_args": {
            "access_key": "XXXXXXXXXXXXXXXXXX",
            "aws_ca_bundle": null,
            "aws_config": null,
            "debug_botocore_endpoint_logs": false,
            "deregistration_connection_termination": false,
            "deregistration_delay_timeout": null,
            "endpoint_url": null,
            "health_check_interval": 15,
            "health_check_path": "/",
            "health_check_port": "81",
            "health_check_protocol": "http",
            "health_check_timeout": 5,
            "healthy_threshold_count": 4,
            "load_balancing_algorithm_type": null,
            "modify_targets": true,
            "name": "name",
            "port": 81,
            "preserve_client_ip_enabled": null,
            "profile": null,
            "protocol": "http",
            "protocol_version": null,
            "proxy_protocol_v2_enabled": null,
            "purge_tags": true,
            "region": "eu-west-1",
            "secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "session_token": null,
            "state": "present",
            "stickiness_app_cookie_duration": null,
            "stickiness_app_cookie_name": null,
            "stickiness_enabled": null,
            "stickiness_lb_cookie_duration": null,
            "stickiness_type": null,
            "successful_response_codes": "200-299",
            "tags": {
                "Env": "Env_PROD",
                "Ioper": "Ioper_T1",
                "Name": "name"
            },
            "target_type": "ip",
            "targets": [
                {
                    "AvailabilityZone": "all",
                    "Id": "192.168.112.221",
                    "Port": 81
                }
            ],
            "unhealthy_threshold_count": 3,
            "validate_certs": true,
            "vpc_id": "vpc-069XXXXXXX2c",
            "wait": false,
            "wait_timeout": 200
        }
    },
    "msg": "Couldn't register targets: An error occurred (ValidationError) when calling the RegisterTargets operation: You must specify a valid Availability Zone or select 'all' for all enabled Availability zones, since the IP address '192.168.112.221' is outside the VPC",
    "response_metadata": {
        "http_headers": {
            "connection": "close",
            "content-length": "415",
            "content-type": "text/xml",
            "date": "Sat, 04 Mar 2023 15:41:37 GMT",
            "x-amzn-requestid": "a980830a-8683-4946-ada7-7d1ab6b7b6ad"
        },
        "http_status_code": 400,
        "request_id": "a980830a-8683-4946-ada7-7d1ab6b7b6ad",
        "retry_attempts": 0
    }
}

Code of Conduct

I agree to follow the Ansible Code of Conduct

markuman · 2023-03-29T12:26:11Z

The property AvailabilityZone is lost in line 734.
https://github.com/ansible-collections/community.aws/blob/main/plugins/modules/elb_target_group.py#L730-L738

this might be a hotfix

diff --git a/plugins/modules/elb_target_group.py b/plugins/modules/elb_target_group.py
index 784fa143..cd2f4162 100644
--- a/plugins/modules/elb_target_group.py
+++ b/plugins/modules/elb_target_group.py
@@ -731,7 +731,10 @@ def create_or_update_target_group(connection, module):
                         instances_to_add = []
                         for target in params['Targets']:
                             if target['Id'] in add_instances:
-                                instances_to_add.append({'Id': target['Id'], 'Port': target['Port']})
+                                tmp = {'Id': target['Id'], 'Port': target['Port']}
+                                if target.get('AvailabilityZone'):
+                                  tmp['AvailabilityZone'] = target['AvailabilityZone']
+                                instances_to_add.append(tmp)
 
                         changed = True
                         try:

@octavian2204 do you have some time to test this hotfix or to work on this bug and contribute a proper fix?

octavian2204 · 2023-03-29T12:33:25Z

@markuman thx a lot for your effort. Unfortunately, will not be able to do it today. Maybe tomorrow. Will keep you in touch.

octavian2204 · 2023-03-30T13:43:25Z

Hi @markuman, your hotfix worked like a charm! Thank you!

elb_target_group: fix lost property AvailabilityZone SUMMARY Closes #1736 ISSUE TYPE Bugfix Pull Request COMPONENT NAME elb_target_group Reviewed-by: Mark Chappell Reviewed-by: Alina Buzachis Reviewed-by: Markus Bergholz <[email protected]>

elb_target_group: fix lost property AvailabilityZone SUMMARY Closes #1736 ISSUE TYPE Bugfix Pull Request COMPONENT NAME elb_target_group Reviewed-by: Mark Chappell Reviewed-by: Alina Buzachis Reviewed-by: Markus Bergholz <[email protected]> (cherry picked from commit d8362a0)

[PR #1767/d8362a0e backport][stable-5] elb_target_group: fix lost property AvailabilityZone This is a backport of PR #1767 as merged into main (d8362a0). SUMMARY Closes #1736 ISSUE TYPE Bugfix Pull Request COMPONENT NAME elb_target_group Reviewed-by: Markus Bergholz <[email protected]>

markuman added the bug This issue/PR relates to a bug label Mar 29, 2023

markuman added easyfix Good for new comers and easy to start with contribution waiting_on_contributor Needs help. Feel free to engage to get things unblocked labels Mar 29, 2023

markuman modified the milestones: 5.5.0, 6.0.0 Mar 29, 2023

tremble modified the milestones: 6.0.0, 5.5.0 Mar 29, 2023

markuman mentioned this issue Apr 5, 2023

elb_target_group: fix lost property AvailabilityZone #1767

Merged

softwarefactory-project-zuul bot closed this as completed in #1767 Apr 18, 2023

patchback bot mentioned this issue Apr 18, 2023

[PR #1767/d8362a0e backport][stable-5] elb_target_group: fix lost property AvailabilityZone #1776

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

community.aws.elb_target_group ignoring targets.AvailabilityZone 'all' param #1736

community.aws.elb_target_group ignoring targets.AvailabilityZone 'all' param #1736

octavian2204 commented Mar 4, 2023 •

edited

Loading

markuman commented Mar 29, 2023

octavian2204 commented Mar 29, 2023

octavian2204 commented Mar 30, 2023

community.aws.elb_target_group ignoring targets.AvailabilityZone 'all' param #1736

community.aws.elb_target_group ignoring targets.AvailabilityZone 'all' param #1736

Comments

octavian2204 commented Mar 4, 2023 • edited Loading

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

markuman commented Mar 29, 2023

octavian2204 commented Mar 29, 2023

octavian2204 commented Mar 30, 2023

octavian2204 commented Mar 4, 2023 •

edited

Loading