aws_ec2 and aws_rds inventories: An option to prefix hostvar names to prevent accidental collisions #535

giner · 2021-10-17T13:55:20Z

Summary

aws_ec2 and aws_rds inventory plugins 'pollute' hostvars with key-values which ansible has not control over. There are two problems with this:

there is no easy way to know which variables come from aws and which from static inventories
aws variables may accidentally overlap with other variables and this is very difficult to catch and debug

It would be useful to have an option of enabling prefixes for host variable names coming from AWS (e.g. aws_ec2_state instead of state).

Issue Type

Feature Idea

Component Name

aws_ec2 inventory (plugins/inventory/aws_ec2.py), aws_rds inventory (plugins/inventory/aws_rds.py)

Additional Information

Examples

EC2 instance variables without prefixes (without the proposed change)

[
  "ami_launch_index",
  "ansible_check_mode",
  "ansible_config_file",
  "ansible_dependent_role_names",
  "ansible_diff_mode",
  "ansible_facts",
  "ansible_forks",
  "ansible_host",
  "ansible_inventory_sources",
  "ansible_play_batch",
  "ansible_play_hosts",
  "ansible_play_hosts_all",
  "ansible_play_name",
  "ansible_play_role_names",
  "ansible_playbook_python",
  "ansible_role_names",
  "ansible_verbosity",
  "ansible_version",
  "architecture",
  "block_device_mappings",
  "capacity_reservation_specification",
  "client_token",
  "cpu_options",
  "ebs_optimized",
  "ena_support",
  "enclave_options",
  "environment",
  "group_names",
  "groups",
  "hibernation_options",
  "hostvars",
  "hypervisor",
  "iam_instance_profile",
  "image_id",
  "instance_id",
  "instance_type",
  "inventory_dir",
  "inventory_file",
  "inventory_hostname",
  "inventory_hostname_short",
  "key_name",
  "launch_time",
  "metadata_options",
  "monitoring",
  "network_interfaces",
  "omit",
  "owner_id",
  "placement",
  "play_hosts",
  "playbook_dir",
  "private_dns_name",
  "private_ip_address",
  "product_codes",
  "public_dns_name",
  "requester_id",
  "reservation_id",
  "role_names",
  "root_device_name",
  "root_device_type",
  "security_groups",
  "source_dest_check",
  "state",
  "state_transition_reason",
  "subnet_id",
  "tags",
  "virtualization_type",
  "vpc_id"
]

EC2 instance variables with prefixes (with the proposed change)

[
  "ansible_check_mode",
  "ansible_config_file",
  "ansible_dependent_role_names",
  "ansible_diff_mode",
  "ansible_facts",
  "ansible_forks",
  "ansible_host",
  "ansible_inventory_sources",
  "ansible_play_batch",
  "ansible_play_hosts",
  "ansible_play_hosts_all",
  "ansible_play_name",
  "ansible_play_role_names",
  "ansible_playbook_python",
  "ansible_role_names",
  "ansible_verbosity",
  "ansible_version",
  "aws_ec2_ami_launch_index",
  "aws_ec2_architecture",
  "aws_ec2_block_device_mappings",
  "aws_ec2_capacity_reservation_specification",
  "aws_ec2_client_token",
  "aws_ec2_cpu_options",
  "aws_ec2_ebs_optimized",
  "aws_ec2_ena_support",
  "aws_ec2_enclave_options",
  "aws_ec2_hibernation_options",
  "aws_ec2_hypervisor",
  "aws_ec2_iam_instance_profile",
  "aws_ec2_image_id",
  "aws_ec2_instance_id",
  "aws_ec2_instance_type",
  "aws_ec2_key_name",
  "aws_ec2_launch_time",
  "aws_ec2_metadata_options",
  "aws_ec2_monitoring",
  "aws_ec2_network_interfaces",
  "aws_ec2_owner_id",
  "aws_ec2_placement",
  "aws_ec2_private_dns_name",
  "aws_ec2_private_ip_address",
  "aws_ec2_product_codes",
  "aws_ec2_public_dns_name",
  "aws_ec2_requester_id",
  "aws_ec2_reservation_id",
  "aws_ec2_root_device_name",
  "aws_ec2_root_device_type",
  "aws_ec2_security_groups",
  "aws_ec2_source_dest_check",
  "aws_ec2_state",
  "aws_ec2_state_transition_reason",
  "aws_ec2_subnet_id",
  "aws_ec2_tags",
  "aws_ec2_virtualization_type",
  "aws_ec2_vpc_id",
  "environment",
  "group_names",
  "groups",
  "hostvars",
  "inventory_dir",
  "inventory_file",
  "inventory_hostname",
  "inventory_hostname_short",
  "omit",
  "play_hosts",
  "playbook_dir",
  "private_ip_address",
  "role_names"
]

RDS instance variables without prefixes (without the proposed change)

[
  "allocated_storage",
  "ansible_check_mode",
  "ansible_config_file",
  "ansible_dependent_role_names",
  "ansible_diff_mode",
  "ansible_facts",
  "ansible_forks",
  "ansible_inventory_sources",
  "ansible_play_batch",
  "ansible_play_hosts",
  "ansible_play_hosts_all",
  "ansible_play_name",
  "ansible_play_role_names",
  "ansible_playbook_python",
  "ansible_role_names",
  "ansible_verbosity",
  "ansible_version",
  "associated_roles",
  "auto_minor_version_upgrade",
  "availability_zone",
  "backup_retention_period",
  "ca_certificate_identifier",
  "copy_tags_to_snapshot",
  "customer_owned_ip_enabled",
  "db_cluster_identifier",
  "db_instance_arn",
  "db_instance_class",
  "db_instance_identifier",
  "db_instance_port",
  "db_instance_status",
  "db_parameter_groups",
  "db_security_groups",
  "db_subnet_group",
  "dbi_resource_id",
  "deletion_protection",
  "domain_memberships",
  "enabled_cloudwatch_logs_exports",
  "endpoint",
  "engine",
  "engine_version",
  "enhanced_monitoring_resource_arn",
  "environment",
  "group_names",
  "groups",
  "hostvars",
  "iam_database_authentication_enabled",
  "instance_create_time",
  "inventory_dir",
  "inventory_file",
  "inventory_hostname",
  "inventory_hostname_short",
  "kms_key_id",
  "license_model",
  "master_username",
  "monitoring_interval",
  "monitoring_role_arn",
  "multi_az",
  "omit",
  "option_group_memberships",
  "pending_modified_values",
  "performance_insights_enabled",
  "play_hosts",
  "playbook_dir",
  "preferred_backup_window",
  "preferred_maintenance_window",
  "private_host",
  "promotion_tier",
  "publicly_accessible",
  "read_replica_db_instance_identifiers",
  "region",
  "role_names",
  "storage_encrypted",
  "storage_type",
  "tag_list",
  "tags",
  "vpc_security_groups"
]

RDS instance variables with prefixes (with the proposed change)

[
  "ansible_check_mode",
  "ansible_config_file",
  "ansible_dependent_role_names",
  "ansible_diff_mode",
  "ansible_facts",
  "ansible_forks",
  "ansible_inventory_sources",
  "ansible_play_batch",
  "ansible_play_hosts",
  "ansible_play_hosts_all",
  "ansible_play_name",
  "ansible_play_role_names",
  "ansible_playbook_python",
  "ansible_role_names",
  "ansible_verbosity",
  "ansible_version",
  "aws_rds_allocated_storage",
  "aws_rds_associated_roles",
  "aws_rds_auto_minor_version_upgrade",
  "aws_rds_availability_zone",
  "aws_rds_backup_retention_period",
  "aws_rds_ca_certificate_identifier",
  "aws_rds_copy_tags_to_snapshot",
  "aws_rds_customer_owned_ip_enabled",
  "aws_rds_db_cluster_identifier",
  "aws_rds_db_instance_arn",
  "aws_rds_db_instance_class",
  "aws_rds_db_instance_identifier",
  "aws_rds_db_instance_port",
  "aws_rds_db_instance_status",
  "aws_rds_db_parameter_groups",
  "aws_rds_db_security_groups",
  "aws_rds_db_subnet_group",
  "aws_rds_dbi_resource_id",
  "aws_rds_deletion_protection",
  "aws_rds_domain_memberships",
  "aws_rds_enabled_cloudwatch_logs_exports",
  "aws_rds_endpoint",
  "aws_rds_engine",
  "aws_rds_engine_version",
  "aws_rds_enhanced_monitoring_resource_arn",
  "aws_rds_iam_database_authentication_enabled",
  "aws_rds_instance_create_time",
  "aws_rds_kms_key_id",
  "aws_rds_license_model",
  "aws_rds_master_username",
  "aws_rds_monitoring_interval",
  "aws_rds_monitoring_role_arn",
  "aws_rds_multi_az",
  "aws_rds_option_group_memberships",
  "aws_rds_pending_modified_values",
  "aws_rds_performance_insights_enabled",
  "aws_rds_preferred_backup_window",
  "aws_rds_preferred_maintenance_window",
  "aws_rds_promotion_tier",
  "aws_rds_publicly_accessible",
  "aws_rds_read_replica_db_instance_identifiers",
  "aws_rds_region",
  "aws_rds_storage_encrypted",
  "aws_rds_storage_type",
  "aws_rds_tag_list",
  "aws_rds_tags",
  "aws_rds_vpc_security_groups",
  "environment",
  "group_names",
  "groups",
  "hostvars",
  "inventory_dir",
  "inventory_file",
  "inventory_hostname",
  "inventory_hostname_short",
  "omit",
  "play_hosts",
  "playbook_dir",
  "private_host",
  "role_names"
]

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

ansibullbot · 2021-10-17T13:58:06Z

Files identified in the description:

[plugins/modules/aws_s3.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/aws_s3.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot · 2021-10-17T13:58:06Z

cc @jillr @linabuzachis @lwade @s-hertel @tremble @wimnat
click here for bot help

ansibullbot · 2021-10-17T14:09:58Z

Files identified in the description:

[lib/ansible/vars/hostvars.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/lib/ansible/vars/hostvars.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

jillr · 2021-10-19T19:55:38Z

Hi @giner Could you give us an example of a use case? Is this something that you could use compose for?

giner · 2021-10-20T06:01:27Z

Hi @jillr, let me illustrate by an example.

There are two machines, somehost1 - AWS EC2 instance (from dynamic inventory) and somehost2 - statically defined instance. Please take a look at the playbook and the results below. somehost1 is skipped (wrong behaviour) and somehost2 is not (correct behaviour).

$ cat play.yml
- hosts: somehost1  # AWS EC2 instance
  gather_facts: no
  tasks:
  - name: Perform this action
    debug:
      msg: Performing the action
    when: state is not defined or state is in ['started']

- hosts: somehost2  # Statically defined instance
  gather_facts: no
  tasks:
  - name: Perform another action
    debug:
      msg: Performing another action
    when: state is not defined or state is in ['started']

$ ansible-playbook -i inventory play.yml

PLAY [somehost1] ***********************************************************************************************************

TASK [Perform this action] *************************************************************************************************
skipping: [somehost1]

PLAY [somehost2] ***********************************************************************************************************

TASK [Perform another action] **********************************************************************************************
ok: [somehost2] =>
  msg: Performing another action

PLAY RECAP *****************************************************************************************************************
somehost1                  : ok=0    changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0
somehost2                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

$ ansible -i inventory -m debug -a "var=state" somehost1

PLAY [Ansible Ad-Hoc] ******************************************************************************************************

TASK [debug] ***************************************************************************************************************
ok: [somehost1] =>
  state:
    code: 16
    name: running

PLAY RECAP *****************************************************************************************************************
somehost1                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

abikouo · 2022-01-18T14:23:26Z

Hi @giner
Could you please give with the following pull request #621?
Thanks
Aubin

Add option to prefix and/or suffix hostvar coming from AWS SUMMARY Closes #535 ISSUE TYPE Feature Pull Request COMPONENT NAME Inventory plugins aws_ec2 aws_rds Reviewed-by: Mark Chappell <None> Reviewed-by: None <None> Reviewed-by: Alina Buzachis <None> Reviewed-by: None <None>

…ble-collections#535) * fix issue ansible-collections#343 for missing AWS_SESSION_TOKEN

ansibullbot added feature This issue/PR relates to a feature request module module needs_triage performance plugins plugin (any type) labels Oct 17, 2021

ansibullbot removed the module module label Oct 17, 2021

jillr removed the needs_triage label Oct 19, 2021

abikouo mentioned this issue Jan 18, 2022

Add option to prefix and/or suffix hostvar coming from AWS #621

Merged

abikouo added the has_pr label Jan 18, 2022

ansible-zuul bot closed this as completed in #621 Jan 19, 2022

abikouo pushed a commit to abikouo/amazon.aws that referenced this issue Oct 24, 2023

fix issue ansible-collections#343 for missing AWS_SESSION_TOKEN (ansi…

8b9ee85

…ble-collections#535) * fix issue ansible-collections#343 for missing AWS_SESSION_TOKEN

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

aws_ec2 and aws_rds inventories: An option to prefix hostvar names to prevent accidental collisions #535

aws_ec2 and aws_rds inventories: An option to prefix hostvar names to prevent accidental collisions #535

giner commented Oct 17, 2021 •

edited by abikouo

Loading

ansibullbot commented Oct 17, 2021

ansibullbot commented Oct 17, 2021

ansibullbot commented Oct 17, 2021

jillr commented Oct 19, 2021

giner commented Oct 20, 2021

abikouo commented Jan 18, 2022

aws_ec2 and aws_rds inventories: An option to prefix hostvar names to prevent accidental collisions #535

aws_ec2 and aws_rds inventories: An option to prefix hostvar names to prevent accidental collisions #535

Comments

giner commented Oct 17, 2021 • edited by abikouo Loading

Summary

Issue Type

Component Name

Additional Information

Examples

EC2 instance variables without prefixes (without the proposed change)

EC2 instance variables with prefixes (with the proposed change)

RDS instance variables without prefixes (without the proposed change)

RDS instance variables with prefixes (with the proposed change)

Code of Conduct

ansibullbot commented Oct 17, 2021

ansibullbot commented Oct 17, 2021

ansibullbot commented Oct 17, 2021

jillr commented Oct 19, 2021

giner commented Oct 20, 2021

abikouo commented Jan 18, 2022

giner commented Oct 17, 2021 •

edited by abikouo

Loading