Cannot use launch_template name in ec2_instance module #451

markuman · 2021-08-10T19:18:28Z

Two issues (TL;DR)

Cannot use launch_template name in ec2_instance module
- broken condition
You cannot use launch_template paremeter with other ec2_instance parameters in the same task (e.g. instance_type).
- that's a general boto3 limitation
- default values of ec2_instance is making this even more diffcult to handle

Summary

Origin Issue: ansible/ansible#51941
First trial of fix: ansible-collections/community.aws#111
Broken condition still exists: https://github.com/ansible-collections/amazon.aws/blob/main/plugins/modules/ec2_instance.py#L1165

However. It's not that easy.

Imo, LaunchTemplate parameters are not changeable in the context of run_instance.
You must read-out the launch_template parameters and throw them aginst run_instance to pretend they came from module.params.

"Failed to create new EC2 instance: An error occurred (Unsupported) when calling the RunInstances operation: The requested configuration is currently not supported. Please check the documentation for supported configurations."

One issues are the default values the modules applies.
{'BlockDeviceMappings': [], 'ClientToken': '37d405af374948e2bd427ff4a7a7fc04', 'InstanceType': 't2.micro', 'LaunchTemplate': {'LaunchTemplateName': 'ubuntu-20.04'}, 'MaxCount': 1, 'MinCount': 1, 'NetworkInterfaces': [{'DeviceIndex': 0, 'SubnetId': 'subnet-3e5fe743'}], 'TagSpecifications': [{'ResourceType': 'volume', 'Tags': [{'Key': 'Name', 'Value': 'lttest01'}]}, {'ResourceType': 'instance', 'Tags': [{'Key': 'Name', 'Value': 'lttest01'}]}]}

This happen when you run ec2_instance without any parameters, just with a LaunchTemplateName.
So the module applies

BlockDeviceMappings: []
InstanceType: t2.micro`

basically when I run

 - name: test launchtemplate usage
      ec2_instance:
        wait: yes
        name: "{{ name }}"
        instance_type: c5a.large
        launch_template:
          name: ubuntu-20.04
        vpc_subnet_id: subnet-3e5fe743

I expect that all values are taken from the launch_template, only the requested will be overwritten (instance_type and vpc_subnet_id.)

but because of the default values, when I request

 - name: test launchtemplate usage
      ec2_instance:
        wait: yes
        name: "{{ name }}"
        launch_template:
          name: ubuntu-20.04
        vpc_subnet_id: subnet-3e5fe743

what instance_type will be choosen? I expect the one that is defined in the launch_template (e.g. t3a.small). But the module will request a t2.micro instance (default value in ec2_instance).

so one point that must be reworked are the parameters that breaks the launch_template when no extra parameters are requested.
that might be the easiert part.

@@ -1273,16 +1262,17 @@ def build_run_instance_spec(params, ec2=None):
         MinCount=1,
     )
     # network parameters
-    spec['NetworkInterfaces'] = build_network_spec(params, ec2)
-    spec['BlockDeviceMappings'] = build_volume_spec(params)
-    spec.update(**build_top_level_options(params))
-    spec['TagSpecifications'] = build_instance_tags(params)
+    if not params.get('launch_template'):
+        spec['NetworkInterfaces'] = build_network_spec(params, ec2)
+        spec['BlockDeviceMappings'] = build_volume_spec(params)
+        spec['TagSpecifications'] = build_instance_tags(params)
+        # IAM profile
+        if params.get('instance_role'):
+            spec['IamInstanceProfile'] = dict(Arn=determine_iam_role(params.get('instance_role')))
+        spec['InstanceType'] = params['instance_type']
 
-    # IAM profile
-    if params.get('instance_role'):
-        spec['IamInstanceProfile'] = dict(Arn=determine_iam_role(params.get('instance_role')))
+    spec.update(**build_top_level_options(params))
 
-    spec['InstanceType'] = params['instance_type']
     return spec

the harder part is to read-out launch_template parameters and compare them to ec2_instance parameters if requested and throw everything against run_instance without LaunchTemplate parameter.

Issue Type

Bug Report

Component Name

ec2_instance

Ansible Version

ansible [core 2.11.3] 
  config file = None
  configured module search path = ['/home/m/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/m/.local/lib/python3.9/site-packages/ansible
  ansible collection location = /home/m/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/m/.local/bin/ansible
  python version = 3.9.6 (default, Jun 30 2021, 10:22:16) [GCC 11.1.0]
  jinja version = 3.0.1
  libyaml = True

Collection Versions

ansible-galaxy collection list

# /home/m/.ansible/collections/ansible_collections
Collection           Version  
-------------------- ---------
amazon.aws           1.5.0    
ansible.netcommon    1.5.0    
community.aws        1.5.0    
community.crypto     1.6.1    
community.dns        2.0.0-a1 
community.docker     1.4.0    
community.general    3.0.0    
community.libvirt    1.0.1    
community.mysql      2.1.0    
community.proxysql   1.2.0-dev
devsec.hardening     7.5.0    
google.cloud         1.0.1    
hetzner.hcloud       1.4.4    
lekker.aws           0.0.3    
markuman.hetzner_dns 1.6.1    
markuman.nextcloud   9.0.0    

# /home/m/.local/lib/python3.9/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0  
ansible.netcommon             2.2.0  
ansible.posix                 1.2.0  
ansible.utils                 2.3.0  
ansible.windows               1.7.0  
arista.eos                    2.2.0  
awx.awx                       19.2.2 
azure.azcollection            1.7.0  
check_point.mgmt              2.0.0  
chocolatey.chocolatey         1.1.0  
cisco.aci                     2.0.0  
cisco.asa                     2.0.2  
cisco.intersight              1.0.15 
cisco.ios                     2.3.0  
cisco.iosxr                   2.3.0  
cisco.meraki                  2.4.2  
cisco.mso                     1.2.0  
cisco.nso                     1.0.3  
cisco.nxos                    2.4.0  
cisco.ucs                     1.6.0  
cloudscale_ch.cloud           2.2.0  
community.aws                 1.5.0  
community.azure               1.0.0  
community.crypto              1.7.1  
community.digitalocean        1.8.0  
community.docker              1.8.0  
community.fortios             1.0.0  
community.general             3.4.0  
community.google              1.0.0  
community.grafana             1.2.1  
community.hashi_vault         1.3.2  
community.hrobot              1.1.1  
community.kubernetes          1.2.1  
community.kubevirt            1.0.0  
community.libvirt             1.0.1  
community.mongodb             1.2.1  
community.mysql               2.1.0  
community.network             3.0.0  
community.okd                 1.1.2  
community.postgresql          1.4.0  
community.proxysql            1.0.0  
community.rabbitmq            1.0.3  
community.routeros            1.2.0  
community.skydive             1.0.0  
community.sops                1.1.0  
community.vmware              1.12.0 
community.windows             1.5.0  
community.zabbix              1.4.0  
containers.podman             1.6.1  
cyberark.conjur               1.1.0  
cyberark.pas                  1.0.7  
dellemc.enterprise_sonic      1.1.0  
dellemc.openmanage            3.5.0  
dellemc.os10                  1.1.1  
dellemc.os6                   1.0.7  
dellemc.os9                   1.0.4  
f5networks.f5_modules         1.10.1 
fortinet.fortimanager         2.1.3  
fortinet.fortios              2.1.2  
frr.frr                       1.0.3  
gluster.gluster               1.0.1  
google.cloud                  1.0.2  
hetzner.hcloud                1.4.4  
hpe.nimble                    1.1.3  
ibm.qradar                    1.0.3  
infinidat.infinibox           1.2.4  
inspur.sm                     1.2.0  
junipernetworks.junos         2.3.0  
kubernetes.core               1.2.1  
mellanox.onyx                 1.0.0  
netapp.aws                    21.6.0 
netapp.azure                  21.8.1 
netapp.cloudmanager           21.8.0 
netapp.elementsw              21.6.1 
netapp.ontap                  21.8.1 
netapp.um_info                21.7.0 
netapp_eseries.santricity     1.2.13 
netbox.netbox                 3.1.1  
ngine_io.cloudstack           2.1.0  
ngine_io.exoscale             1.0.0  
ngine_io.vultr                1.1.0  
openstack.cloud               1.5.0  
openvswitch.openvswitch       2.0.0  
ovirt.ovirt                   1.5.3  
purestorage.flasharray        1.9.0  
purestorage.flashblade        1.6.0  
sensu.sensu_go                1.11.1 
servicenow.servicenow         1.0.6  
splunk.es                     1.0.2  
t_systems_mms.icinga_director 1.20.0 
theforeman.foreman            2.1.2  
vyos.vyos                     2.4.0  
wti.remote                    1.0.1

AWS SDK versions

pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: [email protected]
License: MIT
Location: /home/m/.local/lib/python3.9/site-packages
Requires: 
Required-by: 
---
Name: boto3
Version: 1.18.4
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /home/m/.local/lib/python3.9/site-packages
Requires: s3transfer, jmespath, botocore
Required-by: awslogs
---
Name: botocore
Version: 1.21.4
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /home/m/.local/lib/python3.9/site-packages
Requires: jmespath, urllib3, python-dateutil
Required-by: s3transfer, boto3, awscli

Configuration

OS / Environment

Linux rocket 5.11.22-2-MANJARO #1 SMP PREEMPT Fri May 21 17:45:54 UTC 2021 x86_64 GNU/Linux

Steps to Reproduce

    - name: find ubuntu ami
      ec2_ami_info:
        owners: 099720109477
        region: eu-central-1
        filters:
          name: ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20201026
      register: ami

    - name: find default security group
      amazon.aws.ec2_group_info:
        region: eu-central-1
        filters:
          group-name:
            - default
      register: SG

    - name: ubuntu 20.04 launch template
      community.aws.ec2_launch_template:
        template_name: ubuntu-20.04
        image_id: "{{ ami.images[0].image_id }}"
        instance_type: t3a.small
        credit_specification:
          cpu_credits: unlimited
        network_interfaces:
          - associate_public_ip_address: no
            delete_on_termination: yes
            device_index: 0
            groups:
              - "{{ SG.security_groups[0].group_id }}"
            subnet_id:  "{{ subnet }} "
        user_data: "{{ "uname -a" | b64encode }}"
        region: "{{ region }}"
        key_name: "{{ keypair }}"
        ebs_optimized: yes
        block_device_mappings:
          - device_name: /dev/sda1
            ebs:
              volume_size: 8
              volume_type: gp3
              delete_on_termination: true
              kms_key_id: "{{ KMS_ARN }}"
              encrypted: yes
          - device_name: /dev/sdb
            ebs:
              volume_size: 5
              volume_type: gp3
              delete_on_termination: true
              kms_key_id: "{{ KMS_ARN }}"
              encrypted: yes
        tags:
          ssome: tags

     - name: test launchtemplate usage
      ec2_instance:
        wait: yes
        name: "{{ name }}"
        instance_type: c5a.large
        launch_template:
          name: ubuntu-20.04
        vpc_subnet_id: subnet-3e5fe743

Expected Results

Deploy an ec2 instance with lauch_template parameters for all parameters that are not requested by ec2_instance module.

Actual Results

TASK [Provison ec2 instance(s)] ***********************************************************************************************
failed: [localhost] (item=boom) => {"changed": false, "item": "boom", "msg": "Could not create instance with launch template. Either launch_template.name or launch_template.id parameters are required"}

Code of Conduct

I agree to follow the Ansible Code of Conduct

The text was updated successfully, but these errors were encountered:

ansibullbot · 2021-08-10T19:29:27Z

Files identified in the description:

[plugins/modules/ec2_instance.py](https://github.com/['ansible-collections/amazon.aws', 'ansible-collections/community.aws', 'ansible-collections/community.vmware']/blob/main/plugins/modules/ec2_instance.py)

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibullbot · 2021-08-10T19:29:28Z

cc @jillr @ryansb @s-hertel @tremble @wimnat
click here for bot help

markuman · 2021-08-12T09:12:27Z

maybe the implementation is not that difficult at the end.
the only question that is left: what to do with the default values?

One possibility is, that they are only applied when launch_template parameter is not requested.

if not module.params.get('launch_template'):
    instance_type = 't2.micro'

But that means that they must remove from AnsibleAWSModule() initialisation, because you cannot differentiate if module.params.get('instance_type') is a requested value or a default value. Or is it possible?
Removing default values from AnsibleAWSModule() will result in doc errors I think.

Removing the default value in general will be the easiest fix. But that must be made in some iteration steps with deprecation warnings?

aaron-p-lehmann · 2021-10-07T14:35:48Z

I would love to see this implemented. IMO, the best behavior would be to apply the values in the following order:

default values of the module
values of the template
values explicit values of the module

I have 0 knowledge of boto, so I don't know how easy that is to make happen, though.

markuman · 2021-10-07T17:24:45Z

I think its vice-versa :)

take parameter values that are set by user in ec2_instance task.
take default values for all parameters that a not defined and have a default value.

If launch template is defined

take launch_template parameter values
overwrite launch_template parameter values that are set by user in ec2_instance task.

I have 0 knowledge of boto, so I don't know how easy that is to make happen, though.

I think it's not that difficult, but a lot of time consuming (adjust and expand integration tests).
If you like to get involved, you just missed the ansible-fest 2022. But there are many helping hands on libera irc in #ansible-aws and also #ansible-community

aaron-p-lehmann · 2021-10-07T18:05:36Z

I was thinking of the numbers from least important to most, and you were going vice versa, I think. In any case, I agree with you. I'd be interested in getting involved, but I don't know how much time I'd have to maintain stuff. Also, at my work, we're tied to ansible 2.9, unfortunately, so I wouldn't be able to use and test the thing very much. I'll definitley look into it, though, if we ever get things sorted out so that we can upgrade our ansible.

markuman mentioned this issue Aug 10, 2021

ec2_instance: fix launch_template condition ansible-collections/community.aws#111

Closed

ansibullbot added bug This issue/PR relates to a bug has_pr module module needs_triage plugins plugin (any type) python3 labels Aug 10, 2021

goneri added jira and removed needs_triage labels Aug 24, 2021

mandar242 mentioned this issue Dec 9, 2021

ec2_instance: Fix launch template condition, handle launch template - default value for instance_type #587

Merged

mandar242 self-assigned this Dec 9, 2021

ansible-zuul bot closed this as completed in 9a5a1eb Jan 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot use launch_template name in ec2_instance module #451

Cannot use launch_template name in ec2_instance module #451

markuman commented Aug 10, 2021 •

edited

Loading

ansibullbot commented Aug 10, 2021

ansibullbot commented Aug 10, 2021

markuman commented Aug 12, 2021

aaron-p-lehmann commented Oct 7, 2021

markuman commented Oct 7, 2021

aaron-p-lehmann commented Oct 7, 2021

Cannot use launch_template name in ec2_instance module #451

Cannot use launch_template name in ec2_instance module #451

Comments

markuman commented Aug 10, 2021 • edited Loading

Summary

Issue Type

Component Name

Ansible Version

Collection Versions

AWS SDK versions

Configuration

OS / Environment

Steps to Reproduce

Expected Results

Actual Results

Code of Conduct

ansibullbot commented Aug 10, 2021

ansibullbot commented Aug 10, 2021

markuman commented Aug 12, 2021

aaron-p-lehmann commented Oct 7, 2021

markuman commented Oct 7, 2021

aaron-p-lehmann commented Oct 7, 2021

markuman commented Aug 10, 2021 •

edited

Loading