Improvement/dry provisioning

[felddy]

Dynamic provisioning in the house. This works around this issue: hashicorp/terraform#953

[dav3r]

This all looks good to me. I tested it in my local workspace and it seems to work as advertised, but I didn't try it in Production yet.

Hold the phone- I think I found a bug. Checking now...

[dav3r]

There is a bug, but it's not in your code. However, this seems like a good place to fix it.

The problem is in these four places:

• https://github.com/dhs-ncats/cyhy_amis/blob/develop/terraform/cyhy_nmap_ec2.tf#L94
• https://github.com/dhs-ncats/cyhy_amis/blob/develop/terraform/cyhy_nmap_ec2.tf#L101
• https://github.com/dhs-ncats/cyhy_amis/blob/develop/terraform/cyhy_nessus_ec2.tf#L94
• https://github.com/dhs-ncats/cyhy_amis/blob/develop/terraform/cyhy_nessus_ec2.tf#L101

In all of the above, the code references ${aws_instance.cyhy_nmap.id} or ${aws_instance.cyhy_nessus.id}, which is the first nmap/nessus instance. What we want to do is reference the specific instance that matches the count variable:

• ${aws_instance.cyhy_nmap.*.id[count.index]}
• ${aws_instance.cyhy_nessus.*.id[count.index]}

Unfortunately, when I make that change, I run into this open Terraform issue:
hashicorp/terraform#14536

Example: In my local workspace, I have zero running nmap instances and I want to ramp up to three of them. The existing code has no problem with this because every nmap_cyhy_runner_data_attachment references (incorrectly) the first nmap instance (${aws_instance.cyhy_nmap.id}). The corrected code however fails with an error like this:

* aws_volume_attachment.nmap_cyhy_runner_data_attachment: 1 error(s) occurred:

* aws_volume_attachment.nmap_cyhy_runner_data_attachment[2]: index 2 out of range for list aws_instance.cyhy_nmap.*.id (max 2) in:

${aws_instance.cyhy_nmap.*.id[count.index]}

Anybody have any ideas on how to fix this?

[jsf9k]

The bug report you referenced says:

Replacing array syntax with element() function (as shown in comments in code) gives the desired behaviour.

Did you try that?

[dav3r]

I missed that- trying it now.

[dav3r]

That element() workaround worked- thanks @jsf9k. I had to make one other similar change to nmap.template and nessus.template get it to work. I'm going to do a little more testing, then commit the changes.

[jsf9k]

Please consider my comments before merging.

[jsf9k]

What about using logging instead of print statements?

[jsf9k]

I like imports to be sorted in three groups in this order:

• Python built-ins
• External dependencies
• Local dependencies

Then, within each group, I like the imports to be sorted alphabetically. This makes it easy to find imports as the list grows in length.

What do you think about doing that here?

[jsf9k]

Remove correctlyself and replace with correctly. 😃

[dav3r]

@jsf9k , the element() workaround appears to have a problem.

Example: When I attempt to scale up from 1 to 2 nmap instances, Terraform wants to recreate nmap_cyhy_runner_data_attachment[0], which would trigger the destruction of the cyhy_nmap[0] instance (since that is how we safely destroy the volume attachment):

-/+ aws_volume_attachment.nmap_cyhy_runner_data_attachment[0] (new resource required)
      id:                                        "vai-3307678400" => <computed> (forces new resource)
      device_name:                               "/dev/xvdb" => "/dev/xvdb"
      instance_id:                               "i-06916a3ed89b85a52" => "${element(aws_instance.cyhy_nmap.*.id, count.index)}" (forces new resource)
      skip_destroy:                              "true" => "true"
      volume_id:                                 "vol-02311105b958b2c31" => "vol-02311105b958b2c31"

It is the same issue as mentioned in this comment on Terraform issue 14536.

We need to figure out how to get Terraform to avoid destroying the existing data attachment(s) in this scenario.

[dav3r]

My last commit fixes the issue I described here.

[dav3r]

This now gets my official 👍