Packer >=1.7.3 JSON AWS Variables Key and Value Issue

[BClev]

Background
In versions of Packer before 1.7.3, the following worked fine for JSON:

In Packer var file (e.g. packer-vars.json):

...
    "aws_region": "us-east-1",
    "ami_regions": "us-east-1",
    "kms_key_id": "arn:aws:kms:us-east-1:<account>:key/<id>",
...

In Packer json file under builders:

...
      "region": "{{user `aws_region`}}",
      "ami_regions": ["{{user `ami_regions`}}"],
...
      "kms_key_id": "{{user `kms_key_id`}}",
      "region_kms_key_ids": {
        "{{user `aws_region`}}": "{{user `kms_key_id`}}"
      },
...

In case there were additional regions, I'd just append an increasing numeral to the additional items under region_kms_key_ids (e.g. aws_region1)

Issue
Running this in Packer >1.7.3, the following error occurs:

2021/07/27 16:38:58 packer-builder-amazon-ebs plugin: [INFO] (aws): No AWS timeout and polling overrides have been set. Packer will default to waiter-specific delays and timeouts. If you would like to customize the length of time between retries and max number of retries you may do so by setting the environment variables AWS_POLL_DELAY_SECONDS and AWS_MAX_ATTEMPTS or the configuration options aws_polling_delay_seconds and aws_polling_max_attempts to your desired values.
2021/07/27 16:38:58 packer-builder-amazon-ebs plugin: Cannot copy AMI to AWS session region 'us-east-1', deleting it from `ami_regions`.
2021/07/27 16:38:58 Build 'amazon-ebs' prepare failure: 1 error(s) occurred:

* "{{user `kms_key_id`}}" is not a valid KMS Key Id.

1 error(s) occurred:

* "{{user `kms_key_id`}}" is not a valid KMS Key Id.

I've tried changing the KMS key ID to an Alias or the ID itself. The only fix is to either remove the "region_kms_key_ids" entirely (which works for single region deployments but not multi), or hardcoding either the region or kms_key_id:

      "region_kms_key_ids": {
        "us-east-1": "{{user `kms_key_id`}}",
}, 

or

      "region_kms_key_ids": {
        "{{user `aws_region`}}": "arn:aws:kms:us-east-1:<account>:key/<id>",
}, 

It's almost as if having a variable on each side of the mapping is causing the issue. Again, this worked fine prior to 1.7.3.

[riznob]

I am seeing the same issue when using vpc_filter, subnet_filter, and security_group_filter with user variables as both key and values. The issue exists in versions 1.7.3 and greater.

I have the variables defined in the variables block...

"variables": {
      "tagName": "bakery",
      "tagValue": "packer"
    }

Here is the vpc_filter config...

        "vpc_filter": {
          "filters": {
            "tag:{{user `tagName`}}": "{{user `tagValue`}}"
          }
        }

here is the error...

--> aws: Exactly one VPC should match the filter, but 0 VPC's was found matching filters: {
  Filters: [{
      Name: "tag:bakery",
      Values: ["{{user `tagValue`}}"]
    }]
}

Here is the subnet_filter config...

        "subnet_filter": {
          "filters": {
            "tag:{{user `tagName`}}": "{{user `tagValue`}}"
          },
          "most_free": true,
          "random": false
        }

here is the error...

--> aws: No Subnets was found matching filters: {
  Filters: [{
      Name: "state",
      Values: ["available"]
    },{
      Name: "vpc-id",
      Values: ["vpc-xxxxxxxx"]
    },{
      Name: "tag:bakery",
      Values: ["{{user `tagValue`}}"]
    }]
}

Here is the security_group_filter config...

        "security_group_filter": {
          "filters": {
            "tag:{{user `tagName`}}": "{{user `tagValue`}}"
          }
        }

here is the error...

--> aws: Error launching source instance: MissingParameter: When specifying a security group you must specify a group id for each item
        status code: 400, request id: xxxxxxxxxxxxxxxxxxxxxxxxx

[github-actions]

This issue has been migrated to hashicorp/packer-plugin-amazon#119 due to the Packer Plugin split.

Please follow the new issue for updates.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.