AWS Environment Variable Errors

[lee5i3]

I am running into errors when trying to import my existing AWS resources into Terraform Enterprise remote state..

On Terraform v0.12.13, when I run

terraform import <module> <aws name>

It says

Acquiring state lock. This may take a few moments...

Warning: Value for var.AWS_ACCESS_KEY_ID unavailable

The value of variable "AWS_ACCESS_KEY_ID" is marked as sensitive in the remote
workspace. This operation always runs locally, so the value for that variable
is not available.


Warning: Value for var.AWS_SECRET_ACCESS_KEY unavailable

The value of variable "AWS_SECRET_ACCESS_KEY" is marked as sensitive in the
remote workspace. This operation always runs locally, so the value for that
variable is not available.


Error: Value for undeclared variable

A variable named "AWS_ACCESS_KEY_ID" was assigned a value, but the root module
does not declare a variable of that name. To use this value, add a "variable"
block to the configuration.


Error: Value for undeclared variable

A variable named "AWS_SECRET_ACCESS_KEY" was assigned a value, but the root
module does not declare a variable of that name. To use this value, add a
"variable" block to the configuration.

However when I downgrade to Terraform v0.12.12, it works fine

[lee5i3]

my workaround was to add the following to my variables, import, then remove them

variable "AWS_SECRET_ACCESS_KEY" {
  default = ""
}

variable "AWS_ACCESS_KEY_ID" {
  default = ""
}

[acdha]

This isn't just the AWS keys — I ran into it with some variables we'd set for our accounts which aren't used by the AWS SDK but do have “AWS” in the name.

[teamterraform]

Hi @lee5i3! Thanks for reporting this.

We'd like to understand a bit better what situation is leading to this behavior. It sounds like you don't have either the AWS_SECRET_ACCESS_KEY or AWS_ACCESS_KEY_ID variables defined in your configuration, but yet somehow Terraform is deciding they are set. Is that correct?

Our best guess right now is that in your Terraform Cloud or Terraform Enterprise remote workspace you have configured stored values for these AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID variables, but there is no corresponding variable defined in the configuration. If that's true, then per Terraform's intended behavior it seems like it is the new behavior that is correct: Terraform is correctly reporting that you've set a variable that isn't declared.

To fix it, you can either declare those variables as you showed in your follow-up comment (if you do actually need them) or you can remove them from the settings in the remote workspace in Terraform Cloud or Terraform Enterprise (if they are not actually needed).

Based on the names of these, we wonder if you intended to create these actually as environment variables rather than Terraform variables. Both of those are configurable on the same page in Terraform Cloud and Enterprise, and setting both of these in the Environment Variables section will allow the AWS provider to access them for remote operations run inside Terraform Cloud, and allow you to separately set them locally when you run commands like terraform import which do not support remote operations.

This specific situation aside, the general rule here is that only Terraform variables actually declared in the configuration should be configured in the remote workspace. When Terraform is running local operations against a remote workspace (as with terraform import), it will fetch all of the non-sensitive variables defined on the remote workspace and try to use them for the local operation, leading to this error if the variables in question are not declared.

It sounds like prior versions of Terraform were incorrectly permitting these undeclared variables, and thus allowing the import operation to work in spite of the incorrect configuration.

[lee5i3]

Yes we are using Terraform Cloud, we have the AWS credentials set as an Environment Variables in each workspace, they are not set as Terraform Variables

[triduongtran]

@teamterraform I ran into the exact same issue just like @lee5i3 which I’m trying to explain here:

On terraform 0.12.12 we used to successfully declare environment variables (not terraform variables) AWS_ACCESS_KEY_ID and the respective AWS_SECRET_ACCESS_KEY and let the remote workspace apply changes on AWS. When using terraform import locally we of course have to set the respective environment variables for the CLI because the variables in the workspace are not used.

The scenario I described above used to work with terraform 0.12.12 but since 0.12.13 the same procedure exits with the error as @lee5i3 pointed out.

My expectation here is that terraform should not require me to define a terraform variable for these environment variables in my terraform template.

[acdha]

My project similarly set those as environmental variables specifically following the recommendation to avoid this error. We were bulk provisioning workspaces and set a couple of environmental variables (e.g. TF_VAR_AWS_ACCOUNT_NAME) which the user wouldn’t have permission to retrieve directly.

(And to be clear: 0% chance these were Terraform variables – they were set by reviewed code, and Terraform was used for many tasks for months until the last release.)

[teamterraform]

Thanks for the additional context, all!

It sounds like the remote backend in incorrectly interpreting both the Terraform variables and the environment variables as if they are Terraform variables. In the underlying Terraform Cloud API the two are represented together as a single type of object, so it sounds like there's some missing filtering logic which wasn't previously causing problems because the validation of valid variable names wasn't working at all in prior releases.

We'll get this fixed up so that it will only try to use the ones defined under "Terraform Variables" in the UI, and ignore the ones under "Environment Variables".

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.