backend/remote: Support HCL variable values in local operations #23229
Conversation
Previously any error case in the Context method would cause us to leave the remote workspace locked on exit, requiring manual action to unlock it.
apparentlymart
added
bug
backend/remote
v0.12
| @@ -156,3 +166,76 @@ func stubAllVariables(vv map[string]backend.UnparsedVariableValue, decls map[str | |||
|
|
|||
| return ret | |||
| } | |||
|
|
|||
| type remoteStoredVariableValue struct { | |||
There was a problem hiding this comment.
Since this is a wrapper around one type, it makes me ask why this doesn't happen in go-tfe?
I'm thinking it's so we have access to tfdiags, cty, etc. libraries that we have in terraform, but I thought the question worth asking.
There was a problem hiding this comment.
This type is an adapter between a go-tfe concept (*tfe.Variable) and a Terraform backend concept (backend.UnparsedVariableValue). As you noted, this needs to be on the Terraform side of that adapter so it can use the other Terraform-specific packages that the interface refers to.
I can see in retrospect that one thing that isn't obvious here is that remoteStoredVariableValue is an implementation of backend.UnparsedVariableValue; I'll add an interface implementation assertion adjacent to it and a comment here so that the meaning of this type is more explicit.
There was a problem hiding this comment.
Thanks for adding the context about the interface implementation!
| @@ -0,0 +1,143 @@ | |||
| package remote | |||
For remote operations, the remote system (Terraform Cloud or Enterprise) writes the stored variable values into a .tfvars file before running the remote copy of Terraform CLI. By contrast, for operations that only run locally (like "terraform import"), we fetch the stored variable values from the remote API and add them into the set of available variables directly as part of creating the local execution context. Previously in the local-only case we were assuming that all stored variables are strings, which isn't true: the Terraform Cloud/Enterprise UI allows users to specify that a particular variable is given as an HCL expression, in which case the correct behavior is to parse and evaluate the expression to obtain the final value. This also addresses a related issue whereby previously we were forcing all sensitive values to be represented as a special string "<sensitive>". That leads to type checking errors for any variable specified as having a type other than string, so instead here we use an unknown value as a placeholder so that type checking can pass. Unpopulated sensitive values may cause errors downstream though, so we'll also produce a warning for each of them to let the user know that those variables are not available for local-only operations. It's a warning rather than an error so that operations that don't rely on known values for those variables can potentially complete successfully. This can potentially produce errors in situations that would've been silently ignored before: if a remote variable is marked as being HCL syntax but is not valid HCL then it will now fail parsing at this early stage, whereas previously it would've just passed through as a string and failed only if the operation tried to interpret it as a non-string. However, in situations like these the remote operations like "terraform plan" would already have been failing with an equivalent error message anyway, so it's unlikely that any existing workspace that is being used for routine operations would have such a broken configuration.
apparentlymart
force-pushed
the
b-remote-backend-vars
branch
from
7d1ca08 to
74790d1
Compare
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked and limited conversation to collaborators
For remote operations, the remote system (Terraform Cloud or Enterprise) writes the stored variable values into a .tfvars file before running the remote copy of Terraform CLI.
By contrast, for operations that only run locally (like
terraform import), we fetch the stored variable values from the remote API and add them into the set of available variables directly as partof creating the local execution context.
Previously in the local-only case we were assuming that all stored variables are strings, which isn't true: the Terraform Cloud/Enterprise UI allows users to specify that a particular variable is given as an HCL expression, in which case the correct behavior is to parse and evaluate the expression to obtain the final value.
This also addresses a related issue whereby previously we were forcing all sensitive values to be represented as a special string
"<sensitive>". That leads to type checking errors for any variable specified as having a type other than string, so instead here we use an unknown value as a placeholder so that type checking can pass.Unpopulated sensitive values may cause errors downstream though, so we'll also produce a warning for each of them to let the user know that those variables are not available for local-only operations. It's a warning rather than an error so that operations that don't rely on known values for those variables can potentially complete successfully.
This can potentially produce errors in situations that would've been silently ignored before: if a remote variable is marked as being HCL syntax but is not valid HCL then it will now fail parsing at this early
stage, whereas previously it would've just passed through as a string and failed only if the operation tried to interpret it as a non-string.
However, in situations like these the remote operations like
"terraform plan" would already have been failing with an equivalent error message anyway, so it's unlikely that any existing workspace that is being used for routine operations would have such a broken configuration.
This fixes #23228.