TF_CLI_ARGS with -var-file causes terraform plugin error.

[ephur]

Terraform Version

 terraform -v
Terraform v0.9.5

Affected Resource(s)

Please list the resources as a list, for example:

• provider.terraform
• provider.aws

This looks like an issue with handling of TF_CLI_ARGS

Terraform Configuration Files

Not provided.

Debug Output

snipped a lot of stuff out of the log to what appear to be relevant items, can't expose configuration details that much of debug log exposes.

https://gist.github.com/anonymous/39e50ec98e78b4d66f97e17af3af0d25

Expected Behavior

The plan should have been executed by just executing terraform plan with the environment variables set.

Actual Behavior

The following errors were observed and the plan is not executed. Unsetting environment variable and then

Error asking for user input: 2 error(s) occurred:

* provider.aws: plugin exited before we could connect
* provider.terraform: plugin exited before we could connect

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. export TF_CLI_ARGS="-var-file=../../global.tfvars -var-file=../../dev.tfvars"
2. terraform apply
3. Observe error:

Error asking for user input: 2 error(s) occurred:

* provider.terraform: plugin exited before we could connect
* provider.aws: plugin exited before we could connect

4. unset TF_CLI_ARGS
5. terraform apply -var-file=../../global.tfvars -var-file=../../dev.tfvars
6. After unsetting the env var the command executes fine and the plan completes.

Important Factoids

Are there anything atypical about your accounts that we should know? For example: Running in EC2 Classic? Custom version of OpenStack? Tight ACLs?

References

[ephur]

Confirmed this still exists in 0.9.8

[antonbabenko]

It still exists in 0.9.10. All I wanted was to disable colors when running on CI - TF_CLI_ARGS="-no-color"

[johnmarcou]

I was about to open an issue, but I found that one.
I confirm I had that issue as well.

Here the information:

[moby] ~/Terraform/empty # ls
main.tf
[moby] ~/Terraform/empty # cat main.tf
data "template_file" "config-01" {
  template = "$${hostname}"

  vars {
    hostname = "infra-01"
  }
}

[moby] ~/Terraform/empty # terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

data.template_file.config-01: Refreshing state...
No changes. Infrastructure is up-to-date.

This means that Terraform did not detect any differences between your
configuration and real physical resources that exist. As a result, Terraform
doesn't need to do anything.

[moby] ~/Terraform/empty # TF_CLI_ARGS="-no-color" terraform plan
Error asking for user input: 1 error(s) occurred:

* provider.template: plugin exited before we could connect

[moby] ~/Terraform/empty # terraform version
Terraform v0.9.11


[moby] ~/Terraform/empty # uname -a
Darwin macjohn.local 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64

[jlambert121]

We are having this issue as well. Granted it's 0.9.9. I have a trace log of this I'm happy to send privately if that is of value.

[reimerp]

confirmed against 0.10.6

[agyepiga]

I am having the same issue in Terraform v0.10.7

[DBarthe]

Happening in v0.11.2 when the terraform script contains provisioner 'file' or 'remote-exec' and with TF_CLI_ARGS=-no-color

[johnmarcou]

This is what I am doing as a workaround, loading this in my bashrc.

_terraform() {
  terraform $@ -var-file ~/.terraform.tfvars
}

alias terraform='_terraform'

If you have a better suggestion ?

[evanstachowiak]

Confirmed also in v0.11.7

[lorengordon]

Looks like I can workaround this problem using the command-specific variations...

export TF_CLI_ARGS_apply="-no-color"
export TF_CLI_ARGS_plan="-no-color"

[rwillians]

It also affects terraform output (Terraform v0.11.10).
Whatever arguments and options you pass, the command always returns its help message.

Btw, @lorengordon's workaround worked for me.

[ajorpheus]

Setting the CLI args per commands seems to be fine :

# Create a single var to hold the common CLI args
export TF_CLI_CUSTOM_ARGS="-var-file=$HOME/.aws/terraform.tfvars"

# Note that the variable expansion happens eagerly, so everytime you change the value of the above, need to re-run this command
for command in validate apply plan ; \
do export TF_CLI_ARGS_${command}="$TF_CLI_CUSTOM_ARGS"; done

# Check the value of the CLI args
set | grep -i "TF_CLI"

[pecigonzalo]

This is opened since 2017, is there any roadmap?

[rptaylor]

Still happening in 0.11.13

[emcniece]

Arrived here after installing https://github.com/sickill/stderred to get some stderr visibility. Colours break something in here.

[jlsan92]

Hello everyone! Any plan to fix this? Still happening, TF v0.12.12 🙏

[luk2302]

Still happens in 0.12.13. The core problem itself is a Wrong number of args of a spawned child process. Terraform starts its providers and provisioners as child processes and communicates with those processes afterwards. Note that this issue does not have anything to do with -var-file but with basically anything in TF_CLI_ARGS, in our case the contents are -no-color. The output we see after setting TF_LOG=TRACE is:

2019/11/05 09:50:44 [TRACE] LoadSchemas: retrieving schema for provisioner "local-exec"
2019-11-05T09:50:44.846+0100 [DEBUG] plugin: starting plugin: path=/usr/local/Cellar/tfenv/1.0.2/versions/0.12.13/terraform args=[/usr/local/Cellar/tfenv/1.0.2/versions/0.12.13/terraform, internal-plugin, provisioner, local-exec]
2019-11-05T09:50:44.849+0100 [DEBUG] plugin: plugin started: path=/usr/local/Cellar/tfenv/1.0.2/versions/0.12.13/terraform pid=35888
2019-11-05T09:50:44.849+0100 [DEBUG] plugin: waiting for RPC address: path=/usr/local/Cellar/tfenv/1.0.2/versions/0.12.13/terraform
2019-11-05T09:50:44.870+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [INFO] Terraform version: 0.12.13
2019-11-05T09:50:44.870+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [INFO] Go runtime version: go1.12.9
2019-11-05T09:50:44.870+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/1.0.2/versions/0.12.13/terraform", "internal-plugin", "provisioner", "local-exec"}
2019-11-05T09:50:44.870+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [DEBUG] Attempting to open CLI config file: /Users/lukasrinke/.terraformrc
2019-11-05T09:50:44.870+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019-11-05T09:50:44.871+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [INFO] TF_CLI_ARGS value: "-no-color"
2019-11-05T09:50:44.871+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 [INFO] CLI command args: []string{"internal-plugin", "-no-color", "provisioner", "local-exec"}
2019-11-05T09:50:44.871+0100 [DEBUG] plugin.terraform: 2019/11/05 09:50:44 Wrong number of args; expected: terraform internal-plugin pluginType pluginName
2019-11-05T09:50:44.872+0100 [DEBUG] plugin: plugin process exited: path=/usr/local/Cellar/tfenv/1.0.2/versions/0.12.13/terraform pid=35888 error="exit status 1"

The code responsible for handling, splitting and merging (!) the command line arguments and env variables is wrappedMain in main.go. Run in internal_plugin.go then expects a set amount of aruments but receives an extra one, namely -no-color.

// Prefix the args with any args from the EnvCLI
args, err = mergeEnvArgs(EnvCLI, cliRunner.Subcommand(), args)
with
EnvCLI = "TF_CLI_ARGS".

For the latest version of TF the log form the main process is no longer plugin exited before we could connect but Error: Failed to instantiate provisioner "local-exec" to obtain schema: Unrecognized remote plugin message:

Minimal main.tf to reproduce:
resource "null_resource" "null" { provisioner "file" { } }

Note that this affects everything that goes through internal_plugin.go which I suspect are all provisioners.

[danieldreier]

I have also reproduced this on a local build of the current tip of master for 0.12.18 with:

TF_CLI_ARGS="-no-color" terraform apply

for a trivial null resource tf config:

resource "null_resource" "example" {
  provisioner "local-exec" {
    command = "true"
  }
}

Many thanks to @luk2302 for digging into why this is happening.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.