Init vars escape issue

[1n]

When backend-config is passed to a subprocess with init_vars it is not properly escaped.
Example

import tftest
import logging

logging.basicConfig(level=logging.DEBUG)


if __name__ == "__main__":
    tf = tftest.TerraformTest('.', None, "terraform")
    backend_config = {
        "bucket": "bucket_name",
        "key": "dummy.tfstate",
        "region": "us-east-1",
    }
    tf.setup(init_vars=backend_config, cleanup_on_exit=False)

Output

DEBUG:tftest:['init', ('-no-color', '-input=false', "-backend-config='bucket=bucket_name'", "-backend-config='key=dummy.tfstate'", "-backend-config='region=us-east-1'")]
CRITICAL:tftest:Error running command init: 1 Initializing modules...
Downloading git::ssh://...
...


Initializing the backend...

Error: Invalid backend configuration argument

The backend configuration argument "'bucket" given on the command line is not
expected for the selected backend type.


Error: Invalid backend configuration argument

The backend configuration argument "'key" given on the command line is not
expected for the selected backend type.


Error: Invalid backend configuration argument

The backend configuration argument "'region" given on the command line is not
expected for the selected backend type.


Traceback (most recent call last):
  File "pr.py", line 14, in <module>
    tf.setup(init_vars=backend_config, cleanup_on_exit=False)
  File "/Users/vtk/.virtualenvs/qa/lib/python3.7/site-packages/tftest.py", line 309, in setup
    return self.init(plugin_dir=plugin_dir, init_vars=init_vars, backend=backend)
  File "/Users/vtk/.virtualenvs/qa/lib/python3.7/site-packages/tftest.py", line 317, in init
    return self.execute_command('init', *cmd_args).out
  File "/Users/vtk/.virtualenvs/qa/lib/python3.7/site-packages/tftest.py", line 397, in execute_command
    raise TerraformTestError(message)
tftest.TerraformTestError: Error running command init: 1 Initializing modules...
Downloading git::ssh://...
...

[ludoo]

Thanks for reporting this, I'll take a look as soon as possible.

[ludoo]

Uhm, the problem here is that the specified backend args are not valid for the selected backend (local). I am pushing a test that actually runs init on a simple fixture, can you please try and break it, considering the only valid backend config k/v for local backends is path?

[1n]

Sorry, I haven't mentioned that my module use s3 backend
https://www.terraform.io/docs/backends/types/s3.html

terraform {
  backend "s3" {}
}

I've thought that there is something wrong with k value as it looks strange(with a single quote at the beginning).

The backend configuration argument "'bucket" given on the command line is not
expected for the selected backend type.

This might be related to how either python/subprocess or shell interpret the combination of single and double quotes for result init command.

DEBUG:tftest:['init', ('-no-color', '-input=false', "-backend-config='bucket=bucket_name'", "-backend-config='key=dummy.tfstate'", "-backend-config='region=us-east-1'")]

For example

subprocess.run(['terraform', 'init', '-no-color', '-input=false', "-backend-config='bucket=bucket_name'", "-backend-config='key=
dummy.tfstate'", "-backend-config='region=us-east-1'"])

Output

Initializing modules...

Initializing the backend...

Error: Invalid backend configuration argument

The backend configuration argument "'bucket" given on the command line is not
expected for the selected backend type.


Error: Invalid backend configuration argument

The backend configuration argument "'key" given on the command line is not
expected for the selected backend type.


Error: Invalid backend configuration argument

The backend configuration argument "'region" given on the command line is not
expected for the selected backend type.

CompletedProcess(args=['terraform', 'init', '-no-color', '-input=false', "-backend-config='bucket=bucket_name'", "-backend-config='key=dummy.tfstate'", "-backend-config='region=us-east-1'"], returncode=1)

If single quotes removed for backend-config it works as expected.

subprocess.run(['terraform', 'init', '-no-color', '-input=false', "-backend-config=bucket=bucket_name", "-backend-config=key=dummy.tfstate", "-backend-config=region=us-east-1"])

[ludoo]

Ah, now I understand what you mean. This is super strange as I'm not getting the same error, which puzzles me. Let me try to see what breaks if we remove the quotes.

[ludoo]

Released as v1.5.4 can you test it and see if the issue is fixed?

https://pypi.org/project/tftest/1.5.4/

[1n]

Thanks!
The issue is fixed with v1.5.4.

[ludoo]

Thank you for identifying the issue and suggesting the fix!!!

[snahim-g]

Hello - looks like this is a bug when using gcs as the backend as well. I've had to remove the quotes.

[ludoo]

Hello - looks like this is a bug when using gcs as the backend as well. I've had to remove the quotes.

You mean we should reopen this? Can you provide a simple repro?