Terraform Validate returns error if called with initialised directory as argument

[Blokje5]

When I run terraform validate <path> I get the following error: Error: Module not installed. It seems as if it uses the current folder to check for the .terraform folder, instead of at the given path.

Terraform Version

Terraform v0.12.2

Debug Output

When calling terraform validate environments/int the following output is returned

2019/06/18 13:48:23 [INFO] Terraform version: 0.12.2  
2019/06/18 13:48:23 [INFO] Go runtime version: go1.12.4
2019/06/18 13:48:23 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/0.6.0/versions/0.12.2/terraform", "validate", "environments/int"}
2019/06/18 13:48:23 [DEBUG] Attempting to open CLI config file: /Users/lennardeijsackers/.terraformrc
2019/06/18 13:48:23 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/06/18 13:48:23 [INFO] CLI command args: []string{"validate", "environments/int"}

Error: Module not installed

  on environments/int/iam.tf line 28:
  28: module "qa_bitbucket_pipeline_service_account" {

This module is not yet installed. Run "terraform init" to install all modules
required by this configuration.


Error: Module not installed

  on environments/int/iam.tf line 64:
  64: module "helm_repo_service_account" {

This module is not yet installed. Run "terraform init" to install all modules
required by this configuration.


Error: Module not installed

  on environments/int/main.tf line 2:
   2: module "ecr" {

This module is not yet installed. Run "terraform init" to install all modules
required by this configuration.


Error: Module not installed

  on environments/int/main.tf line 11:
  11: module "helm_repo" {

This module is not yet installed. Run "terraform init" to install all modules
required by this configuration.

When calling terraform validate from inside the environments/int folder:

Success! The configuration is valid.

Expected Behavior

Passing a path to terraform validate should use the .terraform directory at that path, correctly validating the code.

Actual Behavior

Terraform complains that the modules have not been initialised. It seems as if it checks for the .terraform folder at the current path, not the path that is specified as argument.

Steps to Reproduce

1. Create a terraform module (In my case, stored in /modules/module-name)
2. Create a folder such as environments/test.
3. Create some terraform code referencing that module.
4. Call terraform validate environments/test. This should yield: Error: Module not installed
5. cd into the environments test folder and run terraform validate. This should return the normal validate output.

[apparentlymart]

Hi @Blokje5! Thanks for reporting this.

The behavior you described, of looking for .terraform in the current working directory, is consistent with the behavior of all of the other Terraform commands: for those commands that accept a configuration directory as an argument, it is only the directory to look for the configuration in, and isn't used for any other purpose such as locating the .terraform directory.

While I can definitely see the arguments both ways, we know that there are many existing examples of folks relying on the current behavior in their automation for various reasons, such as initializing the same configuration directory multiple times from different working directories and performing different operations in each. While we can debate the relative merits of these approaches in the abstract, we can't change the existing behavior without a lot of consideration of the consequences and tradeoffs.

Interestingly, the most common direction this thought process has taken us in the past is in some sense the opposite of what you expected here: the prevailing idea right now is to remove the directory argument from every command except terraform init and have terraform init "remember" the configuration directory as part of the transient settings in the .terraform directory, so that users can use separated working directories and configuration directories without needing to worry about passing the same arguments to every command.

With all of this said, we recommend not relying on the directory arguments to these commands in general, because in any configuration that uses path.module and/or path.root it can cause details about the specific system running Terraform to be exposed into the state, making the configuration unportable to run on different systems. By far the most common and recommended way is to cd into the configuration directory and run all commands from there without any overriding path arguments. We support the path arguments mostly for backward-compatibility until such time as we have a strong plan for how to improve that situation.

With all of that said, I think unfortunately the behavior you are seeing is the "correct" behavior in the sense of consistency with how the other commands behave and what existing systems are expecting. We don't have any current plans to change the design here, because our attentions are currently elsewhere and most users are able to follow our recommendation to cd into the configuration directory before running any terraform subcommands.

Because there are no immediate plans to change this, I'm going to close this out. It's likely we'll open other issues in future representing specific plans, but those will depend on some more user research to figure out which of the several competing (and contradictory) approaches is likely to be helpful in the most scenarios. Thanks again for reporting this!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.