path.module during apply is module location from plan

[apparentlymart]

Terraform Version

Terraform 0.6.16

Steps to Reproduce

Consider the following configuration:

resource "null_resource" "example" {
    triggers {
        value = "${file("${path.module}/example.txt")}"
    }
}

1. Create an example.txt file alongside the above configuration file, with some placeholder content.
2. terraform plan -out=tfplan
3. Move the entire config directory, including the example.txt file and that tfplan file, to a different location on disk.
4. In the new location, terraform apply tfplan

Expected Behavior

Terraform should create the null_resource with the value set to the content of example.txt, because the plan contains the content of example.txt as was present when the plan was created.

Actual Behavior

An error like the following is produced:

Errors:

  * file: open /original/config/location/example.txt: no such file or directory in:

${file("${path.module}/example.txt")}

Apparently Terraform is re-processing the configuration before applying it, but the path.module variable expands to the module path from plan time, rather than the current module path during apply.

Whether Terraform should re-interpolate the whole config during apply is debatable anyway, but if it does then it should make path.module expand to the current configuration location, rather than the original location.

Important Factoids

The above example is of course a contrived one. The real situation where I experience this is when running Terraform on worker machines as part of a CI pipeline. In this case, we have one job that runs the plan and takes an archive of the whole Terraform context including that plan, and then a subsequent job that extracts that archive and tries to apply the stashed plan.

The follow-up job can't guarantee that the module will be extracted at exactly the same path across both the plan and apply steps, because the absolute module path contains a unique job id and these jobs may in fact be running on two different machines.

[apparentlymart]

Confirmed that this issue is still present in Terraform v0.9.3.

[marck-oemar]

This issue is present in Terraform v0.8.7. I'm using Concourse CI, which uses multiple worker machines to execute the build.

[stevehorsfield]

I suspect this may be related to an issue we see where file paths in submodules often fail to locate the file, even when the file location has not changed.

We've seen this issue in multiple TF versions from 0.9.3 up (at most 0.9.8) but can't say if it is definitely present in all of those revisions.

Example output:

* module.green.module.something.data.template_file.user_data: file: open /terraform/.terraform/modules/2732d34fa9b590be18880d322620ef3f/user-data.yaml: no such file or directory in:

${file("${path.module}/user-data.yaml")}

[apparentlymart]

Hi @stevehorsfield,

As long as the .terraform directory is retained between the plan and apply steps the path should still resolve, since that hash in the path is of the module name, which should be consistent between plan and apply. If you're seeing that even when the file in question is present on disk, that seems like a different bug and so it'd be great if you could file it as a separate issue and we can dig into it more.

While I'm here, I suppose the participants in this issue might be interested to know that once the website is updated for the 0.10.0 release we'll have a new guide describing our current recommendations and caveats for running Terraform within orchestration tools. We know that there are some gotchas and rough edges here, so we thought it best to write them down to at least help people avoid finding and having to solve the same problems separately. Until it's available on the website, you can see the copy of it in the master branch if you're interested. (Some of the details in there apply only to 0.10.0 and above, but much of it is generally-applicable.)

[stevehorsfield]

@apparentlymart - Thanks for that. I should have mentioned that we are using Terraform Enterprise. I don't believe it is retaining the folder structure between plan and apply stages. Please confirm either way. If you still think this is different, I'll create a new issue.

Thanks,

Steve

[apparentlymart]

Hi @stevehorsfield! Sorry I didn't notice your apply here before.

I'm not sure how the timeline of this thread correlates with the Terraform Enterprise changes but I can confirm that Enterprise does now preserve the working directory between plan and apply. This was changed in the last couple months so wasn't true when I originally opened this issue.

Previously the behavior was to retain the config and module source (the same as at the beginning of plan) but it would not preserve any files generated on disk during the plan step itself. This limitation has now been addressed so that should at least address that part of this problem.

(Enterprise also uses the same absolute path between plan and apply, so was never subject to the variant of the problem I'd originally described when opening this issue.)

[apparentlymart]

Oh, I also forgot to note that when I originally filed this I was a user of Terraform, but in the mean time I have joined the Terraform team at Hashicorp. The use-cases I was taking about in the original write-up were from my previous role.

[stevehorsfield]

Thanks @apparentlymart. Sounds like the TFE upgrade will fix our issue. We have that pending and it is certainly a few months out of date.

[etaty]

Just copy pasting my response put in #7613

I found a workaround to this using the current working directory path.cwd
If you remove the current working directory from the state file, then the state doesn't change!

data "archive_file" "slack_notification_zip" {
  type = "zip"
  output_path = "${path.module}/lambda.zip"

  source_dir = "${path.module}/source/"
}

resource "aws_lambda_function" "slack_notification" {
  filename = "${substr(data.archive_file.slack_notification_zip.output_path, length(path.cwd) + 1, -1)}"
  // +1 for removing the "/"
}

[stevehorsfield]

Nice. I like that, might try it. Thanks @etaty

[stevehorsfield]

We've upgraded TFE for other reasons so won't be able to verify the workaround.

[Ray-B]

We too are seeing this issue, since 0.6.16, and forward until now, 0.11.1.

Smaller teams who make use of terraform on smaller projects might characterize this as simply an annoyance, but this issue is insidious - It doesn't seem to scale well.

We have resources with this issue ordering in the many dozens. It inevitably ends up cluttering the terraform plans with pages of changes which need to be mentally parsed by bipedal hominids.

Workarounds exist, including targeted plans (we are testing out the ignore method for now) but the junior members of our operations team, when working against production instances under guidance, quickly grow tired and frustrated with this. It just makes the work painful. :(

I read through the related issues but it wasn't clear if this still in the solution discussion stage, or if someone has kicked off a spike somewhere? @apparentlymart, besides adding a voice, how can we assist?

[joestump]

We ran into this with CodePipeline where each CodeBuild step (plan and apply) have different $PWD on run. ${path.cwd} didn't fix this either.

[teamterraform]

Hi all,

This issue had a similar root cause to #7927, and should be resolved in the same way it was in 0.12: path.module is now relative to the current working directory and thus paths saved in the plan should also be relative to the current working directory.

If you are still seeing issues related to paths when applying a plan in a different location than it was created, please open a new issue with all the details in the issue template and we'll see what's special about your situation and what we could do to accommodate it. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.