Autodownloading Terraform 1.8.2 breaks Atlantis entirely

[skolsuper]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Overview of the Issue

Hashicorp just released Terraform v1.8.2 with a breaking change to the release, by including a license file with the packaged version:

NOTE:

Starting with this release, we are including a copy of our license file in all packaged versions of our releases, such as the release .zip files. If you are consuming these files directly and would prefer to extract the one terraform file instead of extracting everything, you need to add an extra argument specifying the file to extract, like this:

unzip terraform_1.8.2_linux_amd64.zip terraform

Reproduction Steps

atlantis plan or atlantis apply with an unpinned terraform version

Logs

downloading terraform version 1.8.2 at "https://releases.hashicorp.com/terraform/1.8.2/terraform_1.8.2_linux_amd64.zip?checksum=file:https://releases.hashicorp.com/terraform/1.8.2/terraform_1.8.2_SHA256SUMS": expected a single file: /tmp/getter1796155136/archive

Environment details

Additional Context

See #4472 (comment)

👋🏻 TF Core maintainer here, who also happened to cut the release with the change that has impacted you. As mentioned above we did call this out in the changelog but broadly speaking did not anticipate this to be a breaking change that would warrant heads up.

go-getter, which is a library Atlantis appears to be using to install Terraform currently was not designed for this use case, or even broadly interacting with releases.hashicorp.com and we do not intend to make changes in that library to accommodate that use case.

On a more positive note, we do maintain a library that can aid with installation of HashiCorp products, including Terraform - hc-install and we use it ourselves in a number of places. Atlantis would be welcomed to use it as well. That library does not make assumptions about the archive having just one file. Our enterprise packages (although not relevant for Terraform here specifically) already have multiple files and the API accounts for this too, where the consumers specify a path where the license should be placed.

For Terraform CE, the API would let you specify Dir as a path to the directory where the archive gets unpacked, including terraform binary and the license file. What consumers decide to do with those files after the installation would be up to them. The API does not currently make any further assumptions about those files for non-Enterprise versions.

I appreciate this may not be as simple as a drop-in replacement but I hope the API along with examples and documentation is self-explanatory. If you have any questions about it or if it doesn't work for you we'd certainly like to hear it in our issue tracker and be more open to making some changes there (as opposed in go-getter).

[stasostrovskyi]

One possible workaround can be to set https://www.runatlantis.io/docs/server-configuration.html#tf-download to false for the time being.

[radeksimko]

I believe this is a duplicate of #4472 or the other way around.

Either way, I hope you can find this comment helpful: #4472 (comment)

[KevinBonilla]

My team's workaround as suggested by #4472 (comment) is pinning the required version in our main.tf. Annoying if you have large project environments. Will use this until this is fixed.
Here is the example:

Previous main.tf:

terraform {
  required_version = ">= 1.1.0, < 2.0.0"
  ...

Pinned main.tf:

terraform {
  required_version = "1.8.1"

I'm not on the latest version of atlantis helm release but it looks like you might be able to leverage defaultTFVersion in the values.yaml until this is sorted: https://artifacthub.io/packages/helm/atlantis/atlantis?modal=values&path=defaultTFVersion

defaultTFVersion: "1.8.1"

[nitrocode]

If you have 1000s of root dirs, without having to modify anything, one quick and dirty solution would be to softlink terraform1.8.2 to terraform1.8.1 for now.

This is what I did to unblock devs for now

ln -sf /atlantis/bin/terraform1.8.1 /atlantis/bin/terraform1.8.2

repos:
  - id: /.*/
    # temp fix for https://github.com/runatlantis/atlantis/issues/4471
    pre_workflow_hooks:
      - description: Temporary hack for post-terraform 1.8.1
        run: |
          ln -sf /atlantis/bin/terraform1.8.1 /atlantis/bin/terraform1.8.2
          # ...

This is a hack like Fernando mentioned. It's meant as a stop-gap for a better solution.

[FernandoMiguel]

One quick and easy solution would be to softlink terraform1.8.2 to terraform1.8.1 for now.

that's one horrible hack

[nitrocode]

Yes, agreed, it is a horrible hack.

[bezarsnba]

Hi there

As a workaround, we setting on the variables environment the follow value

ATLANTIS_DEFAULT_TF_VERSION=1.8.1
ATLANTIS_TF_DOWNLOAD=false

[FernandoMiguel]

As a workaround, we setting on the variables environment the follow value

ATLANTIS_DEFAULT_TF_VERSION=1.8.1
ATLANTIS_TF_DOWNLOAD=false

that wont work if your project are in different versions.
that would push them all to that version, and probably fail if they have correct pins

[bezarsnba]

As a workaround, we setting on the variables environment the follow value

ATLANTIS_DEFAULT_TF_VERSION=1.8.1
ATLANTIS_TF_DOWNLOAD=false

that wont work if your project are in different versions. that would push them all to that version, and probably fail if they have correct pins

As I said, it's a workaround and worked for us.

[nitrocode]

Thanks @bezarsnba for sharing.

To other members experiencing issues, please be open minded to these workarounds. I know many are frustrated. These are all temporary fixes to allow the app to work as expected until a better solution is available.

Please see the above linked PR #4474 for a better solution that should work for all use-cases.

[rkferreira]

I have also proposed a small fix.
Tested here with testdrive and worked fine.

#4478

[stasostrovskyi]

I agree that there should be a long-term, proper fix, and the sooner the better. However, while we are at the topic we should address a couple more issues:

• What should Atlantis do if required_version cannot be downloaded (for whatever reason)? Should Atlantis fail, like it happened here or fallback to the default version?
• There is still confusion on how to make autodownload feature work with custom workflows and more specifically, how should it work?
• At the end of the day, should Atlantis even download terraform, or should people who want this kind of functionality maybe use tfenv instead?

[Steffen911]

@stasostrovskyi Thank you for the version-pin PR. Do you plan to trigger a release for this to make the change available for users?

[stasostrovskyi]

I'm not doing release myself, but the fix was already cherry picked to release-0.27, so I would expect a new atlantis release soon.

[moemoeq]

here is slightly cleaner method than @nitrocode 's method(overwriting another version.)
this is using init config for downloading the 1.8.2 binary through and stores in plugins dir, to use the terraform1.8.2 command as the actual version.
Of course, this method is not a fundamental solution either, but it will prevent the problem of unintentionally confusing version 1.8.1 with 1.8.2 in the future.

here is my helm values examples

i hops this will helps who struggles to get work with cleaner methods

  initConfig:
    # -- Install providers/plugins into a path shared with the Atlantis pod.
    enabled: true
    image: alpine:latest
    imagePullPolicy: IfNotPresent
    # -- SharedDir is set as env var INIT_SHARED_DIR.
    sharedDir: /plugins
    workDir: /tmp
    # -- Size for the shared volume.
    sizeLimit: 100Mi
    # -- Security context for the container.
    securityContext: {}
    # -- Script to run on the init container.
    # @default -- Check values.yaml.
    script: |
      #!/bin/sh

      wget https://releases.hashicorp.com/terraform/1.8.2/terraform_1.8.2_linux_amd64.zip
      unzip terraform_1.8.2_linux_amd64.zip terraform
      cp /tmp/terraform ${INIT_SHARED_DIR}/terraform1.8.2
      ls -la ${INIT_SHARED_DIR}

[thomaschaplin]

upstream issue - hashicorp/terraform#35075

[atmask]

It looks like #4474 went out in release 0.27.3
Has a new image been pushed to ghcr yet? I cannot get anything on the 0.27.3 tag
Had to ask the question to find my answer...it is available at image tag v0.27.3 not 0.27.3

[nitrocode]

I added @radeksimko's comment regarding hc-install in #4472 (comment) to the OP issue.

One issue I have with hc-install is that it cannot be reworked to autodownloading for both terraform and opentofu, whereas warrensbox can be used for both.

I expressed my concerns in the PR here #4494

[lukemassa]

Closing this as the original issue has been mitigated in #4474, back ported into the 0.27 release branch, and released in 0.27.3. For discussion of a long-term fix, see #4494 and #4483

[RahmanBadru]

What is the fix for this, because none of what i tried in the comments worked

[briannipper]

@RahmanBadru - I found a simple work around that we are using until such time that I have a chance to update atlantis to the latest version.

Using this as a reference:
https://www.runatlantis.io/docs/terraform-versions.html#terraform-versions

I updated the atlantis.yaml file in each repo to pin the version of terraform to 1.8.1

It's not a great option if you have a lot of repos, but it works for us as we only have a handful of repos to deal with in terms of using atlantis.

Hope this helps.

[nitrocode]

What is the fix for this, because none of what i tried in the comments worked

Did you try upgrading to 0.27.3 as per #4471 (comment) ?

[RahmanBadru]

What is the fix for this, because none of what i tried in the comments worked

Did you try upgrading to 0.27.3 as per #4471 (comment) ?

Yeah i did, i use helm though, so i guess i have to bump the chart version

[RahmanBadru]

@RahmanBadru - I found a simple work around that we are using until such time that I have a chance to update atlantis to the latest version.

Using this as a reference: https://www.runatlantis.io/docs/terraform-versions.html#terraform-versions

I updated the atlantis.yaml file in each repo to pin the version of terraform to 1.8.1

It's not a great option if you have a lot of repos, but it works for us as we only have a handful of repos to deal with in terms of using atlantis.

Hope this helps.

I will try that, thanks