feature: ability to disable the dependency lockfile

[aslafy-z]

Current Terraform Version

v1.0.0

Use-cases

I want to use a custom provider binary (locally built) when developing or fixing bugs on it.

Attempted Solutions

• use the CLI configuration filesystem_mirror to host a modified version of the provider: warn for invalid hashes in the lockfile
• then, terraform providers lock: regenerates the lock file from the remote source (not local one)

Proposal

Add a flag -no-lockfile that completly ignore the dependency lockfile.

References

[apparentlymart]

Hi @aslafy-z,

When you're working on a local build of a provider that hasn't been published to a registry, you should use development overrides.

With a development override enabled, terraform plan, terraform apply, and other commands which take actions using providers will ignore the version number and checksums in the lock file and use the executable in the directory you specified, without any expectation that it's any particular released version of the provider.

As noted over there, terraform init's provider installer will still try to install what you've got recorded in the lock file because it's a component that installs providers rather than uses providers (which is also true for similar commands like terraform providers lock), but the other commands will ignore whatever package terraform init installed and use your specified override instead and so what the installer installed won't actually be relevant in practice, until you disable the override again when you are finished with development.

If you have further questions about this after you try it, please start a topic in the community forum where we can work through the specifics for your situation together if needed.

Thanks!

[terlar]

I still think this is a valid request, seems a lot of people are requesting the feature to disable the lockfile for many different reasons.

Two on top of my head:

• read only terraform directory
• managing plugins and pinning by themselves

Reference to people having this sentiment can be found in:

• Override path for .terraform.lock.hcl #27241
• Feature request: Add a new flag terraform init -lockfile=readonly to suppress updating .terraform.lock.hcl #27506
• Add ability to specify a .terraform.lock.hcl file on init #29112
• Terraform overrides .terraform.lock.hcl when symlinked #27158
• Provider lock file is not created in the current working directory AND is not included in TF_DATA_DIR #28402

Then there is also a lot of issues opened related to the terraform lock file and using different platforms. I won't link them here though, since that is somewhat of topic. But that could also be a reason why people would like to opt out.

I was planning to create an issue for this for some time and was half way through, but unfortunately lost the content. If you think it would be valuable for me to create a new issue for this, I can do that again, or if we could re-use this issue, which at least has the correct title.

Thanks!

[apparentlymart]

Hi @terlar,

For the problem of read-only filesystems (and other related situations where we don't want to modify the filesystem, even if that were possible) we added a -lockfile=readonly option to terraform init in one of the v0.15 releases, which has therefore been in all of the v1.0 releases. That tells Terraform not to update a lock file, though it does require that someone would've previously put a lock file in place first, perhaps by populating it on some other machine, or by using some other third-party software to generate it.

We are also aware of the challenges that arise when using the lock file with the optional global provider cache directory, which I assume is what you're referring to with regard to lock file on different platforms. There are documented workarounds for that, and we do intend to improve on it in a later release. That's currently represented by #27811.

If you don't wish to use the lock file to persist selections between runs of Terraform then you can instead treat it as a per-working-directory artifact by putting it in your .gitignore file, or equivalent in other version control systems. In that case, terraform init will re-construct it each time you start using Terraform in a new working directory, which therefore reproduces the effective behavior of Terraform before it had a lock file. However, anyone doing that should keep in mind that they will defeat one of the mechanisms by which Terraform helps ensure that nobody has tampered with the provider packages most recently selected. There are other parts of that strategy, such as TLS certificate verification of the registry server, but some threat models require a "trust-on-first-use" approach and that in particular will be defeated by not storing the lock file in your version control system.

[terlar]

In my case (using Nix to manage the terraform/terraform plugins) and also creating a read-only directory with a set of terraform files it is not really possible to use .gitignore or have the files be re-constructed every time. The only option is to figure out how to generate the lock file up-front (does it have docs on how it calculates the hashes, is it trivial?). This is however a lot of extra effort for no benefit.

Using Nix it has already been possible to verify all these things since the initial versions of terraform, so that is not of any concern. It validates the whole chain of dependencies and not just the plugins.

I don't have experience with other tools, but reading comments there seems to be other tools built around terraform that have the same issues.

So in my opinion best option would be to opt-out, second best to be able to specify another path for the lock-file, so you at least have an option to have it writeable (there is already a ticket for that) or the third best would be to pre-generate the lock-file.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.