Docker provider not able to depend on resource

[Schoonology]

👋🏼 Long-time listener, first-time caller.

Terraform Version

0.12.18

Affected Resource(s)

• The docker provider and all associated resources.
• The digitalocean_droplet resource might be contributing, as it's what I'm using, but my hunch is that the issue is more straightforward than that. See the dependency graph, below.

Terraform Configuration Files

A minimal file that demonstrates the issue:

provider "digitalocean" {
  token   = "token"
  version = "~> 1.11"
}

resource "digitalocean_droplet" "droplet" {
  image  = "docker-18-04"
  name   = "droplet"
  region = "sf02"
  size   = "s-1vcpu-1gb"
}

provider "docker" {
  host    = "tcp://${digitalocean_droplet.droplet.ipv4_address}:2376/"
  version = "~> 2.6"
}

resource "docker_network" "test" {
  name = "test"
}

Debug Output

This Gist has both the debug output of terraform plan, the sample configuration file above, and the SVG generated by terraform graph:

https://gist.github.com/Schoonology/5d524b2c4bd07ea75473cdaca0729782

Expected Behavior

I have a Docker provider dependent on the Droplet hosting the Docker daemon. According to terraform graph, this relationship is understood by Terraform:

Actual Behavior

When I run terraform plan, however, it appears as if the host property of the Docker provider is not understood to be dependent on that new resource.

Steps to Reproduce

In the directory containing the above Terraform spec, run terraform plan.

Thanks!

[mavogel]

Hi @Schoonology , thanks for revealing yourself :)
So, your issue is a more general one, not only for the docker provider. TLDR;: both providers you use here need to be split into 2 files or both resources need be created (plan & apply) after each other. You need to do this manually.

This is because the docker provider expects the host to respond when you hit plan on the docker_network resource.

Possible solutions are described here:

• Fail when trying to use postgressql provider against aws rds instance in same definition terraform-provider-postgresql#2 (comment)
• can't run postgresql with docker terraform-provider-postgresql#28 (comment)

When depends_on of the graph goes over modules (see hashicorp/terraform#18239 (comment)) is implemented, your issue will be solved.

HTH Manuel

[Schoonology]

Thanks, @mavogel!

The two-phase apply workaround is exactly the sort of thing I needed, and will work great while I wait for the core feature to be available.

Closing.

[staffan-einarsson]

Sorry for digging this up after closing, but I would like to expand on this a bit more.

The two phase apply described above works okay-ish for the creation scenario, but a lot of the point of using tools like terraform is to manage resource life cycles. Consider @Schoonology's scenario above but now we want to replace the droplet host with a new one because there is a new ubuntu version out. If this had worked out of the box as stated, terraform would have realized that the docker container needed to be reprovisioned on the new host and by putting a create_before_destroy on the container and an attachable IP we would be able to make the replacement in one go without any downtime. As it is now, accomplishing this would require more than two applies and several successive changes to the files in order to emulate the life cycle management.

The discussion about providers depending on resources is a very long story and is deeply rooted in terraform's core architecture. Forgive me if I sound pessimistic, but I don't see that as something that will drop out of the sky and solve everything anytime soon.

Looking at providers in general, I'm wondering if it is really a good idea to have things in the provider settings that might have finite life time. Things like AWS regions are truly global and will always live outside of my resource life cycle, while things like access tokens might be something I could provision but in the vast majority of cases I would want to handle them manually. But a host definitely belongs to the group of things that might, and is very likely to, have a finite life time.

Is it not better then to put the docker host address as an attribute on the resource? Or at least to have the option of putting it there, and leave the provider setting blank?

I get the feeling that provider settings have at times been used to avoid repetition in the resource definitions, possibly before local variables were a thing in terraform, and this is more of a symptom of HCL being an awkward language for writing clean and well factored code.

[mavogel]

Is it not better than to put the docker host address as an attribute on the resource? Or at least to have the option of putting it there, and leave the provider setting blank?

This could be an option yes. I don't know if this is intended to be for the providers, but definitely possible. @radeksimko what do you think about this?

[radeksimko]

This is a route some providers have taken - yes, e.g. Google provider added project and some other "provider-wide" fields to most resources while defaulting to provider settings:
https://www.terraform.io/docs/providers/google/d/datasource_compute_network.html#project

That said the SDK doesn't really provide any easy way to implement this and the Google provider is benefiting from the fact that large part of their codebase is generated, so doing this in Docker provider may be too tedious (manual) at this point.

It's something worth exploring in the plugin SDK though - if you don't find an issue about this I'd invite you open one!