Depends_on for module

[felnne]

Possible workarounds

For module to module dependencies, this workaround by @phinze may help.

Original problem

This issue was promoted by this question on Google Groups.

Terraform version: Terraform v0.3.7

I have two terraform modules for creating a digital ocean VM and DNS records that are kept purposely modular so they can be reused by others in my organisation.

I want to add a series of provisioners using local_exec after a VM has been created and DNS records made.

Attempted solution

I tried adding a provisioner directly to my terraform file (i.e. not in a resource) which gave an error.

I then tried using the null_resource which worked but was executed at the wrong time as it didn't know to wait for the other modules to execute first.

I then tried adding a depends_on attribute to the null resource using a reference to a module but this doesn't seem to be supported using this syntax:

depends_on = ["module.module_name"]

Expected result

Either a way for a resource to depend on a module as a dependency or a way to "inject" (for lack of a better word) some provisioners for a resource into a module without having to make a custom version of that module (I realise that might be a separate issue but it would solve my original problem).

Terraform config used

# Terraform definition file - this file is used to describe the required infrastructure for this project.

# Digital Ocean provider configuration

provider "digitalocean" {
    token = "${var.digital_ocean_token}"
}


# Resources

# 'whoosh-dev-web1' resource

# VM

module "whoosh-dev-web1-droplet" {
    source = "github.com/antarctica/terraform-module-digital-ocean-droplet?ref=v1.0.0"
    hostname = "whoosh-dev-web1"
    ssh_fingerprint = "${var.ssh_fingerprint}"
}

# DNS records (public, private and default [which is an APEX record and points to public])

module "whoosh-dev-web1-records" {
    source = "github.com/antarctica/terraform-module-digital-ocean-records?ref=v0.1.1"
    hostname = "whoosh-dev-web1"
    machine_interface_ipv4_public = "${module.whoosh-dev-web1-droplet.ip_v4_address_public}"
    machine_interface_ipv4_private = "${module.whoosh-dev-web1-droplet.ip_v4_address_private}"
}


# Provisioning (using a fake resource as provisioners can't be first class objects)

# Note: The "null_resource" is an undocumented feature and should not be relied upon.
# See https://github.com/hashicorp/terraform/issues/580 for more information.

resource "null_resource" "provisioning" {

    depends_on = ["module.whoosh-dev-web1-records"]

    # This replicates the provisioning steps performed by Vagrant
    provisioner "local-exec" {
        command = "ansible-playbook -i provisioning/development provisioning/bootstrap-digitalocean.yml"
    }
}

[jaxxstorm]

+1 I came here to ask for this very same thing

[radeksimko]

👍 It's not just about adding depends_on to module DSL, but also fixing existing implementation of depends_on on raw resources so it accepts modules as dependencies.

[jaxxstorm]

I'm not sure if @felnne covered this as he doesn't have it in his example, but it would also be awesome of modules can depend on other modules

module "whoosh-dev-web1-records" {
    depends_on ['module.module_name]
    source = "github.com/antarctica/terraform-module-digital-ocean-records?ref=v0.1.1"
    hostname = "whoosh-dev-web1"
    machine_interface_ipv4_public = "${module.whoosh-dev-web1-droplet.ip_v4_address_public}"
    machine_interface_ipv4_private = "${module.whoosh-dev-web1-droplet.ip_v4_address_private}"
}

This way, we could bring up infrastructure in the correct order. At the moment, if something like consul depends on say, DNS, we can only have depends within a module itself. This way we can better ensure that services come up in the right order

[franklinwise]

+1 agree. Need to have modules that can be re-used atomically with parents connecting the dependency. This modularity enables easier testing, isolation, understandability... all the benefits that code gets from having module packages.

For example (with my best text art):

  # module folders
  ./consul_project/module-a/             # ability to reference output vars of module-common
  ./consul_project/module-b/             # ability to reference output vars of module-common
  ./consul_project/module-common/        # common to both a and b

  # parent
  ./consul_project/deploy-a-only.tf      # has module definitions for both module-a and module-common
  ./consul_project/deploy-b-only.tf      # likewise, but for module-b
  ./consul_project/deploy-all.tf         # defines all 3 modules

  % terraform plan deploy-b-only

[ketzacoatl]

+1

[ketzacoatl]

I just got really excited thinking depends_on would help me find a workaround to my cyclical dependency issue in #1637.. but then I remembered this issue. -_-

[JeanMertz]

What about the other way around? Having a module call be dependent on some other resource.

[chiefy]

👍 just ran into this - have a module which creates an ASG, and I need it to depend on the user-data template.

[phinze]

Just wanted to mention that while we don't yet support whole-module dependencies, there's nothing stopping you from wiring an output of one module into an input of another, which will effectively draw a dependency between the appropriate resources.

I'm not saying this necessarily will solve all use cases, but when I was trying to figure out why I haven't bumped into the need for depends_on = ['module.foo'], I realized that this is what I tend to do in my config.

[failshell]

👍

[jjhuff]

Did anyone find a workaround?
I packaged up a bunch of common instance bootstrap stuff in a module to reduce repetition, but that module has some params that use interpolated values. Ideally, the module instance would depend on all of the interpolated items. In lieu of that, manual depends_on would work.

module "notifications" {
  source = "../pm_instance"
  ...
  attributes=<<EOF
"notifications":{
    "sens_queue": "${aws_sqs_queue.notifications.arn}",
}
EOF
}

[youanswer]

Please add this feature, so it gives clear opportunity to use output variables of one module in another one.
In my case I wanted to create cluster with docker containers on digital ocean, so I have one module to create N servers, then in this module I was forced to use "null_resource" hack for provisioning, because I need to know IP of a resource to generate certificates, so docker daemon can be used from remote. Another module is used to start containers on created servers, so, obviously, also requires IPs of servers.
Ended up applying first module alone, then adding second module to config, then run terraform apply again.
So when I tried to terraform destroy of course everything crashed :(
I think this feature would be very useful to create complex architectures.

crash log: https://gist.github.com/youanswer/8ebdcd81aea9edc91f88
my structure: https://gist.github.com/youanswer/bc1ca37773df968038a8

[sl1pm4t]

+1

[chrisferry]

👍

[ketzacoatl]

I forgot about modules not supporting depends_on, and thought I could use this as a way to work around #2939.

[glinton]

+1

[maxenglander]

+1

[thegedge]

I'd just like to throw out another example of where this would be nice:

We have a services module that creates bastions and NATs in a VPC. We lock down port 22 so that only the bastion hosts can be accessed from the internet, but in our bastion module we open port 22 ingress over the entire VPC for connections coming from the bastion hosts.

Our NAT module takes as input the bastion host IP (bastion module output) for chef provisioning connections in an aws_instance, but there's no dependency between the NAT aws_instance resources and the security group rule that allows port 22 ingress from the bastions, so terraform will try to build the NAT instances before creating the SG rule that allows us to establish an SSH connection through the bastion.

Hope that wasn't too rambly. In theory I could probably make the bastion module's output format the CIDR blocks from the aws_security_group_rule, but having whole module dependencies would be far nicer.

[roboll]

+1

I've got a situation where one module uploads a number of artifacts to s3, while another module starts up instances that expect the artifacts to exist on s3 and pull them down during boot. As of now, with two discrete modules, there doesn't appear to be a way to tell terraform not to start the instance module until the s3 module has completed its uploads.

[sherabi]

+1

[ketzacoatl]

@roboll, you might benefit from the -target attr for the apply and plan subcommands.

[derekdowling]

Yup. Trying to create an IAM Instance Profile for a machine, and then trying to create my aws instance with a custom terraform puppet aws instance module we've created. Can't attach an IAM Instance Profile to it because it depends_on the IAM Instance Profile being created in the root file. Since there is no way to specify depends_on via an input var for the module I am stuck.

[Binternet]

+1

[foxlollato]

Hello! Im using this scrpit to run plan and apply in the right sequence.
first, mkdir deploy-history
To run use:
$ ./script.sh module1 module2 module3 module4
where moduleN is the nodule name in main.tf. remember to put modules in the right sequence of creation based on your dependencies.

#!/bin/bash
for mod in $@
do
terraform plan -target=module.$mod -out deploy-history/$mod.out && terraform apply deploy-history/$mod.out
if [ "$?" -ne "0" ]
then
echo "[ERROR] Module: ${mod^^}"
exit 1
fi

done

[rkt2spc]

I solved my problems using depends_on in the output block

I tried to create some aws_subnet which depends on a aws_vpc & some aws_vpc_ipv4_cidr_block_association

The problem was I grouped the aws_vpc and aws_vpc_ipv4_cidr_block_association under the same module. While the aws_subnet only directly depends on the aws_vpc it still needs the aws_vpc_ipv4_cidr_block_association to be properly initialized.

So I marked the vpc_id output to wait for the initialization of aws_vpc_ipv4_cidr_block_association and everything worked out correctly

output "vpc_id" {
  depends_on  = ["aws_vpc_ipv4_cidr_block_association.secondary_cidrs"]
  description = "ID of the VPC"
  value       = "${aws_vpc.vpc.id}"
}

[sudoapt-getclean]

Honestly I don't see a problem with current implementation. Modules don't depend on each other, resources inside them do.

This is exactly correct.
The trick you can do to create a dependency between two resources is to simply add some output of the resource you need to depend on as input to the thing that has a dependency (almost always this happens automatically since this is how the DAG is build in the first place)

You can put this value in a tag or some harmless place our you can mess around with it to only make Terraform think it is a dependency:

locals {
  dependent_name = "${var.client_name},${var.dependency}"
  name = "${element(split(",", local.dependent_name), 0 )}"
}
variable "dependency" {}

and now use
name = "${local.name}"
in the resource that you want to depend on something

[charles-salmon]

To work around this, I've done the following:

• Added the following variable to each of my modules:

variable "dependencies" {
  type = "list"
}

• Added the following resource.null_resource to the beginning of each of my modules:

resource "null_resource" "dependency_getter" {
  provisioner "local-exec" {
    command = "echo ${length(var.dependencies)}"
  }
}

• Added the following depends_on attribute to all resource(s) that will be constructed first within the module:

depends_on = [
  "null_resource.dependency_getter",
]

• Added the following resource.null_resource to the end of each of my modules:

resource "null_resource" "dependency_setter" {
  depends_on = [
    # List resource(s) that will be constructed last within the module.
  ]
}

• Added the following output to each of my modules:

output "depended_on" {
  value = "${null_resource.dependency_setter.id}"
}

This allows me to use a module in the following manner:

module "devops_cluster" {
  source           = "./Infrastructure/devops-gke-cluster"
  gcp_project_name = "${var.gcp_project_name}"
  gcp_zone         = "${var.gcp_zone}"

  providers = {
    kubernetes = "kubernetes.devops"
    helm       = "helm.devops"
  }

  dependencies = [
    "${module.devops_dns_zone.depended_on}",
  ]
}

This also makes it possible to specify dependencies on multiple modules.

[hamzawix]

+1 we really need this feature

[mbrowne]

Seems like this issue should be reopened, since only one half of it is now supported? There's also #10462 (which is still open) but it was created in 2016...is that the right place to watch for updates on this issue?

[mkozjak]

+1. Really need this feature, as I already hit the problem a few times.

[jobygoude]

+1

[hugolesta]

+1 This feature will be awesome

[discordianfish]

Can you please re-open this issue? I think this was closed by accident. Only half of this was implemented. @phinze @mitchellh

Everyone +1ing this: Please stop! +1 the issue on top, don't spam with comments. Thanks!

[atrull]

Yeah we need this.

[yves-vogl]

We needs this, too.

Think of this workflow:
EKS -> RDS -> Application Deployment with Helm -> API Gateway -> Cloudfront

Without those dependency you end up e.g. data.kubernetes_service failing with "connection refused" as the EKS does not exist before initial rollout.

[crysyn]

I am defining ECS task definitions and services as a module and would like them to wait on the ECS and/or load balancer (since terraform does not seem to do this on its own). So I would very much like the other half of this to be reopened and completed.

[sudomateo]

@charles-salmon Great workaround! Thank you so much for sharing!

For my personal use case, I modified the approach a bit to ensure the dependencies are enforced on all Terraform runs and not just the initial one.

I changed the output to be.

output "depended_on" {
  value = "${null_resource.dependency_setter.id}-${timestamp()}"
}

I used the timestamp() function here to ensure this will force a change each run. This helps when resources in the depended on module change and we want to ensure the dependent module will respect those changes.

I also changed the null_resource.dependency_getter to be.

resource "null_resource" "dependency_getter" {
  triggers = {
    my_dependencies = "${join(",", var.dependencies)}"
  }
}

This uses the triggers sub-block to ensure it runs on all runs and not just on creation like the provisioner block.

[TylerWanner]

Still can't add depends_on to modules from terraform module registry

[sjmiller609]

@charles-salmon Great workaround! Thank you so much for sharing!

For my personal use case, I modified the approach a bit to ensure the dependencies are enforced on all Terraform runs and not just the initial one.

I changed the output to be.

output "depended_on" {
  value = "${null_resource.dependency_setter.id}-${timestamp()}"
}

I used the timestamp() function here to ensure this will force a change each run. This helps when resources in the depended on module change and we want to ensure the dependent module will respect those changes.

I also changed the null_resource.dependency_getter to be.

resource "null_resource" "dependency_getter" {
  triggers {
    my_dependencies = "${join(",", var.dependencies)}"
  }
}

This uses the triggers sub-block to ensure it runs on all runs and not just on creation like the provisioner block.

an equals sign after the "triggers" did the trick for me in TF 0.12

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.