Implement save_config

[chiradeep]

As currently implemented, the NetScaler provider does not commit the configuration to the NetScaler's persistent store(CLI: save_config). Reason:

1. TF parallelizes where it can and if each resource declaration tried to do a save_config then they could potentially stomp on each other. We would probably have to force serial execution.
2. save_config takes 4-5 seconds. Currently, creating 12-15 resources using TF takes about 1 second. If each resource automatically saved the config then that would increase to 60-80 seconds.

One idea is to have a netscaler_save_config resource that could be run once. But it needs to run at the very end of the config run, so potentially the config author has to make it depend on every resource.
If and when TF implements depends_on support for modules (see hashicorp/terraform#10462) this could be solved by putting the 'main' config in a module and a second module that only runs netscaler_save_config and depends_on the first module

[sjwl]

depends_on would ensure that netscaler_save_config is run at the end the first time. But what about subsequent terraform apply when a resource changes (e.g. - service group membership change)? I'm not sure if netscaler_save_config would be recreated?

[chiradeep]

Good point. The semantics of the (yet-to-be-implemented) depends_on is not defined. Perhaps this is why there is a delay in implementing this feature despite the clamor. What about writing two modules: module A and module B. Module A has regular NetScaler resources and module B has the netscaler_save_config resource. Then the operator would have to be sure to run
terraform apply -target moduleA && terraform taint moduleB.netscaler_save_config && terraform apply -target moduleB every time.

[sjwl]

we could do that. We use gitlab as our CI pipeline so could ensure the terraform apply -target moduleB happens every time once per run.

[chiradeep]

After a little bit of investigation, this doesn't look promising. The save config is not a real resource in the terraform sense - it cannot be updated / deleted / read. Ideally we would have something like Ansible's notify that runs at the end of a playbook. I have 3 approaches to solve this, 2 involving the null resource and one is a simple shell script.

1. Shell script:

export NS_URL=http://<host>:<port>/nitro/v1/config/nsconfig?action=save
export NS_USER=nsroot
export NS_PASSWORD=nsroot
terraform apply && curl -XPOST -H 'Content-type: application/json' -H "X-NITRO-USER:$NS_USER' -H "X-NITRO-PASS:$NS_PASSWORD" $NS_URL/nsconfig?action=save -d '{"nsconfig": {}}' 

2. Null Resource with remote exec (works with NetScaler CPX, for VPX/MPX, do not need cli_script.sh)

variable "NS_IP" {}
variable "NS_SSH_PORT" { default = "22"}
variable "NS_USER" { default = "nsroot"}
variable "NS_PASSWORD" { default = "nsroot"}

resource "null_resource" "saveconfig" {
  # Always run, or else make this depend on ALL the resources
  triggers {
    always = "${uuid()}"
  }

  connection {
    type = "ssh"
    host = "${var.NS_IP}"
    port = "${var.NS_PORT}"
    user = "${var.NS_USER}"
    password = "${var.NS_PASSWORD}"
  }

  provisioner "remote-exec" {
    inline = [
      "/var/netscaler/bins/cli_script.sh 'save config'",
    ]
  }
}

3. Null resource with local-exec

variable "NS_URL" {}
variable "NS_USER" { default = "nsroot"}
variable "NS_PASSWORD" { default = "nsroot"}

resource "null_resource" "saveconfig" {
  triggers {
    always = "${uuid()}"
  }

  provisioner "local-exec" {
    command = "curl -s -XPOST -H 'Content-type: application/json' -H 'X-NITRO-USER:${var.NS_USER}' -H 'X-NITRO-PASS:${var.NS_PASSWORD}' ${var.NS_URL}/nsconfig?action=save -d '{\"nsconfig\": {}}'"
  }
}

[chiradeep]

See 7e5f788

[cdhunt]

I don't see terraform apply taking 60-80 seconds as a reason to not implement SaveConfig in each resource. I think any other approach would be inconsistent with the behavior of other providers' resources. Our typical terraform configs take 30-60 minutes to apply and adding 2 minutes would not be noticeable. Having to implement config "hacks" or calling an external script would add a fair amount of complexity to our CI/CD pipeline.

[chiradeep]

There are NetScaler users with thousands of objects, so the impact would be quite non-trivial to them.

[cdhunt]

I haven't looked through all of the resources, but dropping the SaveConfig inside of if hasChange limits unnecessary calls. If someone creates 1000 new objects in one config, it will be painfully slow, but incremental updates should be acceptably performant.

[chiradeep]

I think it really depends on the environment. Rather than forcing the potentially poor performance on every user, I want to leave that choice to the user. Any other ideas to allow this flexibility?

[cdhunt]

I think another option would be to create a save_config provisioner. That would cause config bloat but puts the user in control.

You could also create a save_config resource and use explicit dependencies. That also bloats the config but moves the save into a single action so apply would almost always be faster.

But how does terraform handle the save failing? I imagine the tfstate would not match what's on the Netscaler and cause problems. That's why I prefer the explicit save in each resource.

[giorgos-nikolopoulos]

Closing this issue due to inactivity.

The current solution is to use the ns_commit.sh script after applying a new configuration as is detailed in this section of the README file.

We will revisit this if and when terraform implements a mechanism similar to Ansible's notify.