add config_clean.sh as a way to clean up MLZ resources

src/clean.sh

@@ -87,41 +87,4 @@ echo "INFO: destroying Terraform using ${mlz_config_file} and ${tfvars_path}..."
 
 # clean up MLZ config resources
 echo "INFO: cleaning up MLZ resources with tag 'DeploymentName=${mlz_env_name}'..."
-
-# Create array of unique subscription IDs. The 'sed' command below search thru the source
-# variables file looking for all lines that do not have a '#' in the line. If a line with
-# a '#' is found, the '#' and ever character after it in the line is ignored. The output
-# of what remains from the sed command is then piped to grep to find the words that match
-# the pattern. These words are what make up the 'mlz_subs' array.
-mlz_sub_pattern="mlz_.*._subid"
-mlz_subs=$(< "${mlz_config_file}" sed 's:#.*$::g' | grep -w "${mlz_sub_pattern}")
-subs=()
-
-for mlz_sub in $mlz_subs
-do
-    mlz_sub_id=$(echo "${mlz_sub#*=}" | tr -d '"')
-    if [[ ! "${subs[*]}" =~ ${mlz_sub_id} ]];then
-        subs+=("${mlz_sub_id}")
-    fi
-done
-
-# delete resource groups where deploymentname is mlz_env_name in each subscription
-for sub in "${subs[@]}";
-do
-  rgs_to_delete=$(az group list --subscription ${sub} --tag DeploymentName="${mlz_env_name}" --query [].name -o tsv)
-  for rg in $rgs_to_delete;
-  do
-    echo "INFO: deleting ${rg}..."
-
-    az group delete \
-      --name "${rg}" \
-      --yes \
-      --only-show-errors \
-      --output none
-  done
-done
-
-echo "INFO: deleting service principal ${mlz_sp_name}..."
-az ad sp delete --id "http://${mlz_sp_name}"
-
-echo "INFO: Complete! Resources for ${mlz_env_name} deleted!"
+. "${this_script_path}/scripts/config/config_clean.sh" "${mlz_config_file}"

src/docs/command-line-deployment.md

@@ -9,9 +9,10 @@
     ```
 
 1. [Quickstart](#Quickstart)
-1. [Configure the Terraform Backend](#Configure-the-Terraform-Backend)
+1. [Setup Mission LZ Resources](#Setup-Mission-LZ-Resources)
 1. [Set Terraform Configuration Variables](#Set-Terraform-Configuration-Variables)
 1. [Deploy Terraform Configuration](#Deploy-Terraform-Configuration)
+1. [Clean up Mission LZ Resources](#Clean-up-Mission-LZ-Resources)
 
 ### Quickstart
 
@@ -92,11 +93,13 @@ src/deploy.sh -s {my_mlz_configuration_subscription_id} \
 
 Need further customization? The rest of this documentation covers in detail how to customize this deployment to your needs.
 
-### Configure the Terraform Backend
+### Setup Mission LZ Resources
 
-The MLZ deployment architecture uses a single Service Principal whose credentials are stored in a central "config" Key Vault. Terraform state storage is distributed into a separate storage account for each tier. When deploying the MLZ architecture, all tiers can be deployed into a single subscription or each tier can be deployed into its own subscription.
+Deployment of MLZ happens through use of a single Service Principal whose credentials are stored in a central "config" Key Vault.
 
-1. Create the `mlz_tf_cfg.var` file using the `mlz_tf_cfg.var.sample` as a template.
+MLZ uses this Service Principal and its credentials from the Key Vault to deploy the resources described in Terraform at `src/core` and stores Terraform state for each component into separate storage accounts.
+
+1. First, create the MLZ Configuration file `mlz_tf_cfg.var` file using the `mlz_tf_cfg.var.sample` as a template.
 
     The information in the `mlz_tf_cfg.var` file, will be used by `mlz_tf_setup.sh` to create and populate a `config.vars` file for each tier and saved inside the deployment folder for each tier (example: \src\core\tier-0\config.vars).
 
@@ -114,7 +117,7 @@ The MLZ deployment architecture uses a single Service Principal whose credential
     mlz_config_location="eastus"
     ```
 
-1. Run `mlz_tf_setup.sh` at [src/scripts/mlz_tf_setup.sh](/src/scripts/mlz_tf_setup.sh) to create:
+1. Then, run `mlz_tf_setup.sh` at [src/scripts/mlz_tf_setup.sh](/src/scripts/mlz_tf_setup.sh) to create:
 
     - A config Resource Group to store the Key Vault
     - Resource Groups for each tier to store the Terraform state Storage Account
@@ -124,10 +127,6 @@ The MLZ deployment architecture uses a single Service Principal whose credential
     - Tier specific Terraform backend config files
 
     ```bash
-    # usage mlz_tf_setup.sh: <mlz_tf_cfg.var path>
-
-    chmod u+x src/scripts/mlz_tf_setup.sh
-
     src/scripts/mlz_tf_setup.sh src/mlz_tf_cfg.var
     ```
 
@@ -197,6 +196,16 @@ src/scripts/init_terraform.sh \
   src/core/tier-1
 ```
 
+### Clean up Mission LZ Resources
+
+After you've deployed your environments with Terraform, it is no longer mandatory to keep Mission LZ Resources like the Service Principal, Key Vault, nor the Terraform state files (though you can re-use these resources and stored Terraform state for updating the deployed environment incrementally using `terraform apply` or destroying them from terraform with `terraform destroy`).
+
+If you no longer have the need for a Service Principal with Contributor rights, the Key Vault that stores this Service Principal's credentials, nor the Terraform state, you can clean up these Mission LZ Resources with the [config_clean.sh](/src/scripts/config/config_clean.sh) script passing in the MLZ Configuration file you created earlier:
+
+```bash
+src/scripts/config/config_clean.sh src/mlz_tf_cfg.var
+```
+
 ### Terraform Providers
 
 The development container definition downloads the required Terraform plugin providers during the container build so that the container can be transported to an air-gapped network for use. The container also sets the `TF_PLUGIN_CACHE_DIR` environment variable, which Terraform uses as the search location for locally installed providers. If you are not using the container to deploy or if the `TF_PLUGIN_CACHE_DIR` environment variable is not set, Terraform will automatically attempt to download the provider from the internet when you execute the `terraform init` command.

src/scripts/config/config_clean.sh

@@ -0,0 +1,82 @@
+#!/bin/bash
+#
+# Copyright (c) Microsoft Corporation.
+# Licensed under the MIT License.
+#
+# shellcheck disable=1090,2154
+#
+# remove mlz configuration resources from an mlz configuration file
+
+set -e
+
+error_log() {
+  echo "${1}" 1>&2;
+}
+
+usage() {
+  echo "config_create.sh: remove mlz configuration resources from an mlz configuration file"
+  error_log "usage: config_clean.sh <mlz config>"
+}
+
+if [[ "$#" -lt 1 ]]; then
+   usage
+   exit 1
+fi
+
+mlz_config_file=$(realpath "${1}")
+this_script_path=$(realpath "${BASH_SOURCE%/*}")
+
+# check for dependencies
+"${this_script_path}/../util/checkforazcli.sh"
+"${this_script_path}/../util/checkforterraform.sh"
+"${this_script_path}/../util/checkforfile.sh" \
+   "${mlz_config_file}" \
+   "The configuration file ${mlz_config_file} is empty or does not exist."
+
+# generate names from config
+. "${mlz_config_file}"
+. "${this_script_path}/generate_names.sh" "${mlz_config_file}"
+
+# Create array of unique subscription IDs. The 'sed' command below search thru the source
+# variables file looking for all lines that do not have a '#' in the line. If a line with
+# a '#' is found, the '#' and ever character after it in the line is ignored. The output
+# of what remains from the sed command is then piped to grep to find the words that match
+# the pattern. These words are what make up the 'mlz_subs' array.
+mlz_sub_pattern="mlz_.*._subid"
+mlz_subs=$(< "${mlz_config_file}" sed 's:#.*$::g' | grep -w "${mlz_sub_pattern}")
+subs=()
+
+for mlz_sub in $mlz_subs
+do
+    mlz_sub_id=$(echo "${mlz_sub#*=}" | tr -d '"')
+    if [[ ! "${subs[*]}" =~ ${mlz_sub_id} ]];then
+        subs+=("${mlz_sub_id}")
+    fi
+done
+
+# delete resource groups where deploymentname is mlz_env_name in each subscription
+for sub in "${subs[@]}";
+do
+  rgs_to_delete=$(az group list --subscription "${sub}" --tag "DeploymentName=${mlz_config_tag}" --query [].name -o tsv)
+  for rg in $rgs_to_delete;
+  do
+    echo "INFO: deleting ${rg}..."
+
+    az group delete \
+      --subscription "${sub}" \
+      --name "${rg}" \
+      --yes \
+      --only-show-errors \
+      --output none
+  done
+done
+
+echo "INFO: deleting service principal ${mlz_sp_name}..."
+az ad sp delete --id "http://${mlz_sp_name}"
+
+echo "INFO: purging key vault ${mlz_kv_name}..."
+az keyvault purge \
+    --name "${mlz_kv_name}" \
+    --subscription "${mlz_config_subid}"
+
+echo "INFO: Complete! Resources for ${mlz_env_name} deleted!"

src/scripts/config/config_create.sh

@@ -52,7 +52,7 @@ if ! $rg_exists &> /dev/null; then
         --subscription "${tf_sub_id}" \
         --location "${mlz_config_location}" \
         --name "${tf_rg_name}" \
-        --tags "DeploymentName=${mlz_env_name}" \
+        --tags "DeploymentName=${mlz_config_tag}" \
         --output none
     echo "INFO: resource group ${tf_rg_name} created!"
 fi

src/scripts/config/generate_names.sh

@@ -9,6 +9,7 @@
 # SC2154: "var is referenced but not assigned". These values come from an external file.
 #
 # Generate MLZ resource names
+# rules from: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules
 
 set -e
 
@@ -35,47 +36,49 @@ tf_name_raw=${3:-notset}
 
 # remove hyphens for resource naming restrictions
 # in the future, do more cleansing
-mlz_sub_id_clean="${mlz_config_subid//-}"
-mlz_env_name_clean="${mlz_env_name//-}"
+mlz_sub_id_clean=$(echo ${mlz_config_subid} | tr -cd '[:alnum:]')
+mlz_env_name_clean=$(echo ${mlz_env_name} | tr -cd '[:alnum:]')
 
 # Universal names
 export container_name="tfstate"
 
 # MLZ naming patterns
-mlz_prefix="mlz-tf"
-mlz_sp_name_full="sp-${mlz_prefix}-${mlz_env_name_clean}"
-mlz_sa_name_full="mlztfsa${mlz_env_name_clean}${mlz_sub_id_clean}"
-mlz_kv_name_full="mlzkv${mlz_env_name_clean}${mlz_sub_id_clean}"
-mlz_acr_name_full="mlzacr${mlz_env_name_clean}${mlz_sub_id_clean}"
-mlz_fe_app_name_full="mlzfeapp${mlz_env_name_clean}${mlz_sub_id_clean}"
-mlz_instance_name_full="mlzfeinstance${mlz_env_name_clean}${mlz_sub_id_clean}"
-mlz_dns_name_full="mlzdep${mlz_env_name_clean}${mlz_sub_id_clean}"
+mlz_prefix="mlz"
+mlz_suffix="${mlz_env_name_clean}${mlz_sub_id_clean}"
+
+mlz_rg_name_full="${mlz_prefix}-config-${mlz_env_name_clean}"
+mlz_sp_name_full="${mlz_prefix}-terraform-sp-${mlz_env_name_clean}"
+mlz_kv_name_full="${mlz_prefix}kv${mlz_suffix}"
+mlz_acr_name_full="${mlz_prefix}acr${mlz_suffix}"
+mlz_fe_app_name_full="${mlz_prefix}-frontend-app-${mlz_env_name_clean}"
+mlz_instance_name_full="${mlz_prefix}feinstance${mlz_suffix}"
+mlz_dns_name_full="${mlz_prefix}dep${mlz_suffix}"
 
 # Name MLZ config resources
-export mlz_rg_name="rg-${mlz_prefix}-${mlz_env_name_clean}"
-export mlz_sp_name="${mlz_sp_name_full}"
+export mlz_config_tag="${mlz_prefix}config${mlz_suffix}"
+export mlz_rg_name="${mlz_rg_name_full:0:63}"
+export mlz_sp_name="${mlz_sp_name_full:0:120}"
 export mlz_sp_kv_name="serviceprincipal-clientid"
 export mlz_sp_kv_password="serviceprincipal-pwd"
 export mlz_login_app_kv_name="login-app-clientid"
 export mlz_login_app_kv_password="login-app-pwd"
-export mlz_sa_name="${mlz_sa_name_full:0:24}" # take the 24 characters of the storage account name
-export mlz_kv_name="${mlz_kv_name_full:0:24}" # take the 24 characters of the key vault name
-export mlz_acr_name="${mlz_acr_name_full:0:24}"
-export mlz_fe_app_name="${mlz_fe_app_name_full:0:24}"
-export mlz_instance_name="${mlz_instance_name_full:0:24}"
-export mlz_dns_name="${mlz_dns_name_full:0:24}"
+export mlz_kv_name="${mlz_kv_name_full:0:24}"
+export mlz_acr_name="${mlz_acr_name_full:0:50}"
+export mlz_fe_app_name="${mlz_fe_app_name_full:0:120}"
+export mlz_instance_name="${mlz_instance_name_full:0:64}"
+export mlz_dns_name="${mlz_dns_name_full:0:60}"
 
 if [[ $tf_name_raw != "notset" ]]; then
   # remove hyphens for resource naming restrictions
   # in the future, do more cleansing
-  tf_sub_id_clean="${tf_sub_id_raw//-}"
-  tf_name="${tf_name_raw//-}"
+  tf_sub_id_clean=$(echo ${tf_sub_id_raw} | tr -cd '[:alnum:]')
+  tf_name=$(echo ${tf_name_raw} | tr -cd '[:alnum:]')
 
   # TF naming patterns
-  tf_prefix="tf-${tf_name}"
-  tf_sa_name_full="tfsa${tf_name}${mlz_env_name_clean}${tf_sub_id_clean}"
+  tf_rg_name_full="${mlz_prefix}-tfstate-${tf_name}-${mlz_env_name_clean}"
+  tf_sa_name_full="tfsa${tf_name}${mlz_suffix}"
 
   # Name TF config resources
-  export tf_rg_name="rg-${tf_prefix}-${mlz_env_name_clean}"
-  export tf_sa_name="${tf_sa_name_full:0:24}" # take the 24 characters of the storage account name
+  export tf_rg_name="${tf_rg_name_full:0:63}"
+  export tf_sa_name="${tf_sa_name_full:0:24}"
 fi

src/scripts/config/mlz_config_create.sh

@@ -165,7 +165,7 @@ if ! $rg_exists &> /dev/null; then
         --subscription "${mlz_config_subid}" \
         --location "${mlz_config_location}" \
         --name "${mlz_rg_name}" \
-        --tags "DeploymentName=${mlz_env_name}" \
+        --tags "DeploymentName=${mlz_config_tag}" \
         --output none
 fi