Move "prometheus-federation" to use template configuration

[stephen-soltesz]

This change update the prometheus-federation configuration to use templates.

This change removes the per-project prometheus-federation directories in favor of a single template directory. This change also adds three project template variable files.

---

This change is 

[nkinkade]

A couple small questions.

---

Reviewed 60 of 62 files at r1.
Review status: 45 of 46 files reviewed at latest revision, 2 unresolved discussions.

---

k8s/prometheus-federation/deployments/prometheus.yml, line 59 at r1 (raw file):

               "-storage.local.retention=2880h",
               "-alertmanager.url=http://alertmanager-public-service.default.svc.cluster.local:9093",
               "-web.external-url=http://status-{{GCLOUD_PROJECT}}.measurementlab.net:9090",

This might be a good time to go ahead and use the newer, GCD domains, no (e.g., status.stating.measurementlab.net)? Though I imagine it would require defining another variable in the per-project configs, or some way of variable manipulation like Ansible has with Jinja2.

---

k8s/prometheus-federation/deployments/prometheus.yml, line 66 at r1 (raw file):

        resources:
          requests:
            memory: "{{PROMETHEUS_RAM}}"

Question: The kexpand documentation indicates that {{<var>}} is an legacy format that is not recommended. They seem to recommend $(var) or $((var)). What is the rationale for using this "legacy" syntax?

---

Comments from Reviewable

[stephen-soltesz]

Review status: 45 of 46 files reviewed at latest revision, 2 unresolved discussions.

---

k8s/prometheus-federation/deployments/prometheus.yml, line 59 at r1 (raw file):

Previously, nkinkade wrote…

This might be a good time to go ahead and use the newer, GCD domains, no (e.g., status.stating.measurementlab.net)? Though I imagine it would require defining another variable in the per-project configs, or some way of variable manipulation like Ansible has with Jinja2.

I would like to defer this to a future PR.

---

k8s/prometheus-federation/deployments/prometheus.yml, line 66 at r1 (raw file):

Previously, nkinkade wrote…

Question: The kexpand documentation indicates that {{<var>}} is an legacy format that is not recommended. They seem to recommend $(var) or $((var)). What is the rationale for using this "legacy" syntax?

kexpand is supposed to implement a proposed template format that was never formally adopted by the kubernetes project.

scraper and scraper-sync use the double {{}} format as does mlabconfig.py.

Since there is no official standard outside of kexpand, I would would prefer to normalize on a single format. And, I also happen to like the double {{}} better. :)

Also, using the double {{}} format in kubernetes files helps distinguish between variables that we expected to be template expanded vs variables that we expect to be expanded by kubernetes from environment variables like $(GITHUB_AUTH_TOKEN).

---

Comments from Reviewable

[nkinkade]

---

Reviewed 2 of 62 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved, some commit checks failed.

---

Comments from Reviewable