Migrating Helm chart template logic to server side

[tennix]

Feature Request

Currently, many resources are created and managed solely by the Helm chart. Embedding logic in the chart template makes it hard to add some features, such as the unit computation and conversion between PD/TiKV's and K8s's. It also makes it difficult to write automatic tests, validation logic and backward compatible code. Besides, users have to use both kubectl and helm to manage tidb clusters, this adds extra complexity.

Now we have introduced aggregated apiserver. We should migrate Helm chart template logic to the controller-manager or aggregated apiserver.

Below are lists of resources we should migrate to server-side:

• tidb-discovery service sync discovery service in controller #1322
• tidb-initializer job New custom resource TidbInitializer #1262
• tidb service Sync TiDB service in control-loop #1190 Sync TiDB service in control-loop #1242
• tidb configmaps and start-script Sync TiDB Configuration and start-script in control-loop #1192
• pd configmaps and start-script sync pd configmap and start script in controller #1263 sync pd and tikv configmap in controller #1330
• tikv configmaps and start-script sync tikv configmap and start script in controller #1264 sync pd and tikv configmap in controller #1330
• pump Sync Pump in controllers #1191
• monitor New custom resource TidbMonitor #1261

ref: server dry-run #999

[gregwebs]

This makes it harder for users to hack changes (Now they have to compile tidb-operator) or to even understand ahead of time what would happen. I am wondering if we can have a dry run feature to emit all the resources that tidb-operator would create. I guess this is similar to another feature request to have a plan feature like terraform, but that may be more focused on update?

[tennix]

Since we will have the aggregated apiserver, before the CRs are written into k8s, we can emit a plan (server-side dry run). I guess this can solve your worries.
But yes, to change the spec or resources we don't expose requires changing Go code can re-compile tidb-operator. We lose some flexibility but gain more correctness. Just as dynamic programming languages vs. static programming languages.

[gregwebs]

yes, server side dry-run (create and update) would help.

[aylei]

@tennix @gregwebs I reference the dry-run issue in the description, however, one thing worries me is that the PD/TiKV/TiDB are still created with ordered dependencies in our controller. I remember that you have a proposal to create them simultaneously, how is it going then? @gregwebs

[gregwebs]

how is it going then? @gregwebs
We just need to finalize testing on the PR now.

[cofyc]

should we remove dependencies on these annotations keys?

tidb-operator/charts/tidb-cluster/templates/tidb-cluster.yaml

Lines 12 to 15 in b918463

	app.kubernetes.io/name: {{ template "chart.name" . }}
	app.kubernetes.io/managed-by: {{ .Release.Service }}
	app.kubernetes.io/instance: {{ .Release.Name }}
	app.kubernetes.io/component: tidb-cluster

lots of code use these keys.

[aylei]

Yes, we should, the controller should not rely on certain labels

[aylei]

tracked in #1418

[aylei]

closed