Add environment name in ControllerConfig

[wbuchwalter]

It would be useful to add the environment name (gke, azure etc.) in the ControllerConfig.

For example, in the dashboard we want to be able to specify the resources.limits for master/worker.
However for GPUs, GKE is using the device plugin, so nvidia.com/gpu while Azure is using alpha.kubernetes.io/nvidia-gpu for example.
The dashboard could first read the environment name in the ControllerConfig first and choose the appropriate resource based on this.

@jlewi thoughts?

[jimexist]

@wbuchwalter did you mean #451 ?

[wbuchwalter]

@jimexist Yes :) Although, not sure if the GetGPUResourceName should be where you put it.
Unless it has other possible use someplace else, I was thinking doing that it the front-end, because there might be a bunch of different customization we might want to do in the dashboard based on the cloud.

[jlewi]

Would it be better to use the ksonnet template as opposed to the controller config to handle GPU config?

In the way we have Cloud specific parts for TFServing
https://github.com/kubeflow/kubeflow/blob/master/kubeflow/tf-serving/tf-serving.libsonnet

We could have cloud specific bits for TFJob and we could use this to specify resources like the GPU resource name conditioned on the cloud.

Using the operator to specify GPU config predates our use of ksonnet. Now that we are using ksonnet it might make more sense to use ksonnet to define suitable templates for different clouds?

[jlewi]

@wbuchwalter Any update on this? Should the different resource names just be handled in the UI?

E.g. in the UI we could let users specify alpha.kubernetes.io/nvidia-gpu or nvidia.com/gpu? They could either specify that directly or they could specify cloud and then the UI would infer the values based on this.

What priority should we assign this issue? If it must get done and will get done for our 0.2 release (end of June) then it should be P1 otherwise P2.

[wbuchwalter]

Haven't worked on this yet, but let's mark it as P1.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.