SetupCluster failures unexpected keyword argument 'client_configuration'

[jlewi]

I updated the docker images for Airflow because I'm trying to fix some issues and setup cluster started failing with the error

[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask: Traceback (most recent call last):
[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask:     "__main__", fname, loader, pkg_name)
[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask:     exec code in run_globals
[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask:   File "/var/lib/data/runs/tf_k8s_tests/2017-12-22T17_30_16/tensorflow_k8s/py/deploy.py", line 210, in <module>
[2017-12-22 17:35:02,353] {base_task_runner.py:98} INFO - Subtask:     main()
[2017-12-22 17:35:02,354] {base_task_runner.py:98} INFO - Subtask:   File "/var/lib/data/runs/tf_k8s_tests/2017-12-22T17_30_16/tensorflow_k8s/py/deploy.py", line 207, in main
[2017-12-22 17:35:02,354] {base_task_runner.py:98} INFO - Subtask:     args.func(args)
[2017-12-22 17:35:02,354] {base_task_runner.py:98} INFO - Subtask:   File "/var/lib/data/runs/tf_k8s_tests/2017-12-22T17_30_16/tensorflow_k8s/py/deploy.py", line 67, in setup
[2017-12-22 17:35:02,354] {base_task_runner.py:98} INFO - Subtask:     util.load_kube_config()
[2017-12-22 17:35:02,354] {base_task_runner.py:98} INFO - Subtask:   File "/var/lib/data/runs/tf_k8s_tests/2017-12-22T17_30_16/tensorflow_k8s/py/util.py", line 448, in load_kube_config
[2017-12-22 17:35:02,354] {base_task_runner.py:98} INFO - Subtask:     **kwargs).load_and_set()
[2017-12-22 17:35:02,355] {base_task_runner.py:98} INFO - Subtask:   File "/usr/local/lib/python2.7/site-packages/kubernetes/config/kube_config.py", line 319, in _get_kube_config_loader_for_yaml_file
[2017-12-22 17:35:02,355] {base_task_runner.py:98} INFO - Subtask:     **kwargs)
[2017-12-22 17:35:02,355] {base_task_runner.py:98} INFO - Subtask: TypeError: __init__() got an unexpected keyword argument 'client_configuration'

[jlewi]

Kubernetes py package 4.0.0 was released on 11-20.
https://pypi.python.org/pypi/kubernetes/4.0.0

I suspect even though we had rebuilt the container since then we hadn't rebuilt the layer container the Kubernetes client so we were still using 3.0.0. So I suspect something changed.

I think util.setup_cluster actually relies on a private function, kube_config._get_kube_config_loader_for_yaml_file, to work around kubernetes-client/python#339 so its not unexpected that this might have broken with a new release.

[jlewi]

Looks like Configuration might have changed
https://github.com/kubernetes-incubator/client-python/blob/master/CHANGELOG.md#v400a1

It looks like KubeConfigLoader is actually sourced from a different repo via a submodule.
Here's the current version
https://github.com/kubernetes-client/python-base/blob/master/config/kube_config.py
client_configuration is not an option.

It looks like we might just need to call loader.load_and_set(client_configuration) instead of passing it to the constructor.

Our py/requirements.txt already specifies that 4.0.0 is required.