AKS Get-Credentials doesn't handle KUBECONFIG with multiple files

[paddycarey]

Describe the bug

When a user's KUBECONFIG environment variable contains more than one file, then az aks get-credentials... mistakenly treats the value as a single path.

As per the official documentation this colon-delimited format is explicitly supported on Mac and Linux.

This behaviour appears to have been introduced by #18704 and is present in v2.26.0 and above.

e.g.

If my KUBECONFIG looks like so:

KUBECONFIG=/home/me/.kube/config:/home/me/.kube/anotherconfig:/home/me/.kube/yetanotherconfig

Then when I run az aks get-credentials... then it will attempt to write the file to a file at the path /home/me/.kube/config:/home/me/.kube/anotherconfig:/home/me/.kube/yetanotherconfig rather than one of the single files contained within.

To Reproduce

$ KUBECONFIG=/home/me/.kube/config:/home/me/.kube/anotherconfig
$ az aks get-credentials ...
$ cat "/home/me/.kube/config:/home/me/.kube/anotherconfig"

Expected behavior

az aks get-credentials... should write AKS credentials to one of the files contained within KUBECONFIG only, but not sure how it should choose which to use.

Environment summary

Install method: homebrew
Mac Version: 11.4

$ az --version
azure-cli                         2.26.0

core                              2.26.0
telemetry                          1.0.6

Python location '/usr/local/Cellar/azure-cli/2.26.0/libexec/bin/python'
Extensions directory '/Users/paddy/.azure/cliextensions'

Python (Darwin) 3.8.11 (default, Jun 29 2021, 03:08:07)
[Clang 12.0.5 (clang-1205.0.22.9)]

Additional Context

N/A

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @Azure/aks-pm.

Issue Details

---

Describe the bug

When a user's KUBECONFIG environment variable contains more than one file, then az aks get-credentials... mistakenly treats the value as a single path.

As per the official documentation this colon-delimited format is explicitly supported on Mac and Linux.

This behaviour appears to have been introduced by #18704 and is present in v2.26.0 and above.

e.g.

If my KUBECONFIG looks like so:

KUBECONFIG=/home/me/.kube/config:/home/me/.kube/anotherconfig:/home/me/.kube/yetanotherconfig

Then when I run az aks get-credentials... then it will attempt to write the file to a file at the path /home/me/.kube/config:/home/me/.kube/anotherconfig:/home/me/.kube/yetanotherconfig rather than one of the single files contained within.

To Reproduce

$ KUBECONFIG=/home/me/.kube/config:/home/me/.kube/anotherconfig
$ az aks get-credentials ...
$ cat "/home/me/.kube/config:/home/me/.kube/anotherconfig"

Expected behavior

az aks get-credentials... should write AKS credentials to one of the files contained within KUBECONFIG only, but not sure how it should choose which to use.

Environment summary

Install method: homebrew
Mac Version: 11.4

$ az --version
azure-cli                         2.26.0

core                              2.26.0
telemetry                          1.0.6

Python location '/usr/local/Cellar/azure-cli/2.26.0/libexec/bin/python'
Extensions directory '/Users/paddy/.azure/cliextensions'

Python (Darwin) 3.8.11 (default, Jun 29 2021, 03:08:07)
[Clang 12.0.5 (clang-1205.0.22.9)]

Additional Context

N/A

Author:	paddycarey
Assignees:	-
Labels:	AKS, Service Attention
Milestone:	-

[yonzhan]

route to service team

[Volatus]

@paddycarey The official docs provide no guidance on how multiple files have to be handled. What's the point of having multiple files on KUBECONFIG anyway?

[Flasheh]

@Volatus The docs state

The KUBECONFIG environment variable holds a list of kubeconfig files. For Linux and Mac, the list is colon-delimited. For Windows, the list is semicolon-delimited.

When working with a bunch of different clusters on multiple projects for example, some people prefer to manage them in different physical config files.

[Volatus]

@Volatus The docs state

The KUBECONFIG environment variable holds a list of kubeconfig files. For Linux and Mac, the list is colon-delimited. For Windows, the list is semicolon-delimited.

When working with a bunch of different clusters on multiple projects for example, some people prefer to manage them in different physical config files.

I see, I can work on fixing this, it's just that I'm not sure what behavior it should have in terms of deciding which file to use within the $KUBECONFIG.

[Flasheh]

Yeah I see the issue now. There's no real solution without making certain assumptions.

What bothers me is that it now writes to the literal value like /home/me/.kube/config:/home/me/.kube/anotherconfig
I wouldn't mind it defaulting to ~/.kube/config with multiple files in $KUBECONFIG but that's somewhat confusing behaviour.

[Volatus]

Yeah I see the issue now. There's no real solution without making certain assumptions.

What bothers me is that it now writes to the literal value like /home/me/.kube/config:/home/me/.kube/anotherconfig
I wouldn't mind it defaulting to ~/.kube/config with multiple files in $KUBECONFIG but that's somewhat confusing behaviour.

The reason I introduced the change for checking $KUBECONFIG was due to the fact that previous, azure cli would not write to the appropriate file if you put KUBECONFIG outside of the default location in $HOME/.kube/. Unfortunately this change broke the functionality with having multiple files but I can write up a fix once it's decided on what behavior to use.

[paddycarey]

Unsure what other tools do, but I noticed that the aws CLI defaults to using the first in the list if set:

By default, the configuration is written to the first file path in the KUBECONFIG environment variable (if it is set) or the default kubeconfig path (.kube/config) in your home directory.

Perhaps similar behaviour would work well here too?