[0.11 backport] k8s: fix missing kubeconfig check from endpoint

[crazy-max]

• backport of k8s: fix missing kubeconfig check from endpoint #1941

[jedevc]

Looks like github might be having some issues? wget golangci-lint step is failing.

[crazy-max]

Looks like github might be having some issues? wget golangci-lint step is failing.

Yes we could use their image: https://github.com/crazy-max/diun/blob/fe0622ecac59190d55eeea5f6a88f05a885300a3/hack/lint.Dockerfile#L11-L16

[gitfxx]

@crazy-max In this code, there seems to be a potential issue where multiple kubeconfig settings may override each other. For example, if I have multiple kubeconfig files, they might overwrite each other's configurations

KUBECONFIG=/k8s-config-x86 buildx create \
--bootstrap \
  --use \
  --bootstrap \
  --name=builder-test-${random} \
  --driver=kubernetes \
  --platform=linux/amd64 \
  --node=builder-test-amd64-${random} \
  --driver-opt=namespace=buildx-builder,image=buildkit:0.9.0-c,nodeselector="pipeline=buildx-builder"
KUBECONFIG=/k8s-config-arm buildx create \
  --bootstrap \
  --append \
  --name=builder-test-${random} \
  --driver=kubernetes \
  --platform=linux/arm64 \
  --node=builder-test-arm64-${random} \
  --driver-opt=namespace=buildx-builder,image=buildkit:0.9.0-c-arm64,nodeselector="buildx-builder=arm64"

After reviewing the code snippet, I attempted to reproduce the issue. When multiple kubeconfig files are present and being read with the same environment variable, KUBECONFIG, there arises a problem of configurations being overwritten.

func ConfigFromEndpoint(endpointName string, s store.Reader) (clientcmd.ClientConfig, error) {
	if strings.HasPrefix(endpointName, "kubernetes://") {
		u, _ := url.Parse(endpointName)
		if kubeconfig := u.Query().Get("kubeconfig"); kubeconfig != "" {
			_ = os.Setenv(clientcmd.RecommendedConfigPathEnvVar, kubeconfig)
		}
		rules := clientcmd.NewDefaultClientConfigLoadingRules()
		apiConfig, err := rules.Load()
		if err != nil {
			return nil, err
		}
		return clientcmd.NewDefaultClientConfig(*apiConfig, &clientcmd.ConfigOverrides{}), nil
	}
	return ConfigFromContext(endpointName, s)
}