Kubernetes task does not fail on failure while using versionSpec

[tbuico-premera]

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: Kubernetes

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:
    • account name: [email protected]
    • team project name: Premera
    • build definition name/build number: (unsure how to locate)

• Agent - Private:

  • If using private agent, provide the OS of the machine running the agent and the agent version:
    • OS: RHEL 7.7
    • Agent Version: 2.175.2
    • Container: alpine:3

Issue Description

This issue is directly linked to #14283

The following task in an ado pipeline is used in order to log into a kubernetes cluster:

- task: Kubernetes@1
  displayName: 'login to k8s'
  inputs:
    connectionType: Kubernetes Service Connection
    kubernetesServiceEndpoint: 'kubernetesServiceEndpoint.com'
    command: login
    versionOrLocation: version
    versionSpec: v1

This task works as long as it can fetch any Kubectl version from the server. It does not seem to attempt to match with the versionSpec parameter
If there is a timeout or another failure for any reason, which causes this task to not fetch a Kubectl version from the server, this task does not fail, exit, or retry.
If the Kubectl version fetched does not match the versionSpec, then this task also does not fail which seems to contradict the documentation. (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes?view=azure-devops#advanced)

This results to any future tasks in the pipeline which executes code on the k8s cluster to fail and causes a lot of pain while troubleshooting why a pipeline failed.

Here is an example of a failure:

2021-02-02T18:03:54.7526089Z ==============================================================================
2021-02-02T18:03:54.7527131Z 			Kubectl Client Version: v1.20.2
2021-02-02T18:03:54.7527682Z 	Kubectl Server Version: Could not find kubectl server version
2021-02-02T18:03:54.7528898Z ==============================================================================
2021-02-02T18:03:54.7552178Z ##[debug]Could not fetch Kubectl version. Please make sure that the Kubernetes server is up and running.
2021-02-02T18:03:54.7553100Z ##[debug]secretName=undefined
2021-02-02T18:03:54.7553892Z ##[debug]configMapName=undefined
2021-02-02T18:03:54.7555211Z ##[debug]SYSTEM_JOBID=4228d824-2dfc-5e83-4d13-fb664a495af1
2021-02-02T18:03:54.8305597Z ##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=KubernetesV1]{"registryType":"Azure Container Registry","command":"login","jobId":"4228d824-2dfc-5e83-4d13-fb664a495af1"}
2021-02-02T18:03:54.8307717Z ##[debug]set KUBECONFIG=/__w/_temp/kubectlTask/1612289014602/config
2021-02-02T18:03:54.8314773Z ##[debug]Processed: ##vso[task.setvariable variable=KUBECONFIG;issecret=false;]/__w/_temp/kubectlTask/1612289014602/config

Retries of the same pipeline without any changes will often work.

Here is an example of a success on Attempt #2, without any changes to code or pipeline.

2021-02-02T18:23:13.9258719Z ==============================================================================
2021-02-02T18:23:13.9260205Z 			Kubectl Client Version: v1.18.12
2021-02-02T18:23:13.9261122Z 			Kubectl Server Version: v1.18.12+vmware.2
2021-02-02T18:23:13.9262008Z ==============================================================================
2021-02-02T18:23:13.9263623Z ##[debug]secretName=undefined
2021-02-02T18:23:13.9264850Z ##[debug]configMapName=undefined
2021-02-02T18:23:13.9290638Z ##[debug]SYSTEM_JOBID=6296ab86-9cad-5f12-83c2-560921aa1aab
2021-02-02T18:23:13.9850833Z ##[debug]Processed: ##vso[telemetry.publish area=TaskEndpointId;feature=KubernetesV1]{"registryType":"Azure Container Registry","command":"login","jobId":"6296ab86-9cad-5f12-83c2-560921aa1aab"}
2021-02-02T18:23:13.9853279Z ##[debug]set KUBECONFIG=/__w/_temp/kubectlTask/1612290193705/config
2021-02-02T18:23:13.9860223Z ##[debug]Processed: ##vso[task.setvariable variable=KUBECONFIG;issecret=false;]/__w/_temp/kubectlTask/1612290193705/config

One can notice that now the Kubectl Server Version is valid on Attempt #2

Task logs

Attached as a zip file
4228d824-2dfc-5e83-4d13-fb664a495af1.zip

[ajinkya599]

versionSpec is to specify the kubectl client version that should be used by the task to communicate with the cluster. It doesn't intend to resolve what's running in the cluster.

In the above case it seems that the server is some times not available while checking for version. It is a conscious call to not block the task execution in such a case as the version logging is intended for debugging purposes only and the subsequent calls might actually succeed.

If the Kubectl version fetched does not match the versionSpec, then this task also does not fail which seems to contradict the documentation. (https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/kubernetes?view=azure-devops#advanced)

@tbuico-premera - If you could share specifically which part this contradicts with, we can fix the documentation to be more clear.

cc: @shigupt202, @DS-MS

[shigupt202]

@tbuico-premera The version in the logs are just for information and, as @ajinkya599 mentioned, are not supposed to interfere with the execution of task and it's results. It's an expected behaviour for the task to not be able to log server version for the first time since Kubeconfig is not present on the agent at this point. Once the login action is successful and the Kubeconfig is present on the agent, it'll be able to fetch server version and log it in subsequent runs of the task.