Public ip not always deleted when deleting a service of type loadbalancer

[bramvdklinkenberg]

What happened:
We have an ELB (Kubernetes) in front of our AKS dev cluster. During the day new service can/are deployed to the cluster and are exposed as type Load Balancer which creates a public ip attached to our ELB. This is done automatically when we create a Pull Reauest in Azure DevOps.
Each PR gets deployed in its own namespace.
When a PR is merged to master the PR namespace will be deleted, but we also have a job . running that cleans up the PR namespaces at night.

We have noticed that deleting a namespace, and thus the service resource, on the cluster not always correctly deletes the health probe and lb rules which causes the public IP to not be deleted. The namespace and its service are deleted on AKS, but the pub ip and its rules are not deleted. It sometimes happens when we delete one service at a time and also when deleting multiple services at the same time.
When I look at the activity log I can see some failure but not exactly why some deletions are failing. I think because the ELB can't always process multiple create/update/delete actions at the same time and somehow gets in some sort of frozen status and times out after a while and eventually hasn't deleted the lb rules, health probes and its pub ip.
Anyone else experiencing this issue?

What you expected to happen:
For the public ip to be properly removed when removing a service of type loadbalancer

How to reproduce it (as minimally and precisely as possible):
Create a deployment, expose it as type loadbalancer, delete the service

Environment:

• Kubernetes version (use kubectl version): 1.14.7
• Size of cluster (how many worker nodes are in the cluster?): 30 nodes

[gstolz]

yes, we are having the same problem!

[awilhelmer]

we have got the same problem ...

[github-actions]

Action required from @Azure/aks-pm

[ghost]

Action required from @Azure/aks-pm

[palma21]

@feiskyer would you be able to help?

[feiskyer]

it is because old version of Kubernetes doesn't support service finalizer. Service finalizer is GA since v1.17, please upgrade the cluster to v1.17 to have the fixes for the issue.