BLB leak after create service

[hello2mao]

BLB leak like below:

[hello2mao]

Create BLB success, but query failed.

Create requestId: d3379fb1-6321-40ed-932c-3e5cf316556d
Query requestId: c4533409-dff5-493c-8386-507d13b5bbba

Error log:

Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.644399   41893 log.go:172] http request: http://100.64.253.28:8693/internal-api/v1/blb?clien
tToken=imkJ3e3i9ysC_BJ4OoxzoyhhmCH51Qzk  do use time: 4.001049092s
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.645554   41893 log.go:172] Response: status code = 200, httpMethod = POST, requestUrl = http
://100.64.253.28:8693/internal-api/v1/blb?clientToken=imkJ3e3i9ysC_BJ4OoxzoyhhmCH51Qzk
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.645626   41893 log.go:172] Response Header:  = map[X-Bce-Request-Id:[d3379fb1-6321-40ed-932c
-3e5cf316556d] Cache-Control:[no-cache] Content-Type:[application/json;charset=UTF-8] Date:[Sun, 10 Feb 2019 07:01:19 GMT] Server:[BWS]]
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.645646   41893 log.go:172] Response body:  = {"address":"192.168.0.182","name":"c-h1HnbAkh/d
efault/deployment-ifr-3mjqe1a3gfrkbmhd","blbId":"lb-b29149d6","desc":null}
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.645745   41893 load_balancer.go:115] EnsureLoadBalancer create blb success, BLB name: c-h1Hn
bAkh/default/deployment-ifr-3mjqe1a3gfrkbmhd, BLB id: lb-b29149d6, BLB address: 192.168.0.182.
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.646022   41893 log.go:172] Request: httpMethod = GET, requestUrl = http://100.64.253.28:8693
/internal-api/v1/blb?address=&bccId=&blbId=lb-b29149d6&name=, requestHeader = map[Authorization:[bce-auth-v1/8e2fdc833cf44b4895afd0bce14f43cf/2019-02-10T07:01:19Z/1800/content-type;
host;x-bce-date/e3dec624a204f2027f00c21a871d329c9e4d25869f7ef26c98bf10c9e7dc6313] User-Agent:[cce-k8s:c-h1HnbAkh] Content-Type:[application/json] X-Bce-Date:[2019-02-10T07:01:19Z] H
ost:[100.64.253.28:8693]]
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.892516   41893 log.go:172] http request: http://100.64.253.28:8693/internal-api/v1/blb?addre
ss=&bccId=&blbId=lb-b29149d6&name=  do use time: 246.420663ms
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.893414   41893 log.go:172] Response: status code = 200, httpMethod = GET, requestUrl = http:
//100.64.253.28:8693/internal-api/v1/blb?address=&bccId=&blbId=lb-b29149d6&name=
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.893504   41893 log.go:172] Response Header:  = map[X-Bce-Request-Id:[c4533409-dff5-493c-8386
-507d13b5bbba] Cache-Control:[no-cache] Content-Type:[application/json;charset=UTF-8] Date:[Sun, 10 Feb 2019 07:01:19 GMT] Server:[BWS]]
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.893521   41893 log.go:172] Response body:  = {"blbList":[],"marker":null,"nextMarker":null,"
maxKeys":1000,"truncated":false}
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.893596   41893 load_balancer.go:124] EnsureLoadBalancer create blb failed: len(lbs) != 1
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.893700   41893 service_controller.go:716] Finished syncing service "default/deployment-ifr-3
mjqe1a3gfrkbmhd" (4.806625322s)
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: E0210 15:01:19.893744   41893 service_controller.go:219] error processing service default/deployment-ifr-3m
jqe1a3gfrkbmhd (will retry): failed to ensure load balancer for service default/deployment-ifr-3mjqe1a3gfrkbmhd: EnsureLoadBalancer create blb failed: len(lbs) != 1
Feb 10 15:01:19 instance-vqso233t-1 kube-cloud-controller-manager[41893]: I0210 15:01:19.893828   41893 event.go:221] Event(v1.ObjectReference{Kind:"Service", Namespace:"default", Name:"deployment-ifr-3mjqe1a3gfrkbmhd", UID:"a82578a0-2d01-11e9-9eb1-fa163e65ca09", APIVersion:"v1", ResourceVersion:"3199330", FieldPath:""}): type: 'Warning' reason: 'CreatingLoadBalancerFailed' Error creating load balancer (will retry): failed to ensure load balancer for service default/deployment-ifr-3mjqe1a3gfrkbmhd: EnsureLoadBalancer create blb failed: len(lbs) != 1

[hello2mao]

[hello2mao]

解决方案：CCM增加重试逻辑，代码：f7660f5

[hello2mao]

Fix
/close