You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Created a blue/green Rollout with active and preview services.
Deploy the services and Rollout. Wait for Rollout to fully deploy
Delete the services
Redeploy the services. The services will be missing the pod hash label added by Rollout
After step 4, because we no longer have Service informer, we do not detect that the services are missing the pod hash. As a consequence, both the active and preview services will be servicing traffic from both the active and preview deployments.
A while back we made a conscious decision not to add a Service informer because we felt user would not be deleting the pod-template-hash labels, and it would save a API server watch on services. However, I did not consider the service delete/recreate use case, which is what happened here, and may be common/desirable to do.
In order quickly correct situations where a Service was deleted and recreated (or pod template hash labels were removed), we should consider adding back the Service informer to immediately restore the labels.
The tricky part is knowing how efficiently associate Service updates which are related to Rollouts (since they would not have any our labels yet).
The text was updated successfully, but these errors were encountered:
A user did the following:
After step 4, because we no longer have Service informer, we do not detect that the services are missing the pod hash. As a consequence, both the active and preview services will be servicing traffic from both the active and preview deployments.
A while back we made a conscious decision not to add a Service informer because we felt user would not be deleting the pod-template-hash labels, and it would save a API server watch on services. However, I did not consider the service delete/recreate use case, which is what happened here, and may be common/desirable to do.
In order quickly correct situations where a Service was deleted and recreated (or pod template hash labels were removed), we should consider adding back the Service informer to immediately restore the labels.
The tricky part is knowing how efficiently associate Service updates which are related to Rollouts (since they would not have any our labels yet).
The text was updated successfully, but these errors were encountered: