Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add back service informer to handle Service recreations quicker #71

Closed
jessesuen opened this issue May 15, 2019 · 1 comment · Fixed by #95
Closed

Add back service informer to handle Service recreations quicker #71

jessesuen opened this issue May 15, 2019 · 1 comment · Fixed by #95
Assignees
Milestone

Comments

@jessesuen
Copy link
Member

jessesuen commented May 15, 2019

A user did the following:

  1. Created a blue/green Rollout with active and preview services.
  2. Deploy the services and Rollout. Wait for Rollout to fully deploy
  3. Delete the services
  4. Redeploy the services. The services will be missing the pod hash label added by Rollout

After step 4, because we no longer have Service informer, we do not detect that the services are missing the pod hash. As a consequence, both the active and preview services will be servicing traffic from both the active and preview deployments.

A while back we made a conscious decision not to add a Service informer because we felt user would not be deleting the pod-template-hash labels, and it would save a API server watch on services. However, I did not consider the service delete/recreate use case, which is what happened here, and may be common/desirable to do.

In order quickly correct situations where a Service was deleted and recreated (or pod template hash labels were removed), we should consider adding back the Service informer to immediately restore the labels.

The tricky part is knowing how efficiently associate Service updates which are related to Rollouts (since they would not have any our labels yet).

@alexmt alexmt self-assigned this May 29, 2019
@alexmt
Copy link
Contributor

alexmt commented May 29, 2019

Working on it

@alexmt alexmt added this to the v0.3.2 milestone May 29, 2019
alexmt pushed a commit to alexmt/argo-rollouts that referenced this issue Jun 1, 2019
alexmt pushed a commit to alexmt/argo-rollouts that referenced this issue Jun 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants