Adding label to ws-manager prevents updating Gitpod with the Installer

[corneliusludmann]

With 44ef470 we added new labels to the ws-manager. However, that seems to prevent users from updating their Gitpod installation with the Installer. Following errors occurs:

for: "k8s.yaml": Deployment.apps "ws-manager-bridge" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"gitpod", "component":"ws-manager-bridge"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable

We need to find a way to update Gitpod with the Installer despite the changes of immutable fields.

[jimmybrancaccio]

I think I may be experiencing this issue as well using main.2647 - https://ghostbin.linuxbox.ninja/paste/6btjh If what I am seeing is something different, let me know and I can create a new issue!

Update: I was able to get my instance updated to main.2651 with the help of @mrsimonemms. I ended up having to delete a few of the DaemonSets, the openvsx-proxy StatefulSet and a bunch of Deployments.

[nandajavarma]

@corneliusludmann As this is a difficult problem to handle from the side of installer at this point, I am wondering if we should add a note to the upgrade guide a workaround for the problem, much like how @jimmybrancaccio solved the issue.

My suggestion is to do add a command to delete the objects with the flag --cascade=orphan on the DaemonSets/Deployments that has an immutable field changed. This makes the DS or deployment object to "orphan" the pods controlled by it, and reattach following an apply. This avoids the necessity of a restart on all the resources.

[mrsimonemms]

I added a note to the 2022.03.0 release on how to upgrade

---

This release introduces some new labels for some resources. As labels are immutable properties, these resources will need to be deleted before upgrade can be made - this is basically all the Gitpod deployments/daemonsets/statefulsets

NB this will cause a short amount of downtime when these releases are deleted and before they are reapplied. Ensure you plan your upgrade accordingly.

kubectl delete deployments.apps blobserve content-service dashboard ide-proxy image-builder-mk3 proxy server ws-manager ws-manager-bridge ws-proxy \
&& kubectl delete daemonset.apps agent-smith registry-facade ws-daemon \
&& kubectl delete statefulset.apps openvsx-proxy

[nandajavarma]

@mrsimonemms perfect! sorry I missed that!

[mrsimonemms]

Don't worry about it. I couldn't find this ticket either.

I think this can be closed. Reopen if you think it needs any more work