Camel K 2.x upgrade 1.x strategy

[squakez]

Originally posted by @squakez in #4156 (comment)

As we are moving to a new version (which may have its own sets of API) which may require breaking change from V1, I wonder if it makes sense and how should be an upgrade process. I think the main problem would be any running Integration, how should we make this transition happens correctly?

[claudio4j]

It is interesting how syndesis upgraded to new versions.

It migrates the old custom resources to new one. This way the apis can be upgrade from v1alpha -> v1beta -> v1 -> ...

[squakez]

It is interesting how syndesis upgraded to new versions.

It migrates the old custom resources to new one. This way the apis can be upgrade from v1alpha -> v1beta -> v1 -> ...

Thanks for the link, was not aware of that in Syndesis. I'm right now looking to solve this problem as it finally we'll need to upgrade the KameletBinding to Binding CR type. However, I'm looking at https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#webhook-conversion which looks to be the official way in order to handle the conversion. In our case it should be easy enough as we only need to change the name of the resource.

[squakez]

I'm trying to follow the tutorial here https://book.kubebuilder.io/multiversion-tutorial/conversion.html

However, we have some problem because the v1alpha1 was made dependent of v1. Now, for conversion sake, we need v1 to depend on v1alpha1, which makes a circular reference Go does not like at all.

[squakez]

I'm thinking that in our case we may take care to convert them on the fly, during a the first reconciliation loop. Basically, when a KameletBinding would be reconciled for the first time (after an upgrade to Camel K 2.0) we may convert it to a Binding (taking care not to alter the status and avoid retriggering an Integration). @lburgazzoli @astefanutti wdyt?

[lburgazzoli]

I think that must be an opt in settings and I would be really careful to do it by default as i.e. in a gitops scenario like with argocd, if the kameletbinding get translated to binding, then argocd would probably detect a drift and re-apply the kameletbinding resulting in a ping-pong scenario

[squakez]

I think that must be an opt in settings and I would be really careful to do it by default as i.e. in a gitops scenario like with argocd, if the kameletbinding get translated to binding, then argocd would probably detect a drift and re-apply the kameletbinding resulting in a ping-pong scenario

Yes, my main concern is to avoid such kind of situations as we don't know how exactly those resources are deployed to the cluster. Right now, with no conversion in place, what it happens in an upgrade scenario is that a KameletBinding is no longer reconciled when we move to Camel K V2. The resource will sit there with nobody reconciling it. Another possibility is to delegate such a conversion on the kamel rebuild, or definitely let the user to manually convert to a Binding when he wants to upgrade.

[squakez]

if the kameletbinding get translated to binding, then argocd would probably detect a drift and re-apply the kameletbinding resulting in a ping-pong scenario

Do you think it would help if both KameletBinding and Binding will coexist? I mean, whoever is creating a KameletBinding will still see that resource, but internally Camel K would use a Binding.

[lburgazzoli]

I think that if we leave the KameltBinding CR we also must reconcile it so yeah they must co-exist.

How technically that's done that's something for which I don't have any strong opinion.
My only requirement would be that tools (gitops, monitoring, etc) must continue to work till the KameltBinding CRD gets removed.

[squakez]

I think that if we leave the KameltBinding CR we also must reconcile it so yeah they must co-exist.

How technically that's done that's something for which I don't have any strong opinion. My only requirement would be that tools (gitops, monitoring, etc) must continue to work till the KameltBinding CRD gets removed.

Okey, that was something I did not take in consideration. I see your point, but this is adding some complexity. Friday afternoon is not very good for designing things... I'll have some thought next week :)

[squakez]

I'm working to maintain KameletBindings reconciliation loop. We have a cost of duplicating code but I guess we can survive with that until we remove the CR. However, we have another problem. The KameletBinding is meant to work with v1alpha1.Kamelet, which is also promoted to v1.Kamelet. That means that the reconciliation loop fails as it does not find the expected type, but the new v1 one. A possible approach here would be to fallback to the new Kamelet version during the reconciliation loop.

EDIT: it seems this mechanism is already provided ootb by kubernetes, so, we should be okey to use v1alpha1. I am now testing the upgrade scenarios to ensure all is good to go.

[squakez]

Upgrade strategy defined in https://camel.apache.org/camel-k/next/contributing/upgrade.html