Should chart have minimum k8s version as 1.25.0?

[tthvo]

Upstream docs specifies >= 1.25 as k8s version constraint, but chart info shows:

cryostat-helm/charts/cryostat/Chart.yaml

Line 9 in 3570abd

kubeVersion: ">= 1.21.0-0"

Related to cryostatio/cryostatio.github.io#194

Should this be 1.25 too? Or our chart has a different version constraint than the operator?

[tthvo]

I remember bumping k8s version to 1.21 (both on helm and operator) as minimum to accommodate the immutable secrets. But I guess its 1.25 should be the correct one for both?

[andrewazores]

Should this be 1.25 too? Or our chart has a different version constraint than the operator?

There are different implementation details and k8s features used between the Operator and Helm chart, so it makes sense that they might have different technical reasons for constraining the minimum k8s versions. From a testing and distribution POV however, it's a lot simpler to set them both to have the same minimum version constraint, governed by whichever requires a newer version.

And anyway, 1.25 has already been EOL for most of a year, so I don't think setting that as our minimum version should really be a concern. Clusters that are being actively developed on should have upgraded well beyond our minimum constraint by now.

[tthvo]

Should this be 1.25 too? Or our chart has a different version constraint than the operator?

There are different implementation details and k8s features used between the Operator and Helm chart, so it makes sense that they might have different technical reasons for constraining the minimum k8s versions. From a testing and distribution POV however, it's a lot simpler to set them both to have the same minimum version constraint, governed by whichever requires a newer version.

And anyway, 1.25 has already been EOL for most of a year, so I don't think setting that as our minimum version should really be a concern. Clusters that are being actively developed on should have upgraded well beyond our minimum constraint by now.

Right, makes sense! I guess I should close this issue or any suggestions for updating the constraints?

[andrewazores]

Since both advertise k8s>=1.21.0 in release 3.0.0, but the Operator will advertise k8s>=1.25.0 in 3.0.1, I think the Helm chart should also be >=1.25.0 for 3.0.1.

For 4.0+ I think an algorithm like this makes sense:

1. Find the minimum k8s version availability for all features used across Operator and Helm - featureLevel
2. Check the oldest supported upstream k8s version - upstreamKube
3. Check the k8s version corresponding to oldest supported OpenShift - openShiftKube
4. Set the Helm/Operator minKubeVersion to max(min(upstreamKube, openShiftKube), featureLevel)

This also means that during Helm/Operator development we should be careful not to introduce new features in a release version if these features are not available in the oldest k8s/OpenShift versions still supported at the time - ie we should try not to push featureLevel higher than either upstreamKube or openShiftKube, but if we do, then of course this does dictate the versions we can support.

[tthvo]

Set the Helm/Operator minKubeVersion to max(min(upstreamKube, openShiftKube), featureLevel)

Just curious: Should we just follow the openShiftKube here -> max(openShiftKube, featureLevel)?

I can help update the constraint for now 3.0.1 at least now.

[tthvo]

For 4.0+ I think an algorithm like this makes sense

I think this algorithm would be helpfuly to document somewhere here when agreed on :D Maybe in CONTRIBUTING.md / README.md.

[andrewazores]

Just curious: Should we just follow the openShiftKube here -> max(openShiftKube, featureLevel)?

This goes back to the note after the algorithm. It has to be max(openShiftKube, featureLevel), because if the feature level we're actually using is higher, then we simply need to have at least that version or else we are undeployable. But, we should be careful not to put ourselves in that position unless we have very good reason to.

[andrewazores]

@ebaron WDYT about the little algorithm above for determining k8s versions?

[tthvo]

Just curious: Should we just follow the openShiftKube here -> max(openShiftKube, featureLevel)?

This goes back to the note after the algorithm. It has to be max(openShiftKube, featureLevel), because if the feature level we're actually using is higher, then we simply need to have at least that version or else we are undeployable. But, we should be careful not to put ourselves in that position unless we have very good reason to.

Oh right, i guess I was wondering if we can only consider k8s version corresponding to min supported OpenShift (i.e. skip step 2)?

[andrewazores]

I think it's important that we remain as widely compatible as reasonable, so if there is an upstream k8s still receiving support after OpenShift moves past it, I would prefer to linger back with the old k8s version too.

[ebaron]

@ebaron WDYT about the little algorithm above for determining k8s versions?

This makes sense to me

[andrewazores]

https://github.com/cryostatio/cryostat-operator/wiki/Kubernetes-Version-Support

https://github.com/cryostatio/cryostat-helm/wiki/Kubernetes-Version-Support