Kubernetes' new CRD validation rule feature

[tylergu]

Following the discussion in #160, I was searching if it is possible to specify a field to be immutable. Then I found a new feature being implemented in Kubernetes apiserver which allows some edge-triggering validation. https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#validation-rules

The new feature allows the CRD to specify some complex validation expressions, in CEL expression.
KEP is written here: https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/2876-crd-validation-expression-language/README.md For example, to express a field as immutable, users would write "self == oldSelf" in the schema.

Since this validation will be done on the server side, edge-triggering is now safe because the server has full knowledge on the resource history. This feature is currently in alpha, on track to become beta in k8s:v1.25.
This allows us to fix some of the bugs we reported without implementing the admission webhook.
I have suggested using this feature to fix some previous bugs found by us, but developers are reluctant to fix: rabbitmq/cluster-operator#992 (comment)

One thing to note here is that the error message returned by this validation is very weird. After specifying the field as immutable, I got error message saying The RabbitmqCluster "test-cluster" is invalid: spec.persistence.storageClassName: Invalid value: "string": cannot change StorageClass where cannot change StorageClass is the custom message written by me. The error message is confusing here because it seems like the value we specified is string, but string is the type of the field. So I was suggesting to improve the error message to include the real values in the error message: kubernetes/enhancements#2876 (comment). But for some weird concerns they refuse to include the real values in the error message...

[tianyin]

close for now; docs should be done in a real document way in
https://github.com/xlab-uiuc/acto/tree/main/docs