Rollouts unprotected from invalid specs

[jessesuen]

Despite having Open API validation in the Rollout CRD, it was still possible for a user to set a bad label on a rollout:

metadata:
  labels:
    this-should-be-a-string-not-an-int: 0

When this happened the entire rollout-controller ceased to function, and was spewing the following in the logs:

E0522 19:14:14.782189       1 reflector.go:134] github.com/argoproj/argo-rollouts/pkg/client/informers/externalversions/factory.go:117: Failed to list *v1alpha1.Rollout: v1alpha1.RolloutList.Items: []v1alpha1.Rollout: v1alpha1.Rollout.Spec: v1alpha1.RolloutSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.ObjectMeta: v1.ObjectMeta.Labels: ReadString: expects " or n, but found 0, error found in #10 byte of ...|version":0,"rollout-|..., bigger context ...|1170dd38c2c2fae7a8c2a","rolling-restart-version":0,"rollout-config-map-hash":"e4984dbdd2eb949fbfd756|...
E0522 19:14:15.788187       1 reflector.go:134] github.com/argoproj/argo-rollouts/pkg/client/informers/externalversions/factory.go:117: Failed to list *v1alpha1.Rollout: v1alpha1.RolloutList.Items: []v1alpha1.Rollout: v1alpha1.Rollout.Spec: v1alpha1.RolloutSpec.Template: v1.PodTemplateSpec.Spec: v1.PodSpec.ObjectMeta: v1.ObjectMeta.Labels: ReadString: expects " or n, but found 0, error found in #10 byte of ...|version":0,"rollout-|..., bigger context ...|1170dd38c2c2fae7a8c2a","rolling-restart-version":0,"rollout-config-map-hash":"e4984dbdd2eb949fbfd756|...

This is essentially the same failure scenario as in: kubernetes/kubernetes#57705

It appears that even having Open API validation does not always save us from bad specs. We likely need to use the unstructued informer approach to protect us.

[jessesuen]

Part of the problem is that current open api schema has zero validation on the metadata field.

    openAPIV3Schema:
      properties:
        apiVersion:
          type: string
        kind:
          type: string
        metadata: {}

Not sure where else we are missing.

[jessesuen]

working on this

[jessesuen]

I'm switching us from https://github.com/ant31/crd-validation to https://github.com/kubernetes-sigs/controller-tools, which can generate the full CRD spec based on the type annotations.