Generated CSVs from PoC in Quarkus extension

[metacosm]

I've been working on a PoC to generate CSVs from the code. The code can be found at quarkiverse/quarkus-operator-sdk#116 and relies on a SNAPSHOT build of Quarkus since we need the feature that got merged today from quarkusio/quarkus#20113.
CSVs are currently generated for each controller:

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  name: managedkafkaagentcontroller
spec:
  customresourcedefinitions:
    owned:
    - kind: ManagedKafkaAgent
      name: managedkafkaagents.managedkafka.bf2.org
      version: v1alpha1
  install:
    spec:
      clusterPermissions:
      - rules:
        - apiGroups:
          - managedkafka.bf2.org
          resources:
          - managedkafkas
          - managedkafkas/status
          - managedkafkaagents
          - managedkafkaagents/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - kafka.strimzi.io
          resources:
          - kafkas
          - kafkas/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - apps
          - extensions
          resources:
          - deployments
          - deployments/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - apps
          resources:
          - replicasets
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - ""
          resources:
          - services
          - configmaps
          - secrets
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - route.openshift.io
          resources:
          - routes
          - routes/custom-host
          - routes/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - admissionregistration.k8s.io
          resources:
          - validatingwebhookconfigurations
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operators.coreos.com
          resources:
          - subscriptions
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - packages.operators.coreos.com
          resources:
          - packagemanifests
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operators.coreos.com
          resources:
          - installplans
          verbs:
          - get
          - list
          - watch
          - patch
          - update
        - apiGroups:
          - ""
          resources:
          - pods
          - nodes
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operator.openshift.io
          resources:
          - ingresscontrollers
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        serviceAccountName: kas-fleetshard-operator
      deployments:
      - name: kas-fleetshard-operator
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: kas-fleetshard-operator
              app.kubernetes.io/version: 999-SNAPSHOT
              app.kubernetes.io/name: kas-fleetshard-operator
          template:
            metadata:
              annotations:
                app.quarkus.io/commit-id: 7046be38820fd54c539f32de1627ddb6174e04d5
                app.quarkus.io/build-timestamp: 2021-09-15 - 10:41:54 +0000
                prometheus.io/scrape: "true"
                prometheus.io/path: /q/metrics
                prometheus.io/port: "8080"
                prometheus.io/scheme: http
              labels:
                app: kas-fleetshard-operator
                app.kubernetes.io/version: 999-SNAPSHOT
                app.kubernetes.io/name: kas-fleetshard-operator
              name: kas-fleetshard-operator
            spec:
              containers:
              - env:
                - name: QUARKUS_PROFILE
                  value: prod
                - name: KUBERNETES_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                image: claprun/kas-fleetshard-operator:999-SNAPSHOT
                imagePullPolicy: Always
                livenessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /q/health/live
                    port: 8080
                    scheme: HTTP
                  initialDelaySeconds: 0
                  periodSeconds: 30
                  successThreshold: 1
                  timeoutSeconds: 10
                name: kas-fleetshard-operator
                ports:
                - containerPort: 8080
                  name: http
                  protocol: TCP
                readinessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /q/health/ready
                    port: 8080
                    scheme: HTTP
                  initialDelaySeconds: 0
                  periodSeconds: 30
                  successThreshold: 1
                  timeoutSeconds: 10
                resources:
                  limits:
                    cpu: 1500m
                    memory: 1Gi
                  requests:
                    cpu: 500m
                    memory: 512Mi
                volumeMounts:
                - mountPath: /config
                  name: logging-config-volume
                  readOnly: false
                  subPath: ""
              serviceAccount: kas-fleetshard-operator
              serviceAccountName: kas-fleetshard-operator
              volumes:
              - configMap:
                  defaultMode: 384
                  name: operator-logging-config-override
                  optional: true
                name: logging-config-volume

and

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  name: managedkafkacontroller
spec:
  customresourcedefinitions:
    owned:
    - kind: ManagedKafka
      name: managedkafkas.managedkafka.bf2.org
      version: v1alpha1
  install:
    spec:
      clusterPermissions:
      - rules:
        - apiGroups:
          - managedkafka.bf2.org
          resources:
          - managedkafkas
          - managedkafkas/status
          - managedkafkaagents
          - managedkafkaagents/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - kafka.strimzi.io
          resources:
          - kafkas
          - kafkas/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - apps
          - extensions
          resources:
          - deployments
          - deployments/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - apps
          resources:
          - replicasets
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - ""
          resources:
          - services
          - configmaps
          - secrets
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - route.openshift.io
          resources:
          - routes
          - routes/custom-host
          - routes/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - admissionregistration.k8s.io
          resources:
          - validatingwebhookconfigurations
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operators.coreos.com
          resources:
          - subscriptions
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - packages.operators.coreos.com
          resources:
          - packagemanifests
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operators.coreos.com
          resources:
          - installplans
          verbs:
          - get
          - list
          - watch
          - patch
          - update
        - apiGroups:
          - ""
          resources:
          - pods
          - nodes
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operator.openshift.io
          resources:
          - ingresscontrollers
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        serviceAccountName: kas-fleetshard-operator
      deployments:
      - name: kas-fleetshard-operator
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: kas-fleetshard-operator
              app.kubernetes.io/version: 999-SNAPSHOT
              app.kubernetes.io/name: kas-fleetshard-operator
          template:
            metadata:
              annotations:
                app.quarkus.io/commit-id: 7046be38820fd54c539f32de1627ddb6174e04d5
                app.quarkus.io/build-timestamp: 2021-09-15 - 10:41:54 +0000
                prometheus.io/scrape: "true"
                prometheus.io/path: /q/metrics
                prometheus.io/port: "8080"
                prometheus.io/scheme: http
              labels:
                app: kas-fleetshard-operator
                app.kubernetes.io/version: 999-SNAPSHOT
                app.kubernetes.io/name: kas-fleetshard-operator
              name: kas-fleetshard-operator
            spec:
              containers:
              - env:
                - name: QUARKUS_PROFILE
                  value: prod
                - name: KUBERNETES_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                image: claprun/kas-fleetshard-operator:999-SNAPSHOT
                imagePullPolicy: Always
                livenessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /q/health/live
                    port: 8080
                    scheme: HTTP
                  initialDelaySeconds: 0
                  periodSeconds: 30
                  successThreshold: 1
                  timeoutSeconds: 10
                name: kas-fleetshard-operator
                ports:
                - containerPort: 8080
                  name: http
                  protocol: TCP
                readinessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /q/health/ready
                    port: 8080
                    scheme: HTTP
                  initialDelaySeconds: 0
                  periodSeconds: 30
                  successThreshold: 1
                  timeoutSeconds: 10
                resources:
                  limits:
                    cpu: 1500m
                    memory: 1Gi
                  requests:
                    cpu: 500m
                    memory: 512Mi
                volumeMounts:
                - mountPath: /config
                  name: logging-config-volume
                  readOnly: false
                  subPath: ""
              serviceAccount: kas-fleetshard-operator
              serviceAccountName: kas-fleetshard-operator
              volumes:
              - configMap:
                  defaultMode: 384
                  name: operator-logging-config-override
                  optional: true
                name: logging-config-volume

Let me know what you think.
[Edit: updated the generated CSV after fixing an issue in the generator that wasn't properly handling defined roles]
/cc @shawkins

[shawkins]

Thanks @metacosm that looks like good progress

[rareddy]

@metacosm IMO you may want to generate single CVS instead of multiple from a project. each controller then becomes an entry in spec: customresourcedefinitions: that way we have a single operator at the end. I do not remember ever seeing more than a single CSV for an operator.

[metacosm]

@rareddy The problem here is that right now, I'm constrained by the module structure of the project. That said, I now deal with creating only one CSV per operator in a given module, provided you annotate your controllers with the same annotation.

Here's what this looks like now:

apiVersion: operators.coreos.com/v1alpha1
kind: ClusterServiceVersion
metadata:
  name: kas-fleetshard-operator.0.11.0
spec:
  customresourcedefinitions:
    owned:
    - kind: ManagedKafka
      name: managedkafkas.managedkafka.bf2.org
      version: v1alpha1
    - kind: ManagedKafkaAgent
      name: managedkafkaagents.managedkafka.bf2.org
      version: v1alpha1
  install:
    spec:
      clusterPermissions:
      - rules:
        - apiGroups:
          - managedkafka.bf2.org
          resources:
          - managedkafkas
          - managedkafkas/status
          - managedkafkaagents
          - managedkafkaagents/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - kafka.strimzi.io
          resources:
          - kafkas
          - kafkas/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - apps
          - extensions
          resources:
          - deployments
          - deployments/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - apps
          resources:
          - replicasets
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - ""
          resources:
          - services
          - configmaps
          - secrets
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - route.openshift.io
          resources:
          - routes
          - routes/custom-host
          - routes/status
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        - apiGroups:
          - admissionregistration.k8s.io
          resources:
          - validatingwebhookconfigurations
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operators.coreos.com
          resources:
          - subscriptions
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - packages.operators.coreos.com
          resources:
          - packagemanifests
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operators.coreos.com
          resources:
          - installplans
          verbs:
          - get
          - list
          - watch
          - patch
          - update
        - apiGroups:
          - ""
          resources:
          - pods
          - nodes
          verbs:
          - get
          - list
          - watch
        - apiGroups:
          - operator.openshift.io
          resources:
          - ingresscontrollers
          verbs:
          - get
          - list
          - watch
          - create
          - delete
          - patch
          - update
        serviceAccountName: kas-fleetshard-operator
      deployments:
      - name: kas-fleetshard-operator
        spec:
          replicas: 1
          selector:
            matchLabels:
              app: kas-fleetshard-operator
              app.kubernetes.io/version: 999-SNAPSHOT
              app.kubernetes.io/name: kas-fleetshard-operator
          template:
            metadata:
              annotations:
                app.quarkus.io/commit-id: 7046be38820fd54c539f32de1627ddb6174e04d5
                app.quarkus.io/build-timestamp: 2021-09-15 - 12:49:34 +0000
                prometheus.io/scrape: "true"
                prometheus.io/path: /q/metrics
                prometheus.io/port: "8080"
                prometheus.io/scheme: http
              labels:
                app: kas-fleetshard-operator
                app.kubernetes.io/version: 999-SNAPSHOT
                app.kubernetes.io/name: kas-fleetshard-operator
              name: kas-fleetshard-operator
            spec:
              containers:
              - env:
                - name: QUARKUS_PROFILE
                  value: prod
                - name: KUBERNETES_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
                image: claprun/kas-fleetshard-operator:999-SNAPSHOT
                imagePullPolicy: Always
                livenessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /q/health/live
                    port: 8080
                    scheme: HTTP
                  initialDelaySeconds: 0
                  periodSeconds: 30
                  successThreshold: 1
                  timeoutSeconds: 10
                name: kas-fleetshard-operator
                ports:
                - containerPort: 8080
                  name: http
                  protocol: TCP
                readinessProbe:
                  failureThreshold: 3
                  httpGet:
                    path: /q/health/ready
                    port: 8080
                    scheme: HTTP
                  initialDelaySeconds: 0
                  periodSeconds: 30
                  successThreshold: 1
                  timeoutSeconds: 10
                resources:
                  limits:
                    cpu: 1500m
                    memory: 1Gi
                  requests:
                    cpu: 500m
                    memory: 512Mi
                volumeMounts:
                - mountPath: /config
                  name: logging-config-volume
                  readOnly: false
                  subPath: ""
              serviceAccount: kas-fleetshard-operator
              serviceAccountName: kas-fleetshard-operator
              volumes:
              - configMap:
                  defaultMode: 384
                  name: operator-logging-config-override
                  optional: true
                name: logging-config-volume

[metacosm]

The problem, though, in this project's case, the sync module doesn't use the quarkus-operator-sdk extension so it's invisible to the CSV generator. Then again, even if it was using the extension, it currently works at the granularity of a maven module…

[rareddy]

sync I understand that is oddball in our case, I would ignore that in auto-generating that. That was one of the reasons I thought filling out template like kubernetes.yaml file will work better for us.

[metacosm]

It would actually help if you could describe the ideal scenario for you: how would you want to provide the information that cannot be inferred automatically?
Another note: the extension will automatically add the data for any icon named <csv name>.icon.png found in your resources directory to the CSV as well. This is limited to PNG files right now but this could be opened to other types as well, though I'm not sure which types are supported by the CSV format.

[rareddy]

+1 for the icon

My thought is currently if kubernetes.yml file exists, then the build is smart enough to switch out the container image address in deployment and labels that do change with version numbers and names etc. We do exactly the same thing with <>.clusterserviceversion.yaml file? for kas-fleetshard we can switch out the kubernetes.yaml with this file. For 'kas-fleetshardwe would still have the odd case ofsyncthat needs merging afterward. Just like howkubernetes.yml` is generated if not there then we would need similar auto-generation as follow on which you are already doing.

For kas-fleetshard to include sync or any additional controllers from other modules, it may be simpler to write a maven plugin that does exactly like above and take the deployment image details from the overall build, then can even build a catalog source image out or artifact such as what is needed for operatorhub.io

[metacosm]

For kas-fleetshard to include sync or any additional controllers from other modules, it may be simpler to write a maven plugin that does exactly like above and take the deployment image details from the overall build, then can even build a catalog source image out or artifact such as what is needed for operatorhub.io

The other option would also be to create a maven module using the other two as dependencies solely for packaging purposes.

[rareddy]

The other option would also be to create a maven module using the other two as dependencies solely for packaging purposes.

I am not familiar with Quarkus maven magic yet, but yes that is the exact intention :)