ROX-15941: Run multiple operators from FSS

[kurlov]

Description

Make FSS able to run multiple ACS Operator versions.
Each operator version has its own deployment.
The metrics service and ServiceAccount can be shared between multiple versions.
For the sake of simplicity, there are two versions installed on startup before reconciler run.

The next steps are:

• fetch operators versions from FM
• installed desired versions on ticker base manner (similar to centrals reconcile)
• cleanup redundant operator versions

Checklist (Definition of Done)

• Unit and integration tests added
• Added test description under Test manual
• Documentation added if necessary (i.e. changes to dev setup, test execution, ...)
• CI and all relevant tests are passing
• Add the ticket number to the PR title if available, i.e. ROX-12345: ...
• Discussed security and business related topics privately. Will move any security and business related topics that arise to private communication channel.

Test manual

First of all, the OLM based ACS Operator should be deleted without removing an existing CRDs

1. List and delete OLM subscription for the ACS Operator

kubectl get subscription -n stackrox-operator
kubectl delete subscription <subscription> -n stackrox-operator

2. Delete the Operator’s ClusterServiceVersion

kubectl get clusterserviceversion -n stackrox-operator
kubectl delete clusterserviceversion rhacs-operator.<version> -n stackrox-operator

3. Check that there is no running ACS Operator

kubectl get pods -n stackrox-operator
NAME                                                              READY   STATUS      RESTARTS      AGE

Then it's time to run and verify
4. Build and deploy

make deploy/dev

5. Check that a two ACS Operator versions are running

kubectl get pods -n stackrox-operator
NAME                                                              READY   STATUS      RESTARTS       AGE
c8dae07dab84cfc9348095952222435b18a827136292688d44a0b210f6nf8dl   0/1     Completed   0              118d
rhacs-operator-controller-manager-3.74.0-5fb9cbcdbb-56nqv         2/2     Running     0              13s
rhacs-operator-controller-manager-3.74.1-5765676ffc-l9bpp         2/2     Running     0              13s
...

5. Check deployments

kubectl get deployments -n stackrox-operator
NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
rhacs-operator-controller-manager-3.74.0   1/1     1            1           27s
rhacs-operator-controller-manager-3.74.1   1/1     1            1           27s

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[kurlov]

this env was created by OLM and it is redundant for FSS

[kurlov]

/test e2e

[ludydoo]

should we make this smaller? This way we could have many more characters available for the repository/version suffix

[kurlov]

I dropped -controller part

[porridge]

LGTM overall, some nitpicks inline and +1 to those from @ludydoo

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: johannes94, kurlov, porridge, SimonBaeumer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [SimonBaeumer,johannes94,kurlov,porridge]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

New changes are detected. LGTM label has been removed.

[kurlov]

/test e2e