Super Privileged Container

adding spc_t in the pod via Tenant Spec works, I can suspend and resume a drive without the relabel issue but that will require a custom scc or a privileged one in any case is doable and supported as long as is enabled First suspended:

$ kubectl directpv suspend drives f34ee3a1-59ff-411b-9f80-bd5baa05f41e --dangerous
Drive 27/sde suspended

One drive offline

$ mc admin info myminio --insecure
●  myminio-pool-0-0.myminio-hl.tenant-certmanager.svc.cluster.local:9000
   Uptime: 5 seconds 
   Version: 2024-03-05T04:48:44Z
   Network: 3/3 OK 
   Drives: 2/2 OK 
   Pool: 1

●  myminio-pool-0-1.myminio-hl.tenant-certmanager.svc.cluster.local:9000
   Uptime: 5 seconds 
   Version: 2024-03-05T04:48:44Z
   Network: 3/3 OK 
   Drives: 1/2 OK 
   Pool: 1

●  myminio-pool-0-2.myminio-hl.tenant-certmanager.svc.cluster.local:9000
   Uptime: 5 seconds 
   Version: 2024-03-05T04:48:44Z
   Network: 3/3 OK 
   Drives: 2/2 OK 
   Pool: 1

Pools:
   1st, Erasure sets: 1, Drives per erasure set: 6

5 drives online, 1 drive offline

Then resumed

$ kubectl directpv resume drives f34ee3a1-59ff-411b-9f80-bd5baa05f41e
Drive 27/sde resumed

All drives online

$ mc admin info myminio --insecure
●  myminio-pool-0-0.myminio-hl.tenant-certmanager.svc.cluster.local:9000
   Uptime: 5 seconds 
   Version: 2024-03-05T04:48:44Z
   Network: 3/3 OK 
   Drives: 2/2 OK 
   Pool: 1

●  myminio-pool-0-1.myminio-hl.tenant-certmanager.svc.cluster.local:9000
   Uptime: 5 seconds 
   Version: 2024-03-05T04:48:44Z
   Network: 3/3 OK 
   Drives: 2/2 OK 
   Pool: 1

●  myminio-pool-0-2.myminio-hl.tenant-certmanager.svc.cluster.local:9000
   Uptime: 5 seconds 
   Version: 2024-03-05T04:48:44Z
   Network: 3/3 OK 
   Drives: 2/2 OK 
   Pool: 1

Pools:
   1st, Erasure sets: 1, Drives per erasure set: 6

6 drives online, 0 drives offline

For above I used the privileged policy

$ oc adm policy add-scc-to-user privileged -n tenant-certmanager -z builder
$ oc adm policy add-scc-to-user privileged -n tenant-certmanager -z deployer
$ oc adm policy add-scc-to-user privileged -n tenant-certmanager -z default
$ oc adm policy add-scc-to-user privileged -n tenant-certmanager -z myminio-sa

And I did not use RHACM as I only installed but still not sure how to use. Also redhat team replied just confirming that this is doable and if it is via Operator, then we will need custom SCC as already discussed in https://access.redhat.com/solutions/7025337

And just in case, from the Tenant perspective, it looks like this:

kind: Tenant
spec:
  pools:
  - containerSecurityContext:
      seLinuxOptions:
        type: spc_t

Notice how we support this option via containerSecurityContext

but if SCC is not in place, then below will be thrown:

create Pod myminio-pool-0-0 in StatefulSet myminio-pool-0 failed error: pods "myminio-pool-0-0" is forbidden:
unable to validate against any security context constraint:
[provider "anyuid": Forbidden: not usable by user or serviceaccount, 
provider restricted-v2: .spec.securityContext.seLinuxOptions.type: 
Invalid value: "spc_t": must be , provider restricted-v2: .initContainers[0].seLinuxOptions.level: 
Invalid value: "": must be s0:c28,c22, 
provider restricted-v2: .initContainers[0].seLinuxOptions.type: Invalid value: "spc_t": 
must be , provider restricted-v2: .containers[0].seLinuxOptions.level: Invalid value: "": must be s0:c28,c22, 
provider restricted-v2: .containers[0].seLinuxOptions.type: Invalid value: "spc_t": 
must be , provider restricted-v2: .containers[1].seLinuxOptions.level: 
Invalid value: "": must be s0:c28,c22, provider restricted-v2: .containers[1].seLinuxOptions.type: 
Invalid value: "spc_t": must be , provider "restricted": 
Forbidden: not usable by user or serviceaccount, 
provider "containerized-data-importer": 
Forbidden: not usable by user or serviceaccount, provider "nonroot-v2": 
Forbidden: not usable by user or serviceaccount, provider "nonroot": 
Forbidden: not usable by user or serviceaccount, provider "hostmount-anyuid": 
Forbidden: not usable by user or serviceaccount, provider "kubevirt-controller": 
Forbidden: not usable by user or serviceaccount, provider "machine-api-termination-handler": 
Forbidden: not usable by user or serviceaccount, provider "bridge-marker": 
Forbidden: not usable by user or serviceaccount, provider "hostnetwork-v2": 
Forbidden: not usable by user or serviceaccount, provider "hostnetwork": 
Forbidden: not usable by user or serviceaccount, provider "hostaccess": 
Forbidden: not usable by user or serviceaccount, provider "linux-bridge": 
Forbidden: not usable by user or serviceaccount, provider "kubevirt-handler": 
Forbidden: not usable by user or serviceaccount, provider "node-exporter": 
Forbidden: not usable by user or serviceaccount, provider "privileged": 
Forbidden: not usable by user or serviceaccount]

And just for fun, SPC stands for Super Privileged Container this was interesting to me.