Add option to set PodDisruptionBudget

[xoxys]

Is your feature request related to a problem? Please describe.

It's not possible to override the default PodDisruptionBudget while the default might not fit for every use case.

Describe the solution you'd like

Add an option to set a custom PodDisruptionBudget or disable it.

[jiuker]

Under which case you need to disable this? @xoxys

[xoxys]

I'm using a single server multi disk tenant on a small setup. During node drains, I would still like to get the tenant started on another node even if this results in a short downtime, which is acceptable in my case. The current behavior is that during node drains, the hard-coded PodDisruptionBudget blocks the node drain and the pod is not automatically transferred to another node. You can still keep the default, but is it required to enforce it for everyone?

[jiuker]

I think this is acceptable, you can now manually delete this resource because this resource is mandatory for cluster health to create.

[xoxys]

I don't understand your answer. Manually deleting is pointless if the operator will ensure it again? So basically the operator does only support multi-server deployments and is not intended to be used for single-server deployments?

[jiuker]

I don't understand your answer. Manually deleting is pointless if the operator will ensure it again? So basically the operator does only support multi-server deployments and is not intended to be used for single-server deployments?

Yes, a single-node minio cluster, which we consider to be a cluster that does not have production properties. @xoxys . BTW, @harshavardhana @pjuarezd cc

[D1StrX]

MinIO in a k8s cluster obstructs the draining in 1.27.x as well
(will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
This is a deployment of 2 pods spread on 2/3 workers. It doesn't delete the pod on the draining node, and draining command logs the above.

Will the PR also fix the issue for the environment outlined above?

[jiuker]

@D1StrX Yes. It will not create for servers are 1 or 2.