DaemonSet, Add Priority class

[oshoval]

As part of https://bugzilla.redhat.com/show_bug.cgi?id=1953482
We are adding Priority Class [1] to each network component.

The motivation is to make the control plane pods less sensitive to preemption
than user workloads.

Pods that are node specific will have the higher build-in priority,
since preempting them from a specific node, makes them unavailable
until they are rescheduled on that specific node.
Pods that are network control plane, but are not node specific
will have system-cluster-critical, which would still make them more important
than user and custom workloads, but less than system-node-critical.

Add system-node-critical to macvtap-cni DaemonSet.
Since macvtap-cni pod should run on each node,
assign system-node-critical pc to it.

[1] https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/

Signed-off-by: Or Shoval [email protected]

None

[oshoval]

@maiqueb can you review please?

[maiqueb]

Thanks, this might be a really good catch.

I think it makes sense, but I find it strange I found no mention whatsoever of this when I checked the multus / kubevirt / ... projects.

For instance, I find it strange to find the macvtap-(dp+cni) pod more important than virt-handler.

Do you mind pointing out some examples of where this is used ?

[oshoval]

Do you mind pointing out some examples of where this is used ?

Some PRs that are freshly merged (there are more on the way):
kubevirt/bridge-marker#23
nmstate/kubernetes-nmstate#777
kubevirt/cluster-network-addons-operator#895

As part of the effort on https://bugzilla.redhat.com/show_bug.cgi?id=1953482
We created a design that assign to all network pods either system-cluster-critical or system-node-critical,
the way we choose which one of those is if it should run on all nodes or not.
In case it should run on all nodes we put it on higher priority (system-node-critical as this PR),
because if it will be preempted it won't help to just schedule it on another node.

Its true that now network component will have higher priority than kubevirt, but its per design,
since those are mandatory pods for healthy system
(Kubevirt uses custom kubevirt-cluster-critical)

Will update the PR with more info as the 2nd comment say as well.

[phoracek]

Its true that now network component will have higher priority than kubevirt, but its per design,
since those are mandatory pods for healthy system
(Kubevirt uses custom kubevirt-cluster-critical)

Note that KubeVirt should switch to native classes too https://bugzilla.redhat.com/show_bug.cgi?id=1953479.

The relative priority between this CNI and KubeVirt is not really that important. To give you a backstory for this - on environments where a single node is used for both user workload and control-plane, control-plane pods should never starve out on account of user workload. If we preempt all user workload and there is still not enough space for both CNI and KubeVirt, the cluster is just flawed and can never function.

[maiqueb]

Thanks.

[phoracek]

You have to rebuild manifests: https://github.com/kubevirt/macvtap-cni/blob/main/manifests/macvtap.yaml

[oshoval]

You have to rebuild manifests: https://github.com/kubevirt/macvtap-cni/blob/main/manifests/macvtap.yaml

Nice done thanks (using make manifests)

we should add check for that i think ?
like we have on kubevirt

[maiqueb]

You have to rebuild manifests: https://github.com/kubevirt/macvtap-cni/blob/main/manifests/macvtap.yaml

Nice done thanks

we should add check for that i think ?
like we have on kubevirt

I guess we should, at least when the templates are updated.

You're welcome to implement that, if you want.

[phoracek]

/lgtm

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: maiqueb, oshoval, phoracek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [maiqueb,phoracek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment