Allow optionally setting node taints defined on the NodeFeatureRule CR

[fmuyassarov]

This PR enables setting node taints defined on NodeFeatureRule CR. User can
define a list of taints under spec.rule of the CR as in this example:

apiVersion: nfd.k8s-sigs.io/v1alpha1
kind: NodeFeatureRule
metadata:
  name: rule
spec:
  rules:
    - name: "rule"
      labels:
        "my-sample-feature": "true"
      taints:
        - effect: PreferNoSchedule
          key: "feature.node.kubernetes.io/special-node"
          value: "true"
        - effect: NoExecute
          key: "feature.node.kubernetes.io/dedicated-node"
      matchFeatures:
        - feature: kernel.loadedmodule
          matchExpressions:
            dummy: {op: Exists}

$ kubectl get no kind-worker -ojson | jq .spec.taints
[
  {
    "effect": "PreferNoSchedule",
    "key": "feature.node.kubernetes.io/special-node",
    "value": "true"
  },
  {
    "effect": "NoExecute",
    "key": "feature.node.kubernetes.io/dedicated-node",
  }
]
$ kubectl get no kind-worker -ojson | jq .metadata.annotations | grep taint
  "nfd.node.kubernetes.io/taints": "feature.node.kubernetes.io/special-node=true:PreferNoSchedule,feature.node.kubernetes.io/dedicated-node:NoExecute",

The only allowed values for the effect are NoExecute, NoSchedule, PrefereNoSchedule
(following k8s). By default this feature is disabled and enabling it requires explicitly setting
--enable-taints flag true. Even if there are taints defined in the NodeFeatureRule CR and the
rule matches, taints will not be set on the node until flag is enabled.

NFD master updates the node object based on the addition/removal of taints in the CR.
Deleting the CR results in deletion of the taints too. nfd.node.kubernetes.io/taints annotation
is added to keep track of taints. The value of the new annotation is a list of taints in the format
of nfd.node.kubernetes.io/taints: "<key>=<value>:<effect>,<key>=<value>:<effect>".

Fixes: #540

[netlify]

✅ Deploy Preview for kubernetes-sigs-nfd ready!

Name	Link
🔨 Latest commit	984a3de
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-nfd/deploys/638a19df59623f0008b6cd96
😎 Deploy Preview	https://deploy-preview-910--kubernetes-sigs-nfd.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site settings.

[fmuyassarov]

Ops, seems I'm behind the master. I'll rebase. Actually I missed to the part where taints are deleted based on the deletion of the NodeFeatureRule CR or the specific taints on the CR (will add it shortly).

[fmuyassarov]

/hold

[fmuyassarov]

/cc @marquiz
I think it is ready for the review and meanwhile I will add a commit with unit tests.

[fmuyassarov]

/hold cancel

[fmuyassarov]

/test pull-node-feature-discovery-build-image-cross-generic

[fmuyassarov]

/cc @ArangoGutierrez @zvonkok

[marquiz]

Thanks @fmuyassarov! I took a first quick pass (incomplete, not even going through all the changes, yet) and this looks really good already.

One suggestion/ask: could you submit the package renames

- whatever "k8s.io/api/core/v1"
+ corev1 "k8s.io/api/core/v1"

and other possible "beautifying" as separate PR(s). This will greatly reduce the "noise" in this PR and make review easier

[marquiz]

BTW, now that #848 is in, we should also add some e2e coverage. But that can be done in a separate PR (and tracked in a separate issue), imo

[fmuyassarov]

BTW, now that #848 is in, we should also add some e2e coverage. But that can be done in a separate PR (and tracked in a separate issue), imo

Yep, I'm on it now 😊

[marquiz]

@fmuyassarov please rebase
ping @ArangoGutierrez

[fmuyassarov]

@fmuyassarov please rebase ping @ArangoGutierrez

done

[marquiz]

Thx @fmuyassarov

[ArangoGutierrez]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ArangoGutierrez, fmuyassarov, marquiz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [marquiz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment