Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

topology-updater: introduce exclude-list #949

Merged
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -119,12 +119,20 @@ templates:
@# Need to prepend each line in the sample config with spaces in order to
@# fit correctly in the configmap spec.
@sed s'/^/ /' deployment/components/worker-config/nfd-worker.conf.example > nfd-worker.conf.tmp
@sed s'/^/ /' deployment/components/topology-updater-config/nfd-topology-updater.conf.example > nfd-topology-updater.conf.tmp
@# The sed magic below replaces the block of text between the lines with start and end markers
@start=NFD-WORKER-CONF-START-DO-NOT-REMOVE; \
end=NFD-WORKER-CONF-END-DO-NOT-REMOVE; \
sed -e "/$$start/,/$$end/{ /$$start/{ p; r nfd-worker.conf.tmp" \
-e "}; /$$end/p; d }" -i deployment/helm/node-feature-discovery/values.yaml
@start=NFD-TOPOLOGY-UPDATER-CONF-START-DO-NOT-REMOVE; \
end=NFD-TOPOLOGY-UPDATER-CONF-END-DO-NOT-REMOVE; \
sed -e "/$$start/,/$$end/{ /$$start/{ p; r nfd-topology-updater.conf.tmp" \
-e "}; /$$end/p; d }" -i deployment/helm/node-feature-discovery/values.yaml
@rm nfd-worker.conf.tmp
@rm nfd-topology-updater.conf.tmp



.generator.image.stamp: Dockerfile_generator
$(IMAGE_BUILD_CMD) \
2 changes: 2 additions & 0 deletions cmd/nfd-topology-updater/main.go
Original file line number Diff line number Diff line change
@@ -155,6 +155,8 @@ func initFlags(flagset *flag.FlagSet) (*topology.Args, *resourcemonitor.Args) {
"NFD server address to connecto to.")
flagset.StringVar(&args.ServerNameOverride, "server-name-override", "",
"Hostname expected from server certificate, useful in testing")
flagset.StringVar(&args.ConfigFile, "config", "/etc/kubernetes/node-feature-discovery/nfd-topology-updater.conf",
"Config file to use.")
Tal-or marked this conversation as resolved.
Show resolved Hide resolved

klog.InitFlags(flagset)

7 changes: 5 additions & 2 deletions cmd/nfd-topology-updater/main_test.go
Original file line number Diff line number Diff line change
@@ -34,19 +34,22 @@ func TestArgsParse(t *testing.T) {
Convey("noPublish is set and args.sources is set to the default value", func() {
So(args.NoPublish, ShouldBeTrue)
So(args.Oneshot, ShouldBeTrue)
So(args.ConfigFile, ShouldEqual, "/etc/kubernetes/node-feature-discovery/nfd-topology-updater.conf")
So(finderArgs.SleepInterval, ShouldEqual, 60*time.Second)
So(finderArgs.PodResourceSocketPath, ShouldEqual, "/var/lib/kubelet/pod-resources/kubelet.sock")
})
})

Convey("When valid args are specified for -kubelet-config-url and -sleep-interval,", func() {
Convey("When valid args are specified for -kubelet-config-url, -sleep-interval and -config,", func() {
args, finderArgs := parseArgs(flags,
"-kubelet-config-uri=file:///path/testconfig.yaml",
"-sleep-interval=30s")
"-sleep-interval=30s",
"-config=/path/nfd-topology-updater.conf")

Convey("args.sources is set to appropriate values", func() {
So(args.NoPublish, ShouldBeFalse)
So(args.Oneshot, ShouldBeFalse)
So(args.ConfigFile, ShouldEqual, "/path/nfd-topology-updater.conf")
So(finderArgs.SleepInterval, ShouldEqual, 30*time.Second)
So(finderArgs.KubeletConfigURI, ShouldEqual, "file:///path/testconfig.yaml")
So(finderArgs.PodResourceSocketPath, ShouldEqual, "/var/lib/kubelet/pod-resources/kubelet.sock")
10 changes: 10 additions & 0 deletions deployment/components/topology-updater-config/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

generatorOptions:
disableNameSuffixHash: true

configMapGenerator:
- files:
- nfd-topology-updater.conf=nfd-topology-updater.conf.example
name: nfd-topology-updater-conf
Tal-or marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
## key = node name, value = list of resources to be excluded.
## use * to exclude from all nodes.
## an example for how the exclude list should looks like
#excludeList:
# node1: [cpu]
# node2: [memory, example/deviceA]
# *: [hugepages-2Mi]
Original file line number Diff line number Diff line change
@@ -10,6 +10,9 @@
- name: kubelet-podresources-sock
hostPath:
path: /var/lib/kubelet/pod-resources/kubelet.sock
- name: nfd-topology-updater-conf
configMap:
name: nfd-topology-updater-conf
Tal-or marked this conversation as resolved.
Show resolved Hide resolved

- op: add
path: /spec/template/spec/containers/0/volumeMounts
@@ -20,6 +23,9 @@
mountPath: /host-var/lib/kubelet/pod-resources/kubelet.sock
- name: host-sys
mountPath: /host-sys
- name: nfd-topology-updater-conf
mountPath: "/etc/kubernetes/node-feature-discovery"
readOnly: true

- op: add
path: /spec/template/spec/containers/0/args/-
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "node-feature-discovery.fullname" . }}-topology-updater-conf
namespace: {{ include "node-feature-discovery.namespace" . }}
labels:
{{- include "node-feature-discovery.labels" . | nindent 4 }}
data:
nfd-topology-updater.conf: |-
{{- .Values.topologyUpdater.config | toYaml | nindent 4 }}
Original file line number Diff line number Diff line change
@@ -68,6 +68,9 @@ spec:
mountPath: "/etc/kubernetes/node-feature-discovery/certs"
readOnly: true
{{- end }}
- name: topology-updater-conf
mountPath: "/etc/kubernetes/node-feature-discovery"
readOnly: true

resources:
{{- toYaml .Values.topologyUpdater.resources | nindent 12 }}
@@ -91,12 +94,19 @@ spec:
{{- else }}
path: /var/lib/kubelet/pod-resources/kubelet.sock
{{- end }}
- name: nfd-topology-updater-conf
configMap:
name: {{ include "node-feature-discovery.fullname" . }}-topology-updater-conf
items:
- key: nfd-topology-updater.conf
path: nfd-topology-updater.conf
{{- if .Values.tls.enable }}
- name: nfd-topology-updater-cert
secret:
secretName: nfd-topology-updater-cert
{{- end }}


{{- with .Values.topologyUpdater.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
10 changes: 10 additions & 0 deletions deployment/helm/node-feature-discovery/values.yaml
Original file line number Diff line number Diff line change
@@ -368,6 +368,16 @@ worker:
priorityClassName: ""

topologyUpdater:
config: ### <NFD-TOPOLOGY-UPDATER-CONF-START-DO-NOT-REMOVE>
## key = node name, value = list of resources to be excluded.
## use * to exclude from all nodes.
## an example for how the exclude list should looks like
#excludeList:
# node1: [cpu]
# node2: [memory, example/deviceA]
# *: [hugepages-2Mi]
### <NFD-TOPOLOGY-UPDATER-CONF-END-DO-NOT-REMOVE>

enable: false
createCRDs: false

Original file line number Diff line number Diff line change
@@ -19,3 +19,4 @@ components:
- ../../components/worker-config
- ../../components/common
- ../../components/topology-updater
- ../../components/topology-updater-config
1 change: 1 addition & 0 deletions deployment/overlays/topologyupdater/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -16,3 +16,4 @@ resources:
components:
- ../../components/common
- ../../components/topology-updater
- ../../components/topology-updater-config
43 changes: 22 additions & 21 deletions docs/deployment/helm.md
Original file line number Diff line number Diff line change
@@ -142,24 +142,25 @@ We have introduced the following Chart parameters.

### Topology updater parameters

| Name | Type | Default | description |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole section is marked within the diff because of reformat of the rows in the table.

|-----------------------------------------------|--------|---------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `topologyUpdater.*` | dict | | NFD Topology Updater configuration |
| `topologyUpdater.enable` | bool | false | Specifies whether the NFD Topology Updater should be created |
| `topologyUpdater.createCRDs` | bool | false | Specifies whether the NFD Topology Updater CRDs should be created |
| `topologyUpdater.serviceAccount.create` | bool | true | Specifies whether the service account for topology updater should be created |
| `topologyUpdater.serviceAccount.annotations` | dict | {} | Annotations to add to the service account for topology updater |
| `topologyUpdater.serviceAccount.name` | string | | The name of the service account for topology updater to use. If not set and create is true, a name is generated using the fullname template and `-topology-updater` suffix |
| `topologyUpdater.rbac` | dict | | RBAC [parameters](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for the topology updater |
| `topologyUpdater.rbac.create` | bool | false | Specifies whether the cluster role and binding for topology updater should be created |
| `topologyUpdater.kubeletConfigPath` | string | "" | Specifies the kubelet config host path |
| `topologyUpdater.kubeletPodResourcesSockPath` | string | "" | Specifies the kubelet sock path to read pod resources |
| `topologyUpdater.updateInterval` | string | 60s | Time to sleep between CR updates. Non-positive value implies no CR update. |
| `topologyUpdater.watchNamespace` | string | `*` | Namespace to watch pods, `*` for all namespaces |
| `topologyUpdater.podSecurityContext` | dict | {} | [PodSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) holds pod-level security attributes and common container settings |
| `topologyUpdater.securityContext` | dict | {} | Container [security settings](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
| `topologyUpdater.resources` | dict | {} | Topology updater pod [resources management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| `topologyUpdater.nodeSelector` | dict | {} | Topology updater pod [node selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) |
| `topologyUpdater.tolerations` | dict | {} | Topology updater pod [node tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
| `topologyUpdater.annotations` | dict | {} | Topology updater pod [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) |
| `topologyUpdater.affinity` | dict | {} | Topology updater pod [affinity](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/) |
| Name | Type | Default | description |
|-----------------------------------------------|--------|---------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `topologyUpdater.*` | dict | | NFD Topology Updater configuration |
| `topologyUpdater.enable` | bool | false | Specifies whether the NFD Topology Updater should be created |
| `topologyUpdater.createCRDs` | bool | false | Specifies whether the NFD Topology Updater CRDs should be created |
| `topologyUpdater.serviceAccount.create` | bool | true | Specifies whether the service account for topology updater should be created |
| `topologyUpdater.serviceAccount.annotations` | dict | {} | Annotations to add to the service account for topology updater |
| `topologyUpdater.serviceAccount.name` | string | | The name of the service account for topology updater to use. If not set and create is true, a name is generated using the fullname template and `-topology-updater` suffix |
| `topologyUpdater.rbac` | dict | | RBAC [parameters](https://kubernetes.io/docs/reference/access-authn-authz/rbac/) for the topology updater |
| `topologyUpdater.rbac.create` | bool | false | Specifies whether the cluster role and binding for topology updater should be created |
| `topologyUpdater.kubeletConfigPath` | string | "" | Specifies the kubelet config host path |
| `topologyUpdater.kubeletPodResourcesSockPath` | string | "" | Specifies the kubelet sock path to read pod resources |
| `topologyUpdater.updateInterval` | string | 60s | Time to sleep between CR updates. Non-positive value implies no CR update. |
| `topologyUpdater.watchNamespace` | string | `*` | Namespace to watch pods, `*` for all namespaces |
| `topologyUpdater.podSecurityContext` | dict | {} | [PodSecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-pod) holds pod-level security attributes and common container settings |
| `topologyUpdater.securityContext` | dict | {} | Container [security settings](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) |
| `topologyUpdater.resources` | dict | {} | Topology updater pod [resources management](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) |
| `topologyUpdater.nodeSelector` | dict | {} | Topology updater pod [node selector](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#nodeselector) |
| `topologyUpdater.tolerations` | dict | {} | Topology updater pod [node tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) |
| `topologyUpdater.annotations` | dict | {} | Topology updater pod [annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) |
| `topologyUpdater.affinity` | dict | {} | Topology updater pod [affinity](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/) |
| `topologyUpdater.config` | dict | | [configuration](../reference/topology-updater-configuration-reference) |
16 changes: 15 additions & 1 deletion docs/reference/topology-updater-commandline-reference.md
Original file line number Diff line number Diff line change
@@ -21,7 +21,8 @@ To quickly view available command line flags execute `nfd-topology-updater -help
In a docker container:

```bash
docker run gcr.io/k8s-staging-nfd/node-feature-discovery:master nfd-topology-updater -help
docker run {{ site.container_image }} \
nfd-topology-updater -help
```

### -h, -help
@@ -32,6 +33,19 @@ Print usage and exit.

Print version and exit.

### -config

The `-config` flag specifies the path of the nfd-topology-updater
configuration file to use.

Default: /etc/kubernetes/node-feature-discovery/nfd-topology-updater.conf

Example:

```bash
nfd-topology-updater -config=/opt/nfd/nfd-topology-updater.conf
```

### -server

The `-server` flag specifies the address of the nfd-master endpoint where to
52 changes: 52 additions & 0 deletions docs/reference/topology-updater-configuration-reference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
title: "Topology-Updater config reference"
layout: default
sort: 5
---

# Configuration file reference of nfd-topology-updater
{: .no_toc}

## Table of contents
{: .no_toc .text-delta}

1. TOC
{:toc}

---

See the
[sample configuration file](https://github.com/kubernetes-sigs/node-feature-discovery/blob/{{site.release}}/deployment/components/topology-updater-config/nfd-topology-updater.conf.example)
for a full example configuration.

## excludeList

The `excludeList` specifies a key-value map of allocated resources
that should not be examined by the topology-updater
agent per node.
Each key is a node name with a value as a list of resources
that should not be examined by the agent for that specific node.
Tal-or marked this conversation as resolved.
Show resolved Hide resolved

Default: *empty*

Example:

```yaml
excludeList:
nodeA: [hugepages-2Mi]
nodeB: [memory]
nodeC: [cpu, hugepages-2Mi]
```

### excludeList.*
`excludeList.*` is a special value that use to specify all nodes.
A resource that would be listed under this key, would be excluded from all nodes.

Default: *empty*

Example:

```yaml
excludeList:
'*': [hugepages-2Mi]
```
Loading