Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: coredns pod supports affinity and stain tolerance #2107

Merged
merged 6 commits into from
Sep 16, 2024
Merged

feat: coredns pod supports affinity and stain tolerance #2107

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
18821e6
feat: coredns pod supports affinity and stain tolerance
mayooot Sep 3, 2024
5fc1505
feat(config/config.go): CoreDNSDeployment struct add Affinity and Aff…
mayooot Sep 4, 2024
0689dc4
feat(chart): regenerate values.schema.json
mayooot Sep 4, 2024
b9509ee
test (chart): add a test case to check coredns deployment affinity an…
mayooot Sep 4, 2024
658df9f
style(chart): delete unused spaces
mayooot Sep 4, 2024
16f462f
feat(config/values.yaml): coredns add affinity and tolerations
mayooot Sep 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions chart/templates/coredns-configmap.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,14 @@ data:
{{- if .Values.controlPlane.coredns.deployment.nodeSelector }}
{{ toYaml .Values.controlPlane.coredns.deployment.nodeSelector | indent 12 }}
{{- end }}
{{- if .Values.controlPlane.coredns.deployment.affinity }}
affinity:
{{ toYaml .Values.controlPlane.coredns.deployment.affinity | indent 12 }}
{{- end }}
{{- if .Values.controlPlane.coredns.deployment.tolerations }}
tolerations:
{{ toYaml .Values.controlPlane.coredns.deployment.tolerations | indent 12 }}
{{- end }}
{{- if .Values.controlPlane.coredns.deployment.topologySpreadConstraints }}
topologySpreadConstraints:
{{ toYaml .Values.controlPlane.coredns.deployment.topologySpreadConstraints | indent 12 }}
Expand Down
242 changes: 242 additions & 0 deletions chart/tests/coredns-configmap_test.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -333,3 +333,245 @@ tests:
- name: metrics
port: 9153
protocol: TCP
- it: should correctly apply affinity and tolerations
set:
controlPlane:
coredns:
deployment:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- coredns
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
asserts:
- equal:
path: data["coredns.yaml"]
value: |-
apiVersion: v1
kind: ServiceAccount
metadata:
name: coredns
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:coredns
rules:
- apiGroups:
- ""
resources:
- endpoints
- services
- pods
- namespaces
verbs:
- list
- watch
- apiGroups:
- discovery.k8s.io
resources:
- endpointslices
verbs:
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:coredns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:coredns
subjects:
- kind: ServiceAccount
name: coredns
namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |-
.:1053 {
errors
health
ready
rewrite name regex .*\.nodes\.vcluster\.com kubernetes.default.svc.cluster.local
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
hosts /etc/NodeHosts {
ttl 60
reload 15s
fallthrough
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
loadbalance
}

import /etc/coredns/custom/*.server
NodeHosts: ""
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: coredns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/name: "CoreDNS"
spec:
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
selector:
matchLabels:
k8s-app: kube-dns
template:
metadata:
labels:
k8s-app: kube-dns
spec:
priorityClassName: ""
serviceAccountName: coredns
nodeSelector:
kubernetes.io/os: linux
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- coredns
topologyKey: kubernetes.io/hostname
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
topologySpreadConstraints:
- labelSelector:
matchLabels:
k8s-app: kube-dns
maxSkew: 1
topologyKey: kubernetes.io/hostname
whenUnsatisfiable: DoNotSchedule
containers:
- name: coredns
image: {{.IMAGE}}
imagePullPolicy: IfNotPresent
resources:
limits:
cpu: 1000m
memory: 170Mi
requests:
cpu: 20m
memory: 64Mi
args: [ "-conf", "/etc/coredns/Corefile" ]
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
readOnly: true
- name: custom-config-volume
mountPath: /etc/coredns/custom
readOnly: true
securityContext:
runAsNonRoot: true
runAsUser: {{.RUN_AS_USER}}
runAsGroup: {{.RUN_AS_GROUP}}
allowPrivilegeEscalation: false
capabilities:
add:
- NET_BIND_SERVICE
drop:
- ALL
readOnlyRootFilesystem: true
livenessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
readinessProbe:
httpGet:
path: /ready
port: 8181
scheme: HTTP
initialDelaySeconds: 0
periodSeconds: 2
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
dnsPolicy: Default
volumes:
- name: config-volume
configMap:
name: coredns
items:
- key: Corefile
path: Corefile
- key: NodeHosts
path: NodeHosts
- name: custom-config-volume
configMap:
name: coredns-custom
optional: true
---
apiVersion: v1
kind: Service
metadata:
name: kube-dns
namespace: kube-system
annotations:
prometheus.io/port: "9153"
prometheus.io/scrape: "true"
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: "CoreDNS"
spec:
type: ClusterIP
selector:
k8s-app: kube-dns
ports:
- name: dns
port: 53
targetPort: 1053
protocol: UDP
- name: dns-tcp
port: 53
targetPort: 1053
protocol: TCP
- name: metrics
port: 9153
protocol: TCP
11 changes: 11 additions & 0 deletions chart/values.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -617,6 +617,17 @@
"type": "object",
"description": "NodeSelector is the node selector to use for coredns."
},
"affinity": {
"type": "object",
"description": "Affinity is the affinity to apply to the pod."
},
"tolerations": {
"items": {
"type": "object"
},
"type": "array",
"description": "Tolerations are the tolerations to apply to the pod."
},
"resources": {
"$ref": "#/$defs/Resources",
"description": "Resources are the desired resources for coredns."
Expand Down
4 changes: 4 additions & 0 deletions chart/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,10 @@ controlPlane:
annotations: {}
# NodeSelector is the node selector to use for coredns.
nodeSelector: {}
# Affinity is the affinity to apply to the pod.
affinity: {}
# Tolerations are the tolerations to apply to the pod.
tolerations: []
# Resources are the desired resources for coredns.
resources:
# Limits are resource limits for the container
Expand Down
6 changes: 6 additions & 0 deletions config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1193,6 +1193,12 @@ type CoreDNSDeployment struct {
// NodeSelector is the node selector to use for coredns.
NodeSelector map[string]string `json:"nodeSelector,omitempty"`

// Affinity is the affinity to apply to the pod.
Affinity map[string]interface{} `json:"affinity,omitempty"`

// Tolerations are the tolerations to apply to the pod.
Tolerations []map[string]interface{} `json:"tolerations,omitempty"`

// Resources are the desired resources for coredns.
Resources Resources `json:"resources,omitempty"`

Expand Down
2 changes: 2 additions & 0 deletions config/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,8 @@ controlPlane:
labels: {}
annotations: {}
nodeSelector: {}
affinity: {}
tolerations: []
resources:
limits:
cpu: 1000m
Expand Down
Loading