This is a Python-based khc(KuberhealthyCheck) for the Kuberhealthy synthetic monitoring operation.
It will query a Kubernetes cluster for certain annotated services.
Once such a service is found, it will be resolved against all the existing DNS pods in the the cluster.
If all nameservers resolve the service correctly, it will move on to find the next annotated service.
Otherwise, a failure will be reported back to the Kuberhealthy master and the khc will exit non-zero.
While there are existing internal/external DNS KuberhealthyChecks upstream, they didn't meet our needs for the following reasons:
- Services aren't checked for IP consistency between what the nameservers resolve them to and what the IPs they actually map to.
- Only one service can be checked per
khc. However, for our purposes, we opted to change the model to be pulled-based via an annotation to simplify adoption across our many teams.
First, create the necessary RBAC permissions necessary for the khc.
Then, apply the khc in internal-dns-khc.yaml.
That's it.
Pods should begin spawning and any service in the cluster that contains the defined ANNOTATION will be checked.
All tags are automatically built and pushed to Dockerhub.
| Variable | Required? | Default | Description |
|---|---|---|---|
| ANNOTATION | Yes | None |
Annotation that will need to be present in services for the check to pick them up |
| DNS_NAMESPACE | Yes | None |
Namespace where the DNS pods are located |
| DNS_NODE_SELECTOR | Yes | None |
The label belonging to the DNS pods |
| MAX_SERVICES | No | 30 |
This is the max number of services that will be returned by the k8s API per call (for pagination purposes). |
If you're confused about what your DNS_NAMESPACE and DNS_NODE_SELECTOR should be, simply test the following kubectl command:
kubectl get pods -o wide --namespace=<DNS_NAMESPACE> -l <DNS_NODE_SELECTOR>
If that returns all the DNS pods for your cluster, then you have the right values.