From 6483ad02df17c98ae759c612015a6bcb712564c9 Mon Sep 17 00:00:00 2001 From: QianChenglong Date: Sun, 24 May 2020 10:29:42 +0800 Subject: [PATCH 1/6] docs: add x509 for faq --- FAQ.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/FAQ.md b/FAQ.md index 302917018..ba80c65aa 100644 --- a/FAQ.md +++ b/FAQ.md @@ -72,6 +72,55 @@ Default 60 seconds, can be changed using `metrics-resolution` flag. We are not r ## Known issues +#### Unable to authenticate the request due to an error: x509: certificate signed by unknown authority + +``` +E0524 01:37:36.055326 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority +``` + +Because Metrics Server is an aggregated server for kube-apiserver. So as an aggregated server need sets `--requestheader-client-ca-file` for validate request which send by kube-apiserver. [headrequest(front-proxy)](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy) + + +For cluster created by kubeadm: + +1. Check kube-apiserver front-proxy args(/etc/kubernetes/manifests/kube-apiserver.yaml) + +``` +- --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt +- --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key +``` + +2. Prepare front-proxy-ca for Metrics Server + +``` +kubectl -nkube-system create configmap front-proxy-ca --from-file=front-proxy-ca.crt=/etc/kubernetes/pki/front-proxy-ca.crt -o yaml | kubectl -nkube-system replace configmap front-proxy-ca -f - +``` + +3. Configure your Metrics Server deployment: + +``` + - args: + - --requestheader-client-ca-file=/ca/front-proxy-ca.crt // ADD THIS! + - --cert-dir=/tmp + - --secure-port=4443 + - --kubelet-insecure-tls // ignore validate kubelet x509 + - --kubelet-preferred-address-types=InternalIP // using InternalIP to connect kubelet + + volumeMounts: + - mountPath: /tmp + name: tmp-dir + - mountPath: /ca // ADD THIS! + name: ca-dir + + volumes: + - emptyDir: {} + name: tmp-dir + - configMap: // ADD THIS! + defaultMode: 420 + name: front-proxy-ca + name: ca-dir +``` + #### Network problems Metrics server needs to contact all nodes in cluster to collect metrics. Problems with network would can be recognized by following symptoms: From cff7187e0dd5c4dee9f1fa7b3e957a1527fc8e76 Mon Sep 17 00:00:00 2001 From: QianChenglong Date: Mon, 25 May 2020 20:11:48 +0800 Subject: [PATCH 2/6] Update FAQ.md Co-authored-by: Marek Siarkowicz --- FAQ.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/FAQ.md b/FAQ.md index ba80c65aa..4e65010e1 100644 --- a/FAQ.md +++ b/FAQ.md @@ -72,8 +72,9 @@ Default 60 seconds, can be changed using `metrics-resolution` flag. We are not r ## Known issues -#### Unable to authenticate the request due to an error: x509: certificate signed by unknown authority +#### Incorrectly configured front-proxy certificate +Metrics Server needs to validate requests coming from kube-apiserver. You can recognize problems with front-proxy certificate configuration if you observe line below in your logs: ``` E0524 01:37:36.055326 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority ``` From 9cd2bceb920659e600f18629a86ddb83c31e83fa Mon Sep 17 00:00:00 2001 From: QianChenglong Date: Mon, 25 May 2020 20:12:38 +0800 Subject: [PATCH 3/6] Update FAQ.md Co-authored-by: Marek Siarkowicz --- FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/FAQ.md b/FAQ.md index 4e65010e1..ac5717b1d 100644 --- a/FAQ.md +++ b/FAQ.md @@ -79,7 +79,7 @@ Metrics Server needs to validate requests coming from kube-apiserver. You can re E0524 01:37:36.055326 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate signed by unknown authority ``` -Because Metrics Server is an aggregated server for kube-apiserver. So as an aggregated server need sets `--requestheader-client-ca-file` for validate request which send by kube-apiserver. [headrequest(front-proxy)](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy) +To fix this problem you need to provide kube-apiserver proxy-client CA to Metrics Server under `--requestheader-client-ca-file` flag. You can read more about this flag in [Authenticating Proxy](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#authenticating-proxy) For cluster created by kubeadm: From 782d79a1f24ac9a7ddb164fb695f4873625657d1 Mon Sep 17 00:00:00 2001 From: QianChenglong Date: Mon, 25 May 2020 20:13:09 +0800 Subject: [PATCH 4/6] Update FAQ.md Co-authored-by: Marek Siarkowicz --- FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/FAQ.md b/FAQ.md index ac5717b1d..08ac5519d 100644 --- a/FAQ.md +++ b/FAQ.md @@ -84,7 +84,7 @@ To fix this problem you need to provide kube-apiserver proxy-client CA to Metric For cluster created by kubeadm: -1. Check kube-apiserver front-proxy args(/etc/kubernetes/manifests/kube-apiserver.yaml) +1. Find your front-proxy certificates by checking arguments passed in kube-apiserver config (by default located in /etc/kubernetes/manifests/kube-apiserver.yaml) ``` - --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt From 89b5430968216d8d95ce0ce59063e7881732988f Mon Sep 17 00:00:00 2001 From: QianChenglong Date: Mon, 25 May 2020 20:13:23 +0800 Subject: [PATCH 5/6] Update FAQ.md Co-authored-by: Marek Siarkowicz --- FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/FAQ.md b/FAQ.md index 08ac5519d..f7d5a1eee 100644 --- a/FAQ.md +++ b/FAQ.md @@ -91,7 +91,7 @@ For cluster created by kubeadm: - --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key ``` -2. Prepare front-proxy-ca for Metrics Server +2. Create configmap including `front-proxy-ca.crt` ``` kubectl -nkube-system create configmap front-proxy-ca --from-file=front-proxy-ca.crt=/etc/kubernetes/pki/front-proxy-ca.crt -o yaml | kubectl -nkube-system replace configmap front-proxy-ca -f - From 9db63393773bf4e3f85567f10fd1dedcb4aa65a9 Mon Sep 17 00:00:00 2001 From: QianChenglong Date: Mon, 25 May 2020 20:13:41 +0800 Subject: [PATCH 6/6] Update FAQ.md Co-authored-by: Marek Siarkowicz --- FAQ.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/FAQ.md b/FAQ.md index f7d5a1eee..ac460b1a6 100644 --- a/FAQ.md +++ b/FAQ.md @@ -97,7 +97,7 @@ For cluster created by kubeadm: kubectl -nkube-system create configmap front-proxy-ca --from-file=front-proxy-ca.crt=/etc/kubernetes/pki/front-proxy-ca.crt -o yaml | kubectl -nkube-system replace configmap front-proxy-ca -f - ``` -3. Configure your Metrics Server deployment: +3. Mount configmap in Metrics Server deployment and add `--requestheader-client-ca-file` flag ``` - args: