error: metrics not available yet #247
Comments
|
Run Add |
|
Does your apiserver open the aggregation layer? --requestheader-client-ca-file=/etc/kubernetes/certs/proxy-ca.crt --requestheader-allowed-names=aggregator --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --proxy-client-cert-file=/etc/kubernetes/certs/proxy.crt --proxy-client-key-file=/etc/kubernetes/certs/proxy.keyand how about your rolebinding? |
|
its probably the self signed cert on your cluster. As in apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: metrics-server containers:
- name: metrics-server
image: k8s.gcr.io/metrics-server-amd64:v0.3.3
imagePullPolicy: Always
args: [ "--kubelet-insecure-tls" ]
volumeMounts:
- name: tmp-dir
mountPath: /tmp |
|
I had the "error: metrics not available yet" message. @latchmihay pointer fixed it. Thank you. |
|
I have similar problem. It says metrics not available for node or pod. It still reports Here is my configuration. |
|
I got it working. |
|
@sreedharbukya I have the same error and try all the method above mentioned but still not work, may u post ur solution? |
|
@regardfs, Please follow below instructions. If you are using kops for creating the cluster. edit following part you have run following commands
Watch out your logs. This should resolve the issue. Next is update your metrics-server deployments to following. |
|
@sreedharbukya Thank you! I have an AWS EKS Cluster and default installation of metrics server from Helm. |
|
As pointed in the documentation, using That said, how do we deploy metrics-server securely using |
|
I have the same error here "metrics not available yet" .. The pod logs is showing this message:
|
|
I'm facing the same problem for nodes, but not for pods. Kubelet Insecure TLS is set : no change. $ kubectl top pod -n default
NAME CPU(cores) MEMORY(bytes)
resource-consumer-5766d495c6-bm47z 15m 5Mi
resource-consumer-5766d495c6-rtkzk 6m 5Mi
unprivileged-hello-world-658f6f7f49-dq6js 0m 6Mi $ kubectl top node
error: metrics not available yet$ kubectl get deploy metrics-server -o yaml | grep -B6 insecure - command:
- /metrics-server
- --logtostderr
- --cert-dir=/tmp
- --secure-port=8443
- --kubelet-preferred-address-types=InternalIP
- --kubelet-insecure-tls |
|
@serathius : Here are logs from metrics-server, filtered for matching "node" : https://gist.github.com/ledroide/0cbc6750cab7d6ae0371b88c97aee44e Exemple of what I'm seeing in these logs : |
|
Can you verify that kubelet is exposing cpu metrics? You can skip using |
|
I have copied the output of the command Output looks good to me. |
|
Looks like node exposes Can you verify that |
|
Of course, still failing : $ kubectl top node
error: metrics not available yet |
|
Last check before I would need to deepdive into code. Need to eliminate if the problem is with Are you running |
|
I think we reach the problem here : the nodes list is empty By the way, I use kubernetes 1.16.2, and no heapster |
|
Solved. $ kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes/kube-poc-compute3
{"kind":"NodeMetrics","apiVersion":"metrics.k8s.io/v1beta1","metadata":{"name":"kube-poc-compute3","selfLink":"/apis/metrics.k8s.io/v1beta1/nodes/kube-poc-compute3","creationTimestamp":"2019-11-12T09:42:21Z"},"timestamp":"2019-11-12T09:41:57Z","window":"30s","usage":{"cpu":"849755531n","memory":"8506004Ki"}} |
|
Solved by following steps :
|
|
Hello, I have the same problem... |
|
I had the same problem with Kubernetes 1.16 installed by kops on AWS. I found that a heapster is still installed on the server. When I removed the heapster (it was installed using helm), the command |
|
Thanks @latchmihay your solution works for me |
|
The problem happened to me and none of the suggestion above worked because the problem wasn't metrics. The problem was because the 443 and 80 was busy. I had apache server running. If yo have any application running on those ports try to kill them before anything. |
didn't had to update the deployment tough. --insecure tls and kops update cluster did the trick for me thank you |
This setup worked for me, except I left the secure-port at 4443, which was how it was installed via the deployment script. Now I can get top on both node and pods. |
I had the same problem, and I had fix it by add args as you, thankyou |
|
@sreedharbukya thanks a lot!!! |
do u have solve it? |
|
I recommend to create separate issue, as in this issue we didn't even manage to fix original problem that reporter had.
Signifies that there is a problem with metrics-server, but says nothing about what's the problem. Meaning that this issue isn't really helpful for others (e.g. @ledroide just recreated MS). Could try some suggestions from this issue, still it would be better to tackle each setup separately. |
|
I had a similiar problem on minikube running on windows 10 home. Your tutorial helped me. |
It worked, Thx. |
|
I had the same problem，this is my config: [root@k8s-master k8s]# kubectl version [root@k8s-master k8s]# kubectl top po heapster logs: W0924 01:44:25.002300 1 manager.go:152] Failed to get all responses in time (got 0/3) [root@k8s-master data]# netstat -nultp is namespace question? |
|
if you use bitnami metrics-server, enable this even k8s > 1.18 |
|
I only needed resource "helm_release" "metrics-server" {
create_namespace = true
namespace = "platform"
name = "metrics-server"
repository = "https://charts.helm.sh/stable"
chart = "metrics-server"
set {
name = "args"
value = "{\"--kubelet-preferred-address-types=InternalIP\"}"
}
} |
|
Edit: this report definitely does not belong here, sorry for contributing to the noise I was able to resolve it by switching from the bitnami helm chart for metrics server, to kustomized deploy of metrics-server from this repo, with very similar to "test" kustomize manifests. Thank you for providing this. I am on kubeadm This is in conflict with the docs in Else I ran into While that's not the subject of this issue report, this issue is one of the top results for "error: Metrics API not available" and it helped me, so I am highlighting it here.
|
|
I also was getting the same problem, by running all the YAML files in metrics-server the problem has resolved:
|
|
I have the same issue but a different cause. I enabled Is this a known issue? |
Thanks you, i'm using minikube 1.22 with kubernetes version 1.21.2 and this was the solution. I enabled creation of apiServer in values and it works fine!.
|
|
If you are installing a bit higher version then update the deployment as, |
and others
ENV:
kubernetes: v1.13.2
docker: 18.06.1-ce
system: ubuntu 18.04
architecture: arm-64
metrics-server: v0.3.1
Before I apply the 1.8+/ development, I try the command
kubectl top nodes
Then I got the error:
Error from server (NotFound): the server could not find the requested resource (get services http:heapster:)
After that I download the heapster from https://github.com/kubernetes-retired/heapster and apply the deploy.
The I run the
kubectl apply -f 1.8+/
and check the pods by
kubectl get pods --all-namespaces
The result are as follows:
NAMESPACE NAME READY STATUS RESTARTS AGE
default back-flask-86cbc75cd9-jdgwr 1/1 Running 0 3h53m
default demo-flask-756f45c879-68fvk 1/1 Running 0 3h53m
kube-system coredns-86c58d9df4-c4bv4 1/1 Running 0 4h16m
kube-system coredns-86c58d9df4-cbdvq 1/1 Running 0 4h16m
kube-system etcd-robbon-vivobook 1/1 Running 0 4h15m
kube-system heapster-f64999bc-2klfs 1/1 Running 0 91s
kube-system kube-apiserver-robbon-vivobook 1/1 Running 0 4h15m
kube-system kube-controller-manager-robbon-vivobook 1/1 Running 0 4h15m
kube-system kube-flannel-ds-amd64-8fxvb 1/1 Running 0 4h13m
kube-system kube-flannel-ds-amd64-xjsk6 1/1 Running 0 4h14m
kube-system kube-proxy-4smxt 1/1 Running 0 4h13m
kube-system kube-proxy-vv5xh 1/1 Running 0 4h16m
kube-system kube-scheduler-robbon-vivobook 1/1 Running 0 4h15m
kube-system kubernetes-dashboard-697f86d999-jk4b7 1/1 Running 0 3h33m
kube-system metrics-server-7d8db6b444-tzgbk 1/1 Running 0 19s
kube-system monitoring-grafana-57cbc5c659-zj6fq 1/1 Running 0 91s
kube-system monitoring-influxdb-7f897d5cc8-c2fsh 1/1 Running 0 91s
But when I run kubectl top nodes to check, I get another error:
error: metrics not available yet
So, what's wrong with this?
The text was updated successfully, but these errors were encountered: