grafana-net-2-dashboard.json and grafana-net-737-dashboard.json is obsolete ,it cause no graph in grafana
justlooks opened this issue · 6 comments
i installed kubernetes-prometheus use
kubectl create -f manifests-all.yaml
and it seems all fine ,but no graph in grafana ,why?
[root@k8s-master prometheus]# kubectl version
Client Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.0", GitCommit:"d3ada0119e776222f11ec7945e6d860061339aad", GitTreeState:"clean", BuildDate:"2017-06-29T23:15:59Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"7", GitVersion:"v1.7.0", GitCommit:"d3ada0119e776222f11ec7945e6d860061339aad", GitTreeState:"clean", BuildDate:"2017-06-29T22:55:19Z", GoVersion:"go1.8.3", Compiler:"gc", Platform:"linux/amd64"}
[root@k8s-master prometheus]# kubectl get pods --namespace monitoring -o wide
NAME READY STATUS RESTARTS AGE IP NODE
alertmanager-1345290433-51n0m 1/1 Running 0 1h 172.16.1.221 k8s-node1
grafana-core-277836579-bpndq 1/1 Running 0 1h 172.16.1.222 k8s-node1
kube-state-metrics-458253108-7mnkl 1/1 Running 0 1h 172.16.1.224 k8s-node1
kube-state-metrics-458253108-p3xsm 1/1 Running 0 1h 172.16.0.107 k8s-master
node-directory-size-metrics-3w1pl 2/2 Running 0 1h 172.16.0.105 k8s-master
node-directory-size-metrics-78jkk 2/2 Running 0 1h 172.16.2.68 k8s-node2
node-directory-size-metrics-j3zj4 2/2 Running 0 1h 172.16.1.223 k8s-node1
prometheus-core-3823466589-j6g4b 1/1 Running 0 1h 172.16.0.108 k8s-master
prometheus-node-exporter-3jvcm 1/1 Running 0 1h 10.161.233.80 k8s-node2
prometheus-node-exporter-bvc4l 1/1 Running 0 1h 10.165.97.219 k8s-node1
prometheus-node-exporter-dwh0k 1/1 Running 0 1h 10.132.41.234 k8s-master
and i checked prometheus metrics on grafana ,for example
in kubernetes pods resources panel , one monitor option call "Memory Working Set"
the query is 'sum (container_memory_working_set_bytes{id="/",instance=~"^$instance$"}) / >sum (machine_memory_bytes{instance=~"^$instance$"}) * 100'
and in prometheus metrics defination i can not find "machine_memory_bytes" or "container_memory_working_set_bytes" ,is the grafana-net-737-dashboard.json file not update timely?
Same problem, no data in graphs
try to add following scrape_configs for cAdvisor into Prometheus configmap (copied from here):
scrape_configs:
- job_name: kubernetes-nodes-cadvisor
scrape_interval: 10s
scrape_timeout: 10s
scheme: https # remove if you want to scrape metrics on insecure port
tls_config:
ca_file: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
kubernetes_sd_configs:
- role: node
relabel_configs:
- action: labelmap
regex: __meta_kubernetes_node_label_(.+)
# Only for Kubernetes ^1.7.3.
# See: https://github.com/prometheus/prometheus/issues/2916
- target_label: __address__
replacement: kubernetes.default.svc:443
- source_labels: [__meta_kubernetes_node_name]
regex: (.+)
target_label: __metrics_path__
replacement: /api/v1/nodes/${1}/proxy/metrics/cadvisor
metric_relabel_configs:
- action: replace
source_labels: [id]
regex: '^/machine\.slice/machine-rkt\\x2d([^\\]+)\\.+/([^/]+)\.service$'
target_label: rkt_container_name
replacement: '${2}-${1}'
- action: replace
source_labels: [id]
regex: '^/system\.slice/(.+)\.service$'
target_label: systemd_service_name
replacement: '${1}'
did it work? I still have the same problem after chaning Prometheus configmap. any clue?
Same problem here guys. Any idea?