Documentation or examples desired
jonerer opened this issue · 1 comments
Hello!
Thanks for an awesome project! We would really like to use this, but a bit unsure how everything fits together.
We're using the prometheus operator (prometheus-community/kube-prometheus-stack chart, which I think is the most popular one) to generate grafana/prometheus etc into a namespace. And we'd like to put dashboards in configmaps. We'd also like to have subfolders within grafana (in the GUI. unsure what that would mean in terms of disk storage).
From reading the PRs I feel like this is probably possible, but it would be greatly appreciated if there were some kind of documentation or examples. There is currently one example for the simplest case linked from the README.md.
I tried to create a dashboard like so:
apiVersion: v1
kind: ConfigMap
metadata:
name: kafkatopicdashboard
namespace: monitoring
labels:
grafana_dashboard: "1"
annotations:
k8s-sidecar-target-directory: "/tmp/dashboards/kafka"
data:
kafka_topics.yaml: |-
{
"annotations": {
[...]
Which did end up getting placed in the correct place in an automatically create /tmp/dashboards/kafka folder, but never showed up on the grafana UI. What's missing?
EDIT: actually, after removing the annotation, (which caused kafka_topics.yaml to be placed in /tmp/dashboards/) it never showed up on the UI either. curious... But the other files (which came with the kube-prometheus-stack) do show up in the UI.
I also added debug logging to grafana, and it logged something like this:
grafana-sc-dashboard [2021-11-29 09:45:34] Config for cluster api loaded...
grafana-sc-dashboard [2021-11-29 09:45:34] Unique filenames will not be enforced.
[... other dashboards loaded]
grafana-sc-dashboard [2021-11-29 09:45:39] Working on configmap monitoring/kafkatopicdashboard
grafana-sc-dashboard [2021-11-29 09:45:39] File in configmap kafkatopics.yaml ADDED
# so it's loaded in sidecar! but then in grafana:
grafana t=2021-11-29T09:46:08+0000 lvl=dbug msg="Scheduling update" logger=alerting.scheduler ruleCount=0
grafana t=2021-11-29T09:46:08+0000 lvl=dbug msg="Start walking disk" logger=provisioning.dashboard type=file name=sidecarProvider path=/tmp/dashboards
grafana t=2021-11-29T09:46:08+0000 lvl=dbug msg="saving new dashboard" logger=provisioning.dashboard type=file name=sidecarProvider provisioner=sidecarProvider file=/tmp/dashboards/k8s-resources-namespace.json fo
grafana t=2021-11-29T09:46:08+0000 lvl=dbug msg="Extracted alerts from dashboard" logger=alerting.extractor alertCount=0
# and a bunch of others, but NOT my kafkatopics.yaml. it's not found in the grafana container.
Thanks in advance