No prebuilt alert in prometheus for ORG exceeding threshold
Closed this issue · 7 comments
Hi Team,
In Grafana we can see there is ORG summary to to view the ORG Memory Total and used.
Like that am trying to create a custom rule under prometheus alerts if ORG mem utilized more than 75% will be receivng alert hich will be helpful in taking proactive action.
But am unable to get a parameter similar to below for alert creating and using this in prometheus alerts receving parse error.
It would be great if i can get some help , ideas ,insight on this.Thanks!
Json from Grafana:
"expr": "sum(avg(cf_application_memory_mb{environment="$environment",deployment="$bosh_deployment",organization_name="$cf_organization_name"} * on (application_id) cf_application_instances{environment="$environment",deployment="$bosh_deployment",organization_name="$cf_organization_name",state="STARTED"}) by(application_id))",
"intervalFactor": 2,
"legendFormat": "Used",
"refId": "A",
"step": 4
},
{
"expr": "avg(cf_organization_total_memory_mb_quota{environment="$environment",deployment="$bosh_deployment",organization_name=~"$cf_organization_name"})",
"interval": "",
"intervalFactor": 2,
"legendFormat": "Quota",
"metric": "",
"refId": "B",
"step": 4
Regards,
Prabhu
Hi @spkrsna ,
I think your problem is that the values like $environment are replaced by grafana with template variables. That means in Prometheus you can use something like
avg (cf_organization_total_memory_mb_quota) by (organization_id) as a starting point.
Wonderful! thanks @benjaminguttmann-avtq for the swift response!! The query given by you is working!
Will try the same on avg app instance memory for a single ORG.
Once again Thanks!
sum(avg(cf_application_memory_mb{environment="$environment",deployment="$bosh_deployment",organization_name="$cf_organization_name"} * on (application_id) cf_application_instances{environment="$environment",deployment="$bosh_deployment",organization_name="$cf_organization_name",state="STARTED"}) by(application_id))
I tried/trying to convert this bigger query but no luck only facing parse errors , your help on this will be helpful. Thanks!
Note: For this param it's saying function is not there.
Hi @spkrsna ,
I am wondering if that query is actually correct bceause the * on (application_id) seems odd to me. in general, make sure to either replace the $* variables with actual values or completly remove them to get the expression evaluated for all availables applications
In grafana dashboard by default we can see CF ORG summary dashboard and for that Memory graph having Used,Quota and JSON query for that one is showing the query i shared.
This issue is stale because it has been open 60 days with no activity. Comment or this will be closed in 5 days.
This issue was automatically closed because it has been stalled for 5 days with no activity.