sensu-plugins/sensu-plugins-mailer

client alerts silenced for sensu mailer handler

malawson opened this issue · 3 comments

malawson commented

I have recently been working with Sensu, specifically the sensu-plugins-mailer
https://github.com/sensu-plugins/sensu-plugins-mailer
i have a question about the handler-mailer.rb integration.

i installed it as such
(DEV) dsensu0 handlers # cat mailer_handler.json
{
"handlers": {
"mailer": {
"mail_from": "dsensu0@meetup.com",
"mail_to": "arnaud@meetup.com",
"smtp_address": "mail.dev.hq.meetup.com",
"smtp_port": "25",
"smtp_domain": "dev.hq.meetup.com",
"type": "pipe",
"command": "/usr/local/sensu-server/etc/sensu/handlers/handler-mailer.rb",
"subscriptions": {
"sensu_server": {
"mail_to": "arnaud@meetup.com"
}
}
}
}
}

and i have these check definitions

(DEV) dsensu0 check_definitions # cat check_sensu_server.json
{
"checks": {
"ghe_primary_freespace_on_datadisk_stats_check": {
"type": "metric",
"command": "/usr/local/sensu-client/etc/sensu/plugins/graphite/check-data.rb --server sensu.int.meetup.com:80 -t collectd192_168_0_31collectd.df-data-user.df_complex-free --below --warn 50252338619 --crit 50252338619 --from -1mins",
"interval": 10,
"subscribers": ["sensu_server"],
"handlers": ["mailer"],
"occurences": 1,
"refresh": 1
},
"ghe_primary_cpu_percent_sysusage_stats_check": {
"type": "metric",
"command": "/usr/local/sensu-client/etc/sensu/plugins/graphite/check-data.rb --server sensu.int.meetup.com:80 -t collectd192_168_0_31collectd.cpu.percent-system --warn 90.0 --crit 95.0 --from -5mins",
"interval": 10,
"subscribers": ["sensu_server"],
"handlers": ["mailer"]
"occurences": 1,
"refresh": 1
}
}
}

But then i get these errors(client alerts silenced) in the logs - although one of the checks is returning CRITICAL - hence i am not getting any emails; not sure what i am missing here. What does this error mean and how could i get around that?

{"timestamp":"2016-06-28T16:18:56.838459-0400","level":"info","message":"handler output","handler":{"mail_from":"dsensu0@meetup.com","mail_to":"arnaud@meetup.com","smtp_address":"mail.dev.hq.meetup.com","smtp_port":"25","smtp_domain":"dev.hq.meetup.com","type":"pipe","command":"/usr/local/sensu-server/etc/sensu/handlers/handler-mailer.rb","subscriptions":{"sensu_server":{"mail_to":"arnaud@meetup.com"}},"name":"mailer"},"output":["client alerts silenced: dsensu0/ghe_primary_cpu_percent_sysusage_stats_check\n"]}

{"timestamp":"2016-06-28T16:25:29.603794-0400","level":"info","message":"handler output","handler":{"mail_from":"dsensu0@meetup.com","mail_to":"arnaud@meetup.com","smtp_address":"mail.dev.hq.meetup.com","smtp_port":"25","smtp_domain":"dev.hq.meetup.com","type":"pipe","command":"/usr/local/sensu-server/etc/sensu/handlers/handler-mailer.rb","subscriptions":{"sensu_server":{"mail_to":"arnaud@meetup.com"}},"name":"mailer"},"output":["client alerts silenced: dsensu0/ghe_primary_freespace_on_datadisk_stats_check\n"]}

i looked at the api and noticed that no clients or checks were silenced

(DEV) dsensu0 ~ # sensu-cli stash list
no values for this request
0 total items
(DEV) dsensu0 ~ #

I also used the API to silence this node/client then unsilenced but still the errors persist. I wonder if there is something else overriding the silencing mechanism....? Any help would be greatly appreciated. Thanks.

malawson commented

i am using this version of sensu
sensu-server-0.20.5
could that be an issue for the mailer handler?

malawson commented

ok i figured it out. first of all my api config wasn't correct/explicitely stating bind, and port values.
also the config file for the sensu mailer had to be modified to be of this format
#15