Logstash not receiving collectd notifications.
Closed this issue · 7 comments
I have setup collectd with certain thresholds to send notifications to logstash on things like high load. Without using this codec I can see the notification string on stdout from logstash, along with all the stats. If I enable the codec all the stats get through but notifications appear to be dropped without warning. Are notifications currently supported?
What this suggests is that the notifications are somehow not strictly adhering to the Collectd binary protocol, or that the collectd codec—which is set to capture the message packet—somehow is not properly receiving/parsing the message.
I will experiment with this as I get time, but have no quick or easy answers for you right now :(
Another possibility is that the message is too long. A collectd packet is, by definition, 1540 bytes. If your message is too big, it might be dropping it. You might try running logstash with the --debug flag enabled and see what comes out.
Thanks for your help. Running with --debug unfortunately doesn't seem to give any output relating to the notifications, only the normal stats messages.
It was dropping the message because I hadn't told the codec to keep messages with a "severity" field. Stay tuned for an update to the plugin.
@gb123451991 you should be able to upgrade to the latest version of the collectd codec with:
bin/plugin update logstash-codec-collectd
The output looks like this:
$ bin/plugin update logstash-codec-collectd
Updating logstash-codec-collectd
Updated logstash-codec-collectd 0.1.9 to 0.1.10
Fantastic, I can now see my notifications coming through into kibana, many thanks! Do you have any idea when this update could get included in a Logstash release?
Whenever the next point release comes out, likely. However, this is why we've made plugins independently update-able--so you wouldn't have to wait for a full release to get a fix for a plugin.