metalmatze/alertmanager-bot

Doesn't work with large alerts count

bykvaadm opened this issue · 6 comments

bykvaadm commented

in test prometheus i had about 160 alarts with same type "instance xxx is down".

  • when sending commnd /alerts to bot in tcpdump i see that it goes to alermanager api and gets all alerts, but in telegrab bot replies "failed to list alerts... context canceled"
  • when receiving webhook, bot prints 160 times template text, w/o jobs, labels etc
  • when i commented 90% of failing jobs, bot began to work normally
metalmatze commented

There's a limit to how many characters can be sent in a single Telegram message.
Starting from v0.4.0 onwards the bot automatically truncates messages that are too long.
Which version are you running?

bykvaadm commented

Image: metalmatze/alertmanager-bot:0.4.2

tabnul commented

Same issue here. Same error when loads of alert are firing.
Running latest version.
According to logs, everything is fine.

  • 👍1
hamiteza commented

Same issue for me. there must be problem in fetching alerts from url when the number of alerts is large

tomdynasty commented

I have similar situation. When i keep send /alerts command for a couple of times, sometimes it will show below. anything update for this?

failed to list alerts... Get http://192.168.164.56:9093/api/v1/alerts: dial tcp 192.168.164.56:9093: connect: connection refused
darkobas2 commented

Any workaround for this ?