alerta/alerta

post_receive | take_action | update attributes behaviour

Opened this issue · 1 comments

Hi,
I have added few custom plugins in my alerta.

  1. In one plugin, in the post receive, I am sending the alert to a Kafka queue so that it can be processed by an external system.
  2. Then I have this another plugin where I have implemented the take_action. The take_actiion is invoked by the external system after it processes the alert that was went to Kafka. This system sends some attributes which I update on alerta.
  3. The external system is also directly calling alerta to create an alert where it populates these attributes.
  4. Then I have this another plugin, where in post receive, which is processing the incoming alerts based on the attributes that were sent by the external system (in Point 2) and also in the alert it creates directly in alert via the POST /alert API (in Point 3).

What I am seeing is a bit inconsistent behaviour where sometimes I am loosing the attributes sent by external system.

I have some doubts. Hoping to get some help and clarity

Question 1 - architecturally is it OK to update the alert attributes in post_receive or is it just meant to notify external system.

Question 2 - as the same alert can come again, which was sent to Kafka, can the use of take_actiin and poat_rexeive cause race conditions or inconsistent behaviour ?

Would appreciate any help. Thanks !

@utsukprani, Im not entirely sure, but it sounds like you are hitting the same problem that I had.. there was a bug that is the "last" plugin was returning "null" (eg. alert not modified), then it would not update commit any of the previous plusgins changes to alert.. You can checkout #1798

I hope that solves your problem. Else let me know, as I am also planning to send data of to Kafka in the near future.. ;-)