gbif/pipelines

Create monitoring to track broken CLIs

Opened this issue · 1 comments

muttcg commented

In most cases, CLI breaks without warning. For example, the Rabbit client may lose connection. It would be useful to receive a notification when this happens to any of the ingestion or validator CLIs, or to add a watchdog that can automatically restart a failed CLI.

There's monitoring for when the crawl queue becomes too long, but with the separation of crawling and pipelines that will no longer alert to crashed pipelines CLIs.