omni/poa-bridge

(Feature) Monitoring of the production bridge POA <--> POA20

igorbarinov opened this issue · 2 comments

Before launching "POA <--> POA20" into production mode we need to have metrics and alerts for the "POA <--> POA20" bridge.

The bridge as a service has several independent parts on validators nodes and on two chains. Let's assume that the prodcution version will use three validators. We need:

  • 3 nodes of validators
    • 3 instances of "parity-bridge". No health API (?)
    • 3 instances of Foundation client with an unlocked account. Health API is not available from outside connection
    • 3 instances of Core client with an unlocked account. Health API is not available from outside
  • 3 balances on Foundation network
  • 3 balances on POA network

Open questions:

  • balances of validators on POA on Ethereum.

Do we have a monitor in our monitoring system capable to monitor balances and calculate thresholds?Let's assume that we will change monitors after modifying quotas.

  • exceptions in logs

Do we have exceptions we need to create alerts for?

  • stop/start of services

How to know that a service is stopped?

  • a difference in balances in Home vs Foreign. Can be used to track risks of
    • stopped services (deposited >> minted)
    • unauthorized minting of POA20 (minted >> deposited)

Another service that could be created to check for all deposit transactions to make sure that all tx were actually relayed.
In order to do that, a developer needs to take all Deposit events, build a query and submit it to Foreign Bridge to see if there were relayed tx.
The same needs to be done for Withdrawals on Foreign side.