nucypher/nucypher

Repetitive RPC calls by EventScanner

Closed this issue · 3 comments

cygnusv commented

From Alchemy stats of a node running v7.2.0-dev.11, filtering for eth_getLogs exclusively:
image

Observations:

  • 24 calls each 5 minutes (1 each 12.5 seconds). For prior version, 8640 calls to eth_getLogs during last 24h (1 call each 10 seconds). So it seems there's not much improvement yet.
  • For some reason, there are several simultaneous calls scanning the same range, normally 3 times. For example, look at the block ranges in the first 6 eth_getLogs calls: the first 3 have the same range, the next 3 the same, etc. I say simultaneous because, at least according to Alchemy, these calls happen in blocks of 6 that happen roughly at the same time (at least, within a second). The next block of 6 happened one minute before, etc.
derekpierre commented

So the 3 separate getLogs calls actually makes sense. We use the EventScanner to look for 3 different events in a single call to scan(): StartRitual, StartAggregationRound, EndRitual - https://github.com/nucypher/nucypher/blob/v7.2.x/nucypher/utilities/events.py#L239 (events are specified here - https://github.com/nucypher/nucypher/blob/v7.2.x/nucypher/blockchain/eth/trackers/dkg.py#L118).

That being said this code would have been the same in 7.0 (https://github.com/nucypher/nucypher/blob/v7.0.4/nucypher/utilities/events.py#L239) and 7.1 (https://github.com/nucypher/nucypher/blob/v7.1.0/nucypher/utilities/events.py#L239), so relative to those versions the number of RPC calls to getLogs should still go down. Perhaps take a look after a full 24 hours has completed.

derekpierre commented

Each event uses a different topic. We may be able to combine topics into one call since the topics value is allowed to be an Array. Will need to read up on that further.

cygnusv commented

So the 3 separate getLogs calls actually makes sense. We use the EventScanner to look for 3 different events in a single call to scan(): StartRitual, StartAggregationRound, EndRitual - https://github.com/nucypher/nucypher/blob/v7.2.x/nucypher/utilities/events.py#L239 (events are specified here - https://github.com/nucypher/nucypher/blob/v7.2.x/nucypher/blockchain/eth/trackers/dkg.py#L118).

That being said this code would have been the same in 7.0 (https://github.com/nucypher/nucypher/blob/v7.0.4/nucypher/utilities/events.py#L239) and 7.1 (https://github.com/nucypher/nucypher/blob/v7.1.0/nucypher/utilities/events.py#L239), so relative to those versions the number of RPC calls to getLogs should still go down. Perhaps take a look after a full 24 hours has completed.

Right, I've been running the version before #3444 for a bit over 24 hours now. I see 4,317 calls, which implies 3 log calls every minute. This is ~50% than with v7.1.0, btw (as expected). After #3444, the expectation is to reduce it to 1440 eth_getLogs calls per day, for a total of less than 4k RPC calls on Polygon per day.

  • 👍1