tradingstrategy-ai/web3-ethereum-defi

Ankr specific timestamp reader

Closed this issue · 1 comments

Block timestamps are needed for mass event ingestion with eth_getLogs. We need to map block number -> timestamp.

Currently we offer two ways to do it

  • extract_timestamp_json_rpc - the naive method
    • Calls eth_getBlockByNumber for the range of all blocks (start, end) we need to get events from
    • Very slow, because to read just one uint64 timestamp avg. eth_getBlockByNumber returns 8 MB data and we often need to read 100 - 1,000 blocks once
    • Also often breaks QuickNode, because their API throttling limits is low
    • However it works on any EVM node
  • GraphQL-based timestamp reading
    • Reads all timestamps for a block range in a single call... you can get 1,000s of timestamps with a single API request
    • Uses GoEthereum standard /grapql endpoint
    • Best for in-house hosted nodes
    • Does not work with any commercial SaaS node provider because they do not support /graphql

To be able to have commercial SaaS node providers as a fallback option for JSON-RPC, we need to be able to read block timestamps so that their nodes do not complain.

  • Commercial SaaS vendors provide their incompatible proprietary APIs for tasks like reading multiple block headers
  • This is exactly like of proprietary low-quality APIs you can think of

As first commercial SaaS vendor to pilot out we'd like to try Ankr
- ank_getBlocks proprietary API can read blocks in batch up to 20 blocks at a time
- Ankr claims to support up to API 1500 request/minute
- Ankr offer pay-as-you-go plan so it is easy get started with them

For the task

  • Create reader/ankr.py module
  • Create extract_timestamps_ankr_get_block method
  • Create has_ankr_support method in eth_defi.chain
  • Create AnkrReorganisationMonitor that is similar to GraphQLReorganisationMonitor - using the underlying extract_timestamps_ankr_get_block
  • Add a compatibility check in create_reorganisation_monitor that will automatically create AnkrReorganisationMonitor if we are connected to Ankr node
  • Create an integration test that take JSON_RPC_ANKR_PRIVATE as the environment variable input to test these against a live Ankr plan

Closed due to no longer working with Ankr