Ankr specific timestamp reader
Closed this issue · 1 comments
miohtama commented
Block timestamps are needed for mass event ingestion with eth_getLogs
. We need to map block number -> timestamp.
Currently we offer two ways to do it
- extract_timestamp_json_rpc - the naive method
- Calls
eth_getBlockByNumber
for the range of all blocks (start, end) we need to get events from - Very slow, because to read just one uint64 timestamp avg.
eth_getBlockByNumber
returns 8 MB data and we often need to read 100 - 1,000 blocks once - Also often breaks QuickNode, because their API throttling limits is low
- However it works on any EVM node
- Calls
- GraphQL-based timestamp reading
- Reads all timestamps for a block range in a single call... you can get 1,000s of timestamps with a single API request
- Uses GoEthereum standard
/grapql
endpoint - Best for in-house hosted nodes
- Does not work with any commercial SaaS node provider because they do not support
/graphql
To be able to have commercial SaaS node providers as a fallback option for JSON-RPC, we need to be able to read block timestamps so that their nodes do not complain.
- Commercial SaaS vendors provide their incompatible proprietary APIs for tasks like reading multiple block headers
- This is exactly like of proprietary low-quality APIs you can think of
As first commercial SaaS vendor to pilot out we'd like to try Ankr
- ank_getBlocks
proprietary API can read blocks in batch up to 20 blocks at a time
- Ankr claims to support up to API 1500 request/minute
- Ankr offer pay-as-you-go plan so it is easy get started with them
For the task
- Create
reader/ankr.py
module - Create
extract_timestamps_ankr_get_block
method - Create
has_ankr_support
method ineth_defi.chain
- Create
AnkrReorganisationMonitor
that is similar toGraphQLReorganisationMonitor
- using the underlyingextract_timestamps_ankr_get_block
- Add a compatibility check in
create_reorganisation_monitor
that will automatically createAnkrReorganisationMonitor
if we are connected to Ankr node - Create an integration test that take
JSON_RPC_ANKR_PRIVATE
as the environment variable input to test these against a live Ankr plan- See TVL tests for inspiration
- See also test_chain
AlexTheLion123 commented
Closed due to no longer working with Ankr