Creating an External Caching application which reads data from a Full Node. Ethereum.
Create a caching layer at RPC for data queries from smart contract, the cache should be updated when write transactions are processed by the Node. Two possible ways to do this:
- Caching at RPC layer inside of a Node and an outside load balancer which directs calls to proper load balancer where cached data might be available
- Implement a caching layer outside of Node, which encapsulates all RPC calls and caches data. (Basically receives calls, processes it in some way, calls an RPC internally, gets output, processes it in some way, sends data back to caller)
Problem: Querying blockchain data can be slow
Solution: Increase data availability and speed up queries.
This repository focuses on the second solution.
- Permanent database - PostgresSQL
- Caching layer - Redis (In memory key value store)
- Server - Express
- Language - Typescript
- Linting - ESLint
- Formatting - Prettier
We would provide a HTTP url, this will act as a HTTPProvider in web3.js/ether.js. This would be client side. People can use this url to get in touch with blockchain network.
const web3 = new Web3(new Web3.providers.HttpProvider('HTTP_URL'))
When the user calls any function. The call will go to the server and will be processed in the following manner -
- First the call will be scanned thorugh cache layer(Redis), if there is any data in cache → return data.
- Second the call will be scanned through the database(Postgres), if there is any data in db → return data and write in cache.
- Lastly the call will be made to a full node which will give us the data and will write that in cace and db.
One more layer is working in the background that is the subscription layer to the blockchain network. This layer periodically fetches data from blockchain network and stores it in the database.
- Clone the repository
- Use your favourite package manager to install dependencies
npm i
oryarn
orpnpm i
- To start the server you can use
pnpm start
- To start the subscription layer use
pnpm subs
Note: All JS and TS files in this repo (expect api/*.ts, indes.ts, subscription.ts) are used for testing this server.