Implement streaming_service.py
medvedev1088 opened this issue · 0 comments
medvedev1088 commented
Streaming service
- Continuously polls for new blocks using Ethereum JSON RPC API (e.g. every 10 seconds).
- Outputs data to GooglePubSubItemExporter, which publishes data to PubSub.
- Saves the last synchronized block to
last_sync_block.txt
file on every period.
This script can be used as a basis https://github.com/blockchain-etl/ethereum-etl/blob/feature/streaming/stream.py
================================
PubSub publisher guide https://cloud.google.com/pubsub/docs/publisher
There is some code in this fork https://github.com/airswap/ethereum-etl/commit/cc7f86e139d317452c9cf0c975b99feb6ed8d7a0#diff-a8e6a25bc32464a59378bac1befac9eb
TODO: Consider using geth subscriptions to handle chain reorganizations https://github.com/ethereum/go-ethereum/wiki/RPC-PUB-SUB#newheads