gonwi/celo-etl

Python scripts for ETL (extract, transform and load) jobs for Celo blockchain blocks and transactions.

Celo ETL

Celo ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.

Celo ETL is a fork of

Quickstart

Install Celo ETL:

pip3 install celo-etl

Export blocks and transactions (Schema, Reference):

> celoetl export_blocks_and_transactions --start-block 0 --end-block 100000 \
--provider-uri https://alfajores-forno.celo-testnet.org --blocks-output blocks.csv --transactions-output transactions.csv

For the latest version, check out the repo and call

> pip3 install -e . 
> python3 celoetl.py

Running Tests

> pip3 install -e .[dev,streaming]
> export ETHEREUM_ETL_RUN_SLOW_TESTS=True
> pytest -vv

Running Tox Tests

> pip3 install tox
> tox

Running in Docker

1. Install Docker https://docs.docker.com/install/

2. Build a docker image

    > docker build -t celo-etl:latest .
    > docker image ls
   

3. Run a container out of the image

    > docker run -v $HOME/output:/celo-etl/output celo-etl:latest export_all -s 0 -e 100000 -b 50000 -p https://alfajores-forno.celo-testnet.org