jovijovi/ether-crawler

A transaction crawler for the Ethereum ecosystem.

Ether Crawler

Transactions crawler for the Ethereum ecosystem.

Features

• Dump transfer coin transaction to database
• Support PostgreSQL, MySQL and SQLite
• Custom crawler concurrency options

Supported Chains

• Ethereum
• Polygon

Development Environment

• typescript 4.8.4
• node v16.18.0
• ts-node v10.9.1
• yarn v1.22.19

Quick Guide

• Install dependency

  yarn

• Build code

  Install all dependencies and compile code.

  make build

• DEV environment dependency

  • Setup

    cd ./devenv
    ./dev.sh up
    cd ..

  • Shutdown

    ./dev.sh down

• Build docker image

  make docker

• Run

  • Params

    • --config Config filepath. Example:

      ts-node ./src/main/index.ts --config ./conf/app.config.yaml

  • Run code directly by ts-node

    yarn dev-run --config ./conf/app.config.yaml

  • Run compiled code by node

    yarn dist-run --config ./conf/app.config.yaml

• Clean

  make clean

Roadmap

• Support more transaction type
• Improve performance
• UT
• Resume job from interruption

License

MIT