/uessc

UEL and Econostat Scraping Software Collection

Primary LanguagePython

UESSC
UEL UEL and Econostat Scraping Software Collection

Requisites

Quickstart

On Linux:

python -m venv env
source env/bin/activate
python -m pip install -r requirements.txt

docker-compose up -d
python src/main.py

On Windows:

python -m venv env
.\env\Scripts\activate
python -m pip install -r requirements.txt

docker-compose up -d
python src\main.py

Example

Project Navigation

  • doc → Documentation
  • src → Source Code
  • src/spiders → Scripts that crawl websites with scrapy
  • src/splash → Scripts that use splash to perform complex interactions with websites

Contributing

Utilize Conventional Commits whenever possible.

Links