- Python version >= 3.9
Run in terminal:
python -m venv venv
For fish shell ;)
. venv/bin/activate.fish
Install poetry
pip install poetry
Go to crawler catalog
cd crawler
Run crawler commands
scrapy crawl global_vol1
- Add tests/performence tests
- Go deeper into EDA about links
- Optimize scrapy