Install scrapy and scrapy-splash
pip install Scrapy
pip install scrapy-splash
pip install apscheduler
pip install apscheduler
Make sure Docker version >= 17 is installed. Run the contrainer, start the service to listen
sudo docker pull scrapinghub/splash
Add your email info in ./configs/mail.json
sudo python run_server.py
python run_crawler.py