- Install and set up Scrapy:
pip3 install scrapy
cd
into the folder- Run crawler by
scrapy crawl anicrawler
- Use
-o
to redirect output, e.g.scrapy crawl anicrawler -o a.json
- Install and set up Selenium:
pip3 install selenium
- For drawing features, install and set up networkx:
pip3 install networkx
- Run any written script, e.g.
python3 find_reposts.py
- Install and set up Selenium and tweepy:
pip3 install selenium tweepy
- Run any wirtten script, e.g.
python3 random_crawler.py