Scrapper for wall street journal
Setup and Run:
- Make sure the inputs are set correctly in inputs.py.
- Make sure the path to firefoxdriver is correct.
- Install required packages by running the command python setup.py
- Run by simply executing python main.py
This scraper is an updated version of Muheeb's code. The code had several problems related to requests timing out due to multithreading issues (multiple erors) and issues with new chrome driver versions. Also prevent ip bans.
Credit goes to Muheeb!