/WebCrawlerForNYT

Web crawler for NYT news

Primary LanguagePython

WebCrawlerForNTYTimes

How does program work?

How to run?

python main.py

Tips:

Single thread works well and multi-thread will stack after some time

Output files will be placed under ./result

Parameters (self.progressx) in Entry Class is required to be modified to mark the start index of remaining articles.