paper_scraper pip3 install scrapy cd paper_scraper # https://docs.scrapy.org/en/latest/intro/tutorial.html#how-to-run-our-spider scrapy crawl semantic_scholar Do not mess with DOWNLOAD_DELAY in settings.py, it keeps you out of trouble.