Web scraping and crawling aren't illegal by themselves. After all, you could scrape or crawl your own website, without a hitch. The problem arises when you scrape or crawl the website of somebody else, without obtaining their prior written permission, or in disregard of their Terms of Service (ToS). https://benbernardblog.com/web-scraping-and-crawling-are-perfectly-legal-right/
wesamkiwan/web_crawler
Web scraping and crawling aren't illegal by themselves. After all, you could scrape or crawl your own website, without a hitch. The problem arises when you scrape or crawl the website of somebody else, without obtaining their prior written permission, or in disregard of their Terms of Service (ToS).
Python