Crawling the most popular Iranian consumer-to-consumer sales, divar.ir
In this project, I managed to crawl recent necessities that my people in Isfahan tend to post them on divar.ir. I also used python and the Scrapy library for this task.
You can run the script by following the below instructions:
1- Firstly, having scrapy installed is a requirement
2- Secondly, going to the root directory of the project (divar_posts)
3- Finally, running the following command : scrapy crawl divar_spider -o your_output_file_name.json
- Web scraping ( web crawling)
MIT License
Copyright (c) [2020] [Shakib Yazdani]
- Twitter - @iamshakibyz
- LinkedIn - shakib-yazdani