shakibyzn/divar-posts-crawling

A simple web scraping script to crawl divar posts from Isfahan

divar-post-crawling

Crawling the most popular Iranian consumer-to-consumer sales, divar.ir

---

Table of Contents

• Description
• Techniques
• License
• Author Info

---

Description

In this project, I managed to crawl recent necessities that my people in Isfahan tend to post them on divar.ir. I also used python and the Scrapy library for this task. You can run the script by following the below instructions:
1- Firstly, having scrapy installed is a requirement
2- Secondly, going to the root directory of the project (divar_posts)
3- Finally, running the following command : scrapy crawl divar_spider -o your_output_file_name.json

Techniques

• Web scraping ( web crawling)

Back To The Top

---

License

MIT License

Copyright (c) [2020] [Shakib Yazdani]

Back To The Top

---

Author Info

• Twitter - @iamshakibyz
• LinkedIn - shakib-yazdani

Back To The Top