this crawler was initially developed to crawl Yelp London's Coffeshops and Restaurants
The crawler was mostly developed By Mr. Aliraza Imani and I (@jeemist) supervised the development process for keeping the project structure proffesional and easy to scale
We do not take responsibility for any harmful usage of this repository considering that our main purpose for fetching Yelp's data was for academic research in Machine Learning Field
If you want to use this code, make sure of making a virtual enviroment under the env directory
our main configs are under config.py file and our database models are Models directory
to start using make sure you set your database authentication information in config.py and run create_database.py
In order to keep this projec scalable we made the Crawler directory for our different Crawlers (based on Adapter Design Pattern)
If you wanted to develop this repository for your own, try to keep your database code in DataProviders directory's files
any comments on this code or our approach to this problems is welcome; you can contact us via:
email => arash.jeem@gmail.com
Github => jimist
Telegram => jeemist
Linkedin => jimist
Twitter => jeemist
thank your for visiting this repository, if you found it helpful don't hesitate for a star :-D