Scraping Techcrunch.com web pages and dumping into PostgresSQL This project based on scrapy package, So, first of all you have to install scrape package in your computer.
- Crate database and table- you can find table structure in directory “poster_table/techCrunch.sql”
- Change database and connection settings in file “settings.py”
- To run program go to the root directory in your terminal then run “scrapy crawl hse”