crawlTutiempo

A simple crawler for the history weather data in the en.tutiempo.net using scrapy framework.

Requirements

pip install scrapy
git clone https://github.com/Ethan-Zhang/crawlTutiempo.git

Start crawling site with the args of city, start_year, end_year

cd crawlTutiempo
scrapy crawl tutiempo -a city=Beijing -a start_year=2014 -a end_year=2015 -o item.json

View the history weather data in item.json

If you don't know the city name, crawl the ctiy name list

scrapy crawl cityname

View city name list in city.json