/scrapy_example

This repository store some example to learn scrapy better

Primary LanguagePythonMIT LicenseMIT

Overview

This repository store some example to learn scrapy better, which include tutorial, zhihu, cnbeta, xiaobaihe, douban, coursera

Requirements

  • Python: 2.7.8
  • Scrapy: 0.24
  • System: Mac OS X 10.10.1

Install

$ git clone git@github.com:Andrew-liu/scrapy_example.git

Usage

You can use the six scrapy simply, just to do below:

$ cd zhuhu  #into some one scarpy
$ scrapy crwal zhihu  #xiaobaihe or tutorial or cnbeta or xiaobaihe, coursera

If you want to store the result into the format of json, just to do :

$ scrapy crwal zhihu -o filename.json

More Detail

License

Copyright (c) 2014 Andrew Liu

Licensed under the MIT License