node-spider-man
Features
- Transform the page into JSON format response according to the rules
- Use fetchQueue to manage the fetchRequests in the order
- Support sync and async mode.
- sync: Execute each fetchTask after the previous one finished
- async: Execute each fetchTask after the delayFetch time
- Support queue finished callback
- Support to PAUSE or CONTINUE the spider's work(>=0.3.0)
How to use
npm install node-spider-man --save
var spiderMan = require('node-spider-man');
Demo
The demo shows the experiement that we can grab what we want from the internet using node-spider-man
Lisence @Apache-2.0
Copyright to @2012-2016 Staydan.com