aimer1124/JianshuSpider

Use Node.js,HighChart,BootStrap,Mongo,Cucumber with Gulp to scrapy information from Jianshu.

JianshuSpider

中文文档

The function in this project is all my self mind, because i want to improve my Node.js.So it's open source, welcome PR Fork or Star.

Project

• Source

  • https://github.com/aimer1124/JianshuSpider

• Demand

  • Teambition-JianshuSpider

• UI

  • Home
  • Article
  • Author
  • Collections
  • Search

• ShowCase

  • Home

  • Articles

  • Authors

  • Collections

  • Search

  
  

  

Code

• Start

  • DB: sudo mongod

  • Project: gulp

  • Test: gulp test

• Version

  • Node: 8.11.3

  • Mongo: 4.0.2

• Dependencies

  • Backend

    • express: http://expressjs.com/,node web framework

      • body-parser: Parse HTTP request body.

      • cookie-parser: Parse cookie header and populate req.cookies. See also cookies and keygrip.

      • debug: tiny node.js debugging utility modelled after node core's debugging technique.

      • morgan: HTTP request logger.

      • serve-favicon: Serve a favicon.

    • async: https://github.com/caolan/async,provides straight-forward, powerful functions for working with asynchronous JavaScript

    • cheerio: https://github.com/cheeriojs/cheerio,Fast, flexible, and lean implementation of core jQuery designed specifically for the server.

    • moment: http://momentjs.com,Parse, validate, manipulate, and display dates in JavaScript.

    • mongoose: http://mongoosejs.com/,elegant mongodb object modeling for node.js

    • node-schedule: https://github.com/node-schedule/node-schedule,a flexible cron-like and not-cron-like job scheduler for Node.js.

    • SuperAgent: http://visionmedia.github.io/superagent/,light-weight progressive ajax API crafted for flexibility, readability, and a low learning curve after being frustrated with many of the existing request APIs.

  • FrontEnd

    • HighCharts: http://www.highcharts.com,set up interactive charts in their web pages

    • jade: http://jade-lang.com/, robust, elegant, feature rich template engine for Node.js

    • bootstrap: http://getbootstrap.com/,developing responsive, mobile first projects on the web.

    • bootstrap-table: An extended Bootstrap table with radio, checkbox, sort, pagination, and other added features.

    • bootstrap-select: utilizes Bootstrap's dropdown.js to style and bring additional functionality to standard select elements.

  • Develop

    • browser-sync: http://browsersync.io, Time-saving synchronised browser testing.

    • gulp: http://gulpjs.com,Automate and enhance your workflow

      • gulp-mocha: https://www.npmjs.com/package/gulp-mocha/,a thin wrapper around Mocha and your issue is most likely with Mocha.

      • gulp-nodemon: https://www.npmjs.com/package/gulp-nodemon/, gulp + nodemon + convenience

  • Test

    • cucumber: https://cucumber.io/,Simple, human collaboration

    • selenium-webdriver: http://www.seleniumhq.org/projects/webdriver/,Driving a browser natively as a user would either locally or on a remote machine using the Selenium Server it marks a leap forward in terms of browser automation.

    • chromedriver: https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/chrome/ChromeDriver.html,A WebDriver implementation that controls a Chrome browser running on the local machine.

    • should: https://github.com/shouldjs/should.js, BDD style assertions for node.js

    • SuperTest: https://github.com/visionmedia/supertest, Super-agent driven library for testing node.js HTTP servers using a fluent API

• Change

  • History