Crawler cases demo

That the project written especially for HollyJS 2017 in Moscow.

npm test

//OR to run specific demo
npm test ./src/puppeteer-adblock-demo

Each demo written as separate applications. So fill free to change and mutate it. To tests your own tho. All demo are under the ./src directory

Demo list

cheerio-demo - how to use cheerio on server side
puppeteer-adblock-demo - how to create adblock logic for puppeteer
puppeteer-cheerio-demo - wrap puppeteer content to cheerio
puppeteer-demo - the minimal code needs to run puppeteer
puppeteer-download-demo - how to download file using puppeteer
puppeteer-linkedin-cookie-demo - how to scrape site using cookie againts the login - steps.
puppeteer-login-github - common steps to login somewhere and receive some userfull private information.
puppeteer-optimization-demo - experiments to optimize puppeteer load speed
request-demo - common logic to to receive static html
request-puppeteer-together-demo - how to switch requestjs to puppeteer runtime
save-from-demo - show how to fight with ubnormal behavior sites, without puppeteer using sanbox on serverside.
scheme-demo - examples to check SEO optimized sites
seasonvar-demo - example with complex workflow to download video from common website.

npm install
mv ./src/config.template.js ./src/config.js

The config used to set some private infos. Like github login / password. Site tokens.