neogeek/raspar

⛏ A simple to use Promise-based web scraper with local caching.

raspar

A simple to use Promise-based web scraper with local caching.

Usage

import fetch from 'raspar';

const contents = await fetch('http://www.google.com/humans.txt');

console.log(contents);

Options

import fetch from 'raspar';

const options = {
  cacheDirectory: 'temp/cache/',
  requestOptions: {
    headers: {
      'User-Agent': 'request'
    },
    method: 'POST'
  },
  ttl: 1800
};

const contents = await fetch('http://www.google.com/humans.txt', options);

console.log(contents);

Name	Description	Default Value
cacheDirectory	Directory to store cache.	temp/cache/
requestOptions	Request options object. Read more github.com/node-fetch/node-fetch	{}
ttl	TTL (Time to live) in seconds.	1800