/okra-scrape

Here will test your skills at using Puppeteer and scraping web pages.

Primary LanguageJavaScript

Okra Scrape Pages

Requirements:

  • Nodejs >=18.3
  • docker and compose --> Only if you are to use the dockerized mongo instance

Install packages

$ npm install

Running the scraping script on cli

$ node utils/scrape_organization.js

Running the live server

$ npm start

The server can be accessed on http://localhost:3000/

This will return the data of the scrapped website, the website to scrap is set via an environment variable SCRAPE_URL that can be modified in ./config/index.js

To use a local docker instance as your mongo DB, simply run the following to spin up a fast instance

$ docker compose up

This exposes mongoDB on port 27017