price-finder

!! THIS PROJECT IS DEPRECATED !!

Please reach out if you'd like to support building a new version, thank you!

Finds the price of retail items online, either by scraping the web page or through product APIs.

Quick Examples

Find an item's current price online

const PriceFinder = require('price-finder');
const priceFinder = new PriceFinder();

// Atoms for Peace : Amok  (from Amazon)
const uri = 'http://www.amazon.com/Amok/dp/B00BIQ1EL4';
priceFinder.findItemPrice(uri, function(err, price) {
    console.log(price); // 8.91
});

Find the price, name, and category of an item online

const PriceFinder = require('price-finder');
const priceFinder = new PriceFinder();

// Plants vs Zombies  (from Google Play)
let uri = 'https://play.google.com/store/apps/details?id=com.popcap.pvz_na';
priceFinder.findItemDetails(uri, function(err, itemDetails) {
    console.log(itemDetails.price);    // 0.99
    console.log(itemDetails.name);     // Plants vs. Zombies™
    console.log(itemDetails.category); // Mobile Apps
});

// Don't Starve  (from Steam)
uri = 'http://store.steampowered.com/app/219740';
priceFinder.findItemDetails(uri, function(err, itemDetails) {
    console.log(itemDetails.price);    // 14.99
    console.log(itemDetails.name);     // Don't Starve
    console.log(itemDetails.category); // Video Games
});

Price Finder Documentation

Configuration Options
API
- findItemPrice(uri, callback)
- findItemDetails(uri, callback)
Debugging Price Finder
Supported Sites
Contributing
- Tests
- Adding Sites

Configuration Options

When creating a new PriceFinder object, a configuration object can be specified. The following options are configurable:

retryStatusCodes : An array of status codes (Numbers) which when returned from the page scrape request, will trigger a retry request (meaning it will attempt to scrape the page again). Defaults to [503].
retrySleepTime : If a retry status code is returned from a page scrape request, this is the amount of time (in milliseconds) that the code will sleep prior to re-issuing the request. Defaults to 1000 (ms).

For example:

const PriceFinder = require('price-finder');

const priceFinder = new PriceFinder({
  retrySleepTime: 2000,
});

API

findItemPrice(`uri`, `callback`)

Given a uri (that is for a supported site), this function will scrape the page and attempt to find the current price listed on the page, sending it to the callback. The callback's arguments are:

error : If an error occurred during processing, this will contain the error information. If no errors occurred, this will be null.
price : The current price of the item listed on the page (a Number).

findItemDetails(`uri`, `callback`)

Given a uri (that is for a supported site), this function will scrape the page and attempt to find the item details listed on the page, sending it to the callback. The callback's arguments are:

error : If an error occurred during processing, this will contain the error information. If no errors occurred, this will be null.
itemDetails : This object contains three things:
- price : The current price of the item listed on the page (a Number).
- name : The name of the product (if supported by the site implementation).
- category : The category of the product (if supported by the site implementation).

Debugging Price Finder

The debug package is used within price-finder to output debugging information useful in tracking down any potential issues. To enable, export the DEBUG environment variable set to price-finder* to pick up all files (or include a specific library to only enable a certain module). For example:

$ DEBUG=price-finder* node app.js

Supported Sites

The current supported sites are listed below.

Amazon
Best Buy
- API support is available but requires an API key. To enable, set the BESTBUY_KEY environment variable to the value of the API key. For more information on how to obtain an API key, refer to the Best Buy developer documentation.
Crutchfield
eBags
GameStop
GOG
Google Play
Greenman Gaming (*)
Infibeam (*)
Newegg
Nintendo
PriceMinister (*)
Snapdeal
Sony Playstation
Steam
Target
Walmart

(*Support unknown at this time)

Don't see your site listed? Please consider contributing to the project!

Contributing

The price-finder project is a Node.js module, so before cloning the repository make sure node is installed. Once cloned, install dependencies by issuing:

$ yarn

Tests

The project uses the Mocha test framework along with the Should assertion library for tests (please add tests for any new features).

Unit Tests

To run the unit tests execute:

$ yarn test

These tests can be run in watch mode, listening for any file changes and re-running when that occurs. To do so execute:

$ yarn test:watch

End To End Tests

End-to-end tests exist which will test the price-finder module using real URIs, scraping the pages to verify the code works correctly. Because these tests can take a while to run, debug logging has been enabled in the npm script.

Note that these tests should be run on a limited basis while coding since some sites have been known to throw up CAPTCHA's after repeated, automated page requests.

To execute the end to end tests run:

$ yarn test-e2e

If you would like to run a single end to end test (rather than all of them), use the test-e2e-single script. For example:

$ yarn test-e2e-single test/e2e/amazon-uris-test.js

Adding Sites

This project was built to easily drop in support for new sites. The site-manager iterates over all files contained within the sites directory, and adds it to the list of available sites. When a request is issued to price-finder to look up a price, it asks each site if the uri is supported by the site, and if so, uses that site to find the price (or name, category).

Yeoman Generator

A generator exists to create the site, along with the site's unit and end to end test. For more information on this generator, please see the project page: https://github.com/dylants/generator-price-finder-site

For reference, the site interface is:

class Site {
  constructor(uri) {
    // init Site, save off uri
  }

  /**
   * Returns the URI used to find the page data
   * (most likely the same URI used in constructing this Site)
   *
   * @return {String} The URI used to find the page data
   */
  getURIForPageData();

  /**
   * Returns true if the page data is JSON
   *
   * @return {Boolean} true if the page data is JSON, false otherwise
   */
  isJSON();

  /**
   * Returns the price found on the page
   *
   * @param  {Object} $/pageData jQuery object used to search the page, or
   *                             JSON page data if JSON based site
   * @return {String}            The price found on the page
   */
  findPriceOnPage($);

  /**
   * Returns the category of the item found on the page
   *
   * @param  {Object} $/pageData jQuery object used to search the page, or
   *                             JSON page data if JSON based site
   * @return {String}            The category found on the page
   */
  findCategoryOnPage($);

  /**
   * Returns the name of the item found on the page
   *
   * @param  {Object} $/pageData jQuery object used to search the page,
   *                             or JSON page data if JSON based site
   * @param  {String} category   The product's category
   * @return {String}            The name found on the page
   */
  findNameOnPage($, category);

  /**
   * Returns true if this site supports the incoming URI
   *
   * @param  {String}  uri The URI to test
   * @return {Boolean}     true if this Site supports the URI, false otherwise
   */
  static isSite(uri);
}

Etc

Contributors

TheRat3000/price-finder