/benandjerrys_iron_scraper

This web scraper uses iron.io to generate a database of store locations for Ben & Jerry's ice cream flavors

Primary LanguageRuby

Ben & Jerry's Iron.io Scraper

This web scraper uses iron.io to generate a database of store locations for Ben & Jerry's ice cream flavors. There is also a scraper that obtains detailed information about each of the flavours. Together, these two scrapers provide the data for the Ben & Jerry's API.

The Iron.io scraper is based on the original Ben & Jerry's scraper.

Running the Scraper Locally

  • Install the Ruby gem dependencies: mechanize, pg, active_record (included with Rails) and iron_mq.
  • Create a message queue on IronMQ with the zip codes you want to scrape data for.
  • Get the Token and Project ID from iron.io and set them as ENV['IRON_TOKEN'] and ENV['IRON_PROJECT_ID'].
  • Rename database.yml.example to database.yml and replace development values with those of your Heroku database.
  • Then run this script from your terminal: development/run_scraper_locally