/integral-web-scraper

Downloads a set of web pages from your URLs list, saves them into a local SQLite file and calls your transformer class to deal with them, saving the results into the DB. Can further export transformed HTML or create an ePub.

Primary LanguageRuby

Integral::WebScraper

Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file lib/integral/web_scraper. To experiment with that code, run bin/console for an interactive prompt.

TODO: Delete this and the text above, and describe your gem

Installation

Add this line to your application's Gemfile:

gem 'integral-web_scraper'

And then execute:

bundle install

Usage

TODO: Write usage instructions here

Development

After checking out the repo, run bin/setup to install dependencies. Then, run rake spec to run the tests. You can also run bin/console for an interactive prompt that will allow you to experiment.

To install this gem onto your local machine, run bundle exec rake install. To release a new version, update the version number in version.rb, and then run bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the .gem file to rubygems.org.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/[USERNAME]/integral-web_scraper.