Concurrently download http requests in a synchronous environment. Basically this is an em-http-request wrapper that slots into a normal ruby script.
This would be useful inside a rails environment or a rake task.
Add this line to your application's Gemfile:
gem 'concurrent_downloader'
And then execute:
$ bundle
Or install it yourself as:
$ gem install concurrent_downloader
Check out spec/lib/concurrent_downloader_spec.rb for lots of usage examples. Here is one:
queue = []
100.times do |i|
queue << {
:method => "post",
:path => "/test"
}
end
responses = []
ConcurrentDownloader.process_queue!(queue, :host => "http://localhost", :concurrent_downloads => 10) do |queue_item, response|
if response.status == 200
responses << response
end
end
# if all responses are 200
responses.size.should == 100
The test suite passes in both 1.8.7 and 1.9.3
- Fork it
- Create your feature branch (
git checkout -b my-new-feature) - Commit your changes (
git commit -am 'Added some feature') - Push to the branch (
git push origin my-new-feature) - Create new Pull Request
Oliver Clarke ollie@clarketus.net