websolr-gem
A Websolr gem to make RSolr/Sunspot-based applications faster
This gem makes some changes to the connection class that RSolr defaults to. Namely, it configures Faraday to use the Typhoeus adapter, which is significantly faster (especially when dealing with SSL/TLS). It also adds support for HTTP Keep-Alive and Websolr's Advanced Authentication system.
Preliminary tests of the gem found latency and speed increased dramatically:
| Reindex 50K Documents (batch size of 50) |
1K Random Searches | 1K Random Operations (read/update) |
|
|---|---|---|---|
| No Gem | 496.8 s | 196.7 s | 63.6 s |
| With Gem | 279.1 s | 44 s | 17.6 s |
| Speed Increase | 78% | 347% | 261% |
| Latency Decrease | 43.8% | 77.6% | 72.3% |
Another benefit of using Typhoeus is access to the Hydra, which makes it possible to make requests in parallel, theoretically allowing one to speed up indexing significantly.
Installation
Add the gem to your Gemfile and run bundle install:
gem 'websolr'This will add Websolr and Typhoeus to your app. The gem creates an initializer that sets up and manages the connection when the application loads.
Todo:
- Test with apps that do not use Sunspot
- Support for getting settings via YAML?
- Support for read-only, or different credentials for read/write ops?
- Support for queueing failed requests?
- Support for throttling?
- Support for parallel reindexing?
- Test coverage
- Documentation