Spawn
=====
This plugin provides a 'spawn' method to easily fork OR thread long-running sections of
code so that your application can return results to your users more quickly.
This plugin works by creating new database connections in ActiveRecord::Base for the
spawned block.
The plugin also patches ActiveRecord::Base to handle some known bugs when using
threads (see lib/patches.rb).
Usage
-----
Here's a simple example of how to demonstrate the spawn plugin.
In one of your controllers, insert this code (after installing the plugin of course):
spawn do
logger.info("I feel sleepy...")
sleep 11
logger.info("Time to wake up!")
end
If everything is working correctly, your controller should finish quickly then you'll see
the last log message several seconds later.
If you need to wait for the spawned processes/threads, then pass the objects returned by
spawn to Spawn::wait(), like this:
N.times do |i|
# spawn N blocks of code
spawn_ids[i] = spawn do
something(i)
end
end
# wait for all N blocks of code to finish running
wait(spawn_ids)
If you want your forked child to run at a lower priority than the parent process, pass in
the :nice option like this:
spawn(:nice => 7) do
do_something_nicely
end
By default, spawn will use the fork to spawn child processes. You can configure it to
do threading either by telling the spawn method when you call it or by configuring your
environment.
For example, this is how you can tell spawn to use threading on the call,
spawn(:method => :thread) do
something
end
When using the :thread setting, spawn will check to make sure that you have set
allow_concurrency=true in your configuration. If you want this setting then
put this line in one of your environment config files:
config.active_record.allow_concurrency = true
If it is not set, then spawn will raise an exception.
To (optionally) configure the spawn method in your configuration, add a line to
your configuration file(s) like this:
Spawn::method :thread
If you don't set any configuration, the :method will default to :fork. To
specify different values for different environments, pass the environment as
the 2nd argument:
Spawn::method :fork, 'production'
Spawn::method :yield, 'test'
This allows you to set your production and development environments to use different
methods according to your needs.
Forking vs. Threading
---------------------
There are several tradeoffs for using threading vs. forking. Forking was chosen as the
default primarily because it requires no configuration to get it working out of the box.
Forking advantages:
- more reliable? - the ActiveRecord code is generally not deemed to be thread-safe.
Even though spawn attempts to patch known problems with the threaded implementation,
there are no guarantees. Forking is heavier but should be fairly reliable.
- keep running - this could also be a disadvantage, but you may find you want to fork
off a process that could have a life longer than its parent. For example, maybe you
want to restart your server without killing the spawned processes.
We don't necessarily condone this (i.e. haven't tried it) but it's technically possible.
- easier - forking works out of the box with spawn, threading requires you set
allow_concurrency=true. Also, beware of automatic reloading of classes in development
mode (config.cache_classes = false).
Threading advantages:
- less filling - threads take less resources... how much less? it depends. Some
flavors of Unix are pretty efficient at forking so the threading advantage may not
be as big as you think... but then again, maybe it's more than you think. ;-)
- debugging - you can set breakpoints in your threads
Acknowledgements
----------------
This plugin was initially inspired by Scott Persinger's blog post on how to use fork
in rails for background processing.
http://geekblog.vodpod.com/?p=26
Further inspiration for the threading implementation came from Jonathon Rochkind's
blog post on threading in rails.
http://bibwild.wordpress.com/2007/08/28/threading-in-rails/
Also thanks to all who have helped debug problems and suggest improvements including:
Ahmed Adam, Tristan Schneiter, Scott Haug, Andrew Garfield, Eugene Otto, Dan Sharp,
Olivier Ruffin
Garry Tan, Matt Jankowski (Rails 2.2.x fixes)
Tim Kadom, Mauricio Marcon Zaffari, Danial Pearce, Hongli Lai, Scott Wadden (passenger fixes)
<your name here>
Copyright (c) 2007-present Tom Anderson (tom@squeat.com), see LICENSE