Tiamat¶ ↑

Summary¶ ↑

Automatic parallelism across multiple cores and machines: a plugin for Pure.

Synopsis¶ ↑

require 'tiamat/autoconfig'
require 'benchmark'

mod = Pure.define do
  def total(left, right)
    left + right
  end

  def left
    (1..500_000).inject(0) { |acc, n| acc + n }
  end

  def right
    (1..500_000).inject(0) { |acc, n| acc + n }
  end
end

# compute using two threads
puts Benchmark.realtime { mod.compute(2).total }  # => 0.4432079792022705

# compute using two local Ruby interpreters
Tiamat.open_local(2) {
  puts Benchmark.realtime { mod.compute.total }   # => 0.2420041561126709
}

Install¶ ↑

% gem install tiamat

Or from the (non-gem) .tgz package,

% rake install

Description¶ ↑

Tiamat is a worker plugin for the pure functional package (Pure). It links Ruby interpreters together with DRb, forming a back-end for Pure’s parallelizing engine.

Tiamat does not modify any of the standard classes.

Tiamat has been tested on MRI 1.8.6, 1.8.7, 1.9.1, 1.9.2, and jruby-1.4.

Links¶ ↑

Home: quix.github.com/tiamat
Feature Requests, Bug Reports: github.com/quix/tiamat/issues
Manual Download: github.com/quix/tiamat/archives/master
Repository: github.com/quix/tiamat

Adding `require` Paths to Local Servers¶ ↑

require 'tiamat/autoconfig'
require 'pure/dsl'

require 'matrix'
require 'complex'

mod = pure do
  def f(x, y)
    x + y
  end
end

a = Matrix[[1, 2], [3, 4]]
b = Matrix[[1, Complex(0, 1)], [Complex(0, -1), 1]]

Tiamat.open_local(2, 'matrix', 'complex') {
  puts mod.compute(:x => a, :y => b).f  # => Matrix[[2, 2+1i], [3-1i, 5]]
}

Running Remote Servers¶ ↑

First install Tiamat on the remote machines. To start the server,

% tiamat-server [drb address] [compiler name] [requires]

For example,

% tiamat-server druby://192.168.4.1:27272 Pure::Compiler::RubyParser pure/compiler/ruby_parser matrix complex

This will make the Matrix and Complex classes available on the remote server.

To connect (continuing the example in the synopsis),

Tiamat.open_remote('druby://192.168.4.1:27272', 'druby://192.168.4.2:27272') {
  puts Benchmark.realtime { mod.compute.total }
}

Security¶ ↑

Running tiamat-server is among the most insecure things you can possibly do with a computer. The purpose of tiamat-server is to execute arbitrary code. Only run it inside a secure network or private tunnel.

Configuration¶ ↑

Configuration consists of two attributes: Pure.parser for extracting method definitions and Tiamat.compiler for converting the extracted data into a callable Ruby object. (Though strictly speaking Tiamat.compiler is only necessary for launching local servers.)

require 'tiamat/autoconfig' finds the optimal parser-compiler pair for the running version of Ruby. A configuration may also be chosen manually, e.g. require 'tiamat/config/ruby_parser'.

Restrictions¶ ↑

RubyParser + Ruby2Ruby provides the ability to extract code, transform the AST, send it over the wire, and reconstruct it on another machine. Therefore the syntax of files which contain pure blocks is limited to the syntax supported by RubyParser.

About DRbConnError¶ ↑

require 'tiamat/autoconfig'
require 'pure/dsl'

Tiamat.open_local(2) {
  pure do
    def hello(out)
      out.puts("hello")
    end
  end.compute(:out => STDOUT).hello  # => raises DRb::DRbConnError
}

This fails because we are trying to send STDOUT to another Ruby interpreter. Some Ruby objects have no Marshal.dump, such as IO instances (like STDOUT) and Procs. To compensate, the remote DRb server tries to phone home with a proxy object but no local server is running.

If you are interested in a quick fix, try placing this at the beginning of your code:

require 'drb'
DRb.start_service

See the DRb documentation for more information.

Tiamat does not start a local DRb service by default because the passing of undumpable arguments to a pure function is likely to be an error, as the generated proxies are not thread-safe.

Author¶ ↑

James M. Lawrence <quixoticsycophant@gmail.com>

License¶ ↑

Copyright (c) 2009 James M. Lawrence.  All rights reserved.

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation files
(the "Software"), to deal in the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

quix/tiamat