Pmux is a lightweight file-based MapReduce system, written in Ruby. Applying the philosophy of Unix pipeline processing to distributed computing on a GlusterFS cluster, pmux provides a tool capable of handling large amounts of data stored in files.
- ruby 1.8.7, 1.9.2 or higher
- msgpack-rpc
- net-ssh, net-scp
- gflocator
- GlusterFS 3.3.0 or higher, native client (FUSE)
on all GlusterFS server nodes:
gem install pmux
on the GlusterFS client node
gem install pmux
gem install gflocator
sudo gflocator
Each of GlusterFS server nodes, SSH with authentication key is required.
show status
$ pmux --status
host0.example.com: pmux 0.1.0, num_cpu=8, ruby 1.9.3
show status of remote machine
$ pmux --status -h host1.example.com
host1.example.com: pmux 0.1.0, num_cpu=2, ruby 1.9.3
distributed grep
$ pmux --mapper="grep PATTERN" /glusterfs/xxx/*.log