/ruby-cute

Ruby Critically Useful Tools for Experiments

Primary LanguageRubyOtherNOASSERTION

Ruby-Cute

Ruby-Cute is a set of Commonly Used Tools for Experiments, or Critically Useful Tools for Experiments, depending on who you ask. It is a library aggregating various Ruby snippets useful in the context of (but not limited to) development of experiment software on distributed systems testbeds such as Grid'5000.

Installation

To install latest release from RubyGems:

$ gem install --user-install ruby-cute

From sources:

$ git clone https://github.com/ruby-cute/ruby-cute
$ cd ruby-cute
$ gem build ruby-cute.gemspec
$ gem install --user-install ruby-cute-*.gem

Then, type the following for having ruby cute in your path (this is only necessary if you want to use interactive mode).

$ export PATH=$PATH:$(ruby -e 'puts "#{Gem.user_dir}/bin"')

If you want to use Ruby-Cute outside Grid'5000 you need to create a configuration file with your credentials. By default you need to create a file called .grid5000_api.yml located in your home directory:

$ cat > ~/.grid5000_api.yml << EOF
  uri: https://api.grid5000.fr/
  username: user
  password: **********
  version: sid
  EOF

For more details have a look at G5K Module.

Overview

Ruby-Cute is structured in different modules that allows you to:

  • communicate with Grid'5000 through the G5K Module. For more details please refer to G5K Module.

  • execute commands in several remote machines in parallel. Two modules are available for that:

An example of use of Ruby-Cute in a real use case is available in Virtualization on Grid'5000

Using pry -- an interactive ruby shell

Sometimes it may be useful to work in interactive mode. For this we can use an interactive ruby shell such as irb that is shipped by default with Ruby, however, we highly recommend to use pry, it features syntax highlighting, method auto completion and command shell integration. For installing pry type the following:

$ gem install pry

or, for installing in the user home directory:

$ gem install --user-install pry

When Ruby-Cute is installed, it provides a wrapper for an interactive shell that will automatically load the necessary libraries. The following will get a pry prompt (if installed).

$ cute
[1] pry(main)>

The variable $g5k is available which can be used to access the Grid'5000 API through the G5K Module. For example, let's request the name of the sites available in Grid'5000. (Before starting be sure to set up a configuration file for the module, please refer to G5K Module)

[2] pry(main)> $g5k.site_uids()
 => ["grenoble", "lille", "luxembourg", "lyon", "nancy", "nantes", "reims", "rennes", "sophia", "toulouse"]

We can get the status of nodes in a given site by using:

[8] pry(main)> $g5k.nodes_status("lyon")
 => {"taurus-2.lyon.grid5000.fr"=>"besteffort", "taurus-16.lyon.grid5000.fr"=>"besteffort", "taurus-15.lyon.grid5000.fr"=>"besteffort", ...}

Within this shell you have preloaded G5K Module, Taktuk and Net::SSH::Multi, you can go directly to their respective documentation to know how to take advantage of them.

Experiment example

The following example shows how to use the G5K Module in an experiment. This example implements the experiment described in MPI on Grid5000.

require 'cute'
require 'net/scp'

g5k = Cute::G5K::API.new()
# We reuse a job if there is one available.
G5K_SITE = "nancy" # the chosen site has to have Infiniband 20G (e.g nancy, grenoble)
if g5k.get_my_jobs(G5K_SITE).empty?
   job = g5k.reserve(:site => G5K_SITE, :resources => "{ib20g='YES'}/nodes=2/core=1",:walltime => '00:30:00', :keys => "~/my_ssh_jobkey" )
else
   job = g5k.get_my_jobs(G5K_SITE).first
end

nodes = job["assigned_nodes"]

grid5000_opt = {:user => "oar", :keys => ["~/my_ssh_jobkey"], :port => 6667 }

machine_file = Tempfile.open('machine_file')

nodes.each{ |node| machine_file.puts node }

machine_file.close

netpipe ="http://pkgs.fedoraproject.org/repo/pkgs/NetPIPE/NetPIPE-3.7.1.tar.gz/5f720541387be065afdefc81d438b712/NetPIPE-3.7.1.tar.gz"

# We use the first node reserved.
Net::SCP.start(nodes.first, "oar", grid5000_opt) do |scp|
   scp.upload! machine_file.path, "/tmp/machine_file"
end

Net::SSH.start(nodes.first, "oar", grid5000_opt) do |ssh|
   ssh.exec!("mkdir -p netpipe_exp")
   ssh.exec!("export http_proxy=\"http://proxy:3128\"; wget -O ~/netpipe_exp/NetPIPE.tar.gz #{netpipe}")
   ssh.exec!("cd netpipe_exp && tar -zvxf NetPIPE.tar.gz")
   ssh.exec!("cd netpipe_exp/NetPIPE-3.7.1 && make mpi")
   ssh.exec("mpirun --mca plm_rsh_agent \"oarsh\" -machinefile /tmp/machine_file ~/netpipe_exp/NetPIPE-3.7.1/NPmpi")
end

g5k.release(job) # Frees resources.

Contact information

Ruby-Cute is maintained by the Algorille team at LORIA/Inria Nancy - Grand Est, and specifically by:

Past contributors include:

Questions/comments should be directed to Lucas Nussbaum and Emmanuel Jeanvoine.