## Copyright (C) 2004-2007 Javier Fernández Baldomero, Mancia Anguita López ## ## This program is free software ## ## You should have received a copy of the GNU General Public ## License along with MPITB/Octave; see the file COPYING. If not, ## write to the Free Software Foundation, Inc., 51 Franklin Street, ## Fifth Floor, Boston, MA 02110-1301, USA. ## ## (or see URL http://www.fsf.org/licensing/licenses/gpl.html) *************** * MPI TOOLBOX * *************** OCT bindings for [LAM 7.1.3 / Open-MPI 1.2.3] calls (MPI 1.2 - MPI 2.0) under Octave [2.1.73 / 2.9.12] Linux (tested with Kernel 2.6.20, Fedora Core 6) LAM/MPI License also included f/reference, see file LAM_license.txt (or see URL http://www.lam-mpi.org/community/license.php) Open-MPI "New BSD License" also included f/reference, see file OMPI_license.txt (or see URL http://www.open-mpi.org/community/license.php) (or see URL http://www.opensource.org/licenses/bsd-license.php) *********** * INSTALL * *********** The preferred setup is a user's own local copy of [LAM/OMPI] & MPITB, since - usually not many cluster users will use MPITB (usually just one) - MPITB requires a dynamically linked MPI, which _was_ not usual rpm packages used to be configured for only static libmpi.a recently, FC6 comes with a dynlibs lam-libs rpm package fortunately, Open-MPI compiles to dynamic by default - you will not depend on lazy sysadmins who do not care We have seen clusters where MPI existed only in the headed node or where the version on headed-node was newer than on headless ones or even of a different implementation (LAM on headed / OMPI elsewhere) that can cause difficult-to-diagnose malfunction specially if path/env is not the same for interactive and remote shells BTW, your octave must support DLDs, since the whole MPITB consists of DLD (.oct) files. Type "octave_config_info" at the Octave prompt and look for the "DLD" flag. ------------------- - Install summary - ------------------- $ cd ~/octave # choose normal ubication # tar zxvf <mpitb_file>.tgz # Decompress and untar gzip files (old) $ tar jxvf <mpitb_file>.tar.bz2 # Decompress and untar bzip2 files (current) $ cd mpitb # enter the new subdir $ cat BASHRC_stub # study the suggested .bashrc stubs # configure your account so that both Octave & (LAM / OpenMPI) # are correctly included in PATH and LD_LIBRARY_PATH # cyclelogin (logout/login) for changes to take effect # ############### try it out #### $ octave ... Minimal test: MPI_Init, [a b]=MPI_Initialized, MPI_Finalize, [a b]=MPI_Finalized ... octave:1> MPI_Init # working? ans = 0 # great! octave:2> # ############### if it doesn't work, you might want to try some other # precompiled Tbx (remember to fix your .bashrc accordingly) $ ln -s DLD-oct<>-ompi<>-gcc<>-i386 DLD $ octave ... # ############### or directly ignore the pre-compiled version and make your own $ ls -lad DLD # remove the DLD link $ rm DLD $ mkdir DLD # create a new DLD subdir $ cd src $ ls -la Make* # double-check Makefile.inc is the one you want $ rm Makefile.inc # change it if it's not $ ln -s Makefile.inc.OMPI Makefile.inc $ make # rebuild MPITB for your system # ############################### this will generate a new DLD subdir $ cd .. $ octave ... Minimal test: MPI_Init, [a b]=MPI_Initialized, MPI_Finalize, [a b]=MPI_Finalized ... octave:1> MPI_Init # working? ans = 0 # great! octave:2> # ############################### You can discover the versions of your different software components with: ========================================================================= $ octave-config --print VERSION 2.9.12 $ octave --eval ver ---------------------------------------------------------------------- GNU Octave Version 2.9.12 GNU Octave License: GNU General Public License Operating System: Linux 2.6.20-... ---------------------------------------------------------------------- ========================================================================= $ laminfo -version lam full LAM/MPI: 7.1.3 $ lamboot -V LAM 7.1.3/MPI 2 C++/ROMIO - Indiana University Arch: i686-pc-linux-gnu Prefix: <...>/lam-7.1.3 Configured by: <...> Configured on: Tue Jun 26 12:44:08 CEST 2007 Configure host: hostname.host.domain SSI rpi: crtcp lamd sysv tcp usysv ========================================================================= $ ompi_info --version ompi full Open MPI: 1.2.3 Open MPI SVN revision: r15136 Open RTE: 1.2.3 Open RTE SVN revision: r15136 OPAL: 1.2.3 OPAL SVN revision: r15136 $ mpirun -V mpirun (Open MPI) 1.2.3 ========================================================================= You can discover which software your MPITB was precompiled for, using: ========================================================================= $ cd ~/octave/mpitb $ ls -lad DLD* lrwxrwxrwx 1 - - 37 jul 13 20:00 DLD -> DLD-oct2.9.12-ompi1.2.3-gcc4.1.2-i386 drwxr-xr-x 2 - - 16384 jul 9 20:34 DLD-oct2.9.12-lam_7.1.3-gcc4.1.2-i386 drwxrwxr-x 2 - - 16384 jul 9 19:34 DLD-oct2.9.12-ompi1.2.3-gcc4.1.2-i386 $ ls -la DLD/MPI_Init.oct -rwxrwxr-x 1 - - 15420 jul 9 19:26 DLD/MPI_Init.oct $ ldd !$ linux-gate.so.1 => (0x008a9000) ... libmpi.so.0 => /home/.../openmpi-1.2.3/lib/libmpi.so.0 (0x00560000) libopen-rte.so.0 => /home/.../openmpi-1.2.3/lib/libopen-rte.so.0 (0x00ba9000) libopen-pal.so.0 => /home/.../openmpi-1.2.3/lib/libopen-pal.so.0 (0x00156000) ... ========================================================================= Notice the DLD link points to a precompiled DLD*ompi* subdir. It is not empty, MPI_Init.oct is there for instance, and "ldd" shows it depends on an Open-MPI libmpi.so library. You can discover what software your MPITB is prepared to recompile with, using: ========================================================================= $ cd ~/octave/mpitb/src $ ls -la Make* -rwxr-xr-x 1 - - 17808 jul 9 20:51 Makefile lrwxrwxrwx 1 - - 17 jul 10 13:14 Makefile.inc -> Makefile.inc.OMPI -rwxr-xr-x 1 - - 5408 jul 10 10:57 Makefile.inc.LAM -rwxr-xr-x 1 - - 5402 jul 10 10:56 Makefile.inc.OMPI -rwxr-xr-x 1 - - 7294 jul 9 13:02 Makefile.old-gcc-no-mkoctfile $ $ which mpiCC ~/openmpi-1.2.3/bin/mpiCC $ which ompi_info ~/openmpi-1.2.3/bin/ompi_info $ which mpirun ~/openmpi-1.2.3/bin/mpirun $ ldd `which mpirun` linux-gate.so.1 => (0x00ddd000) libopen-rte.so.0 => /home/.../openmpi-1.2.3/lib/libopen-rte.so.0 (0x00d72000) libopen-pal.so.0 => /home/.../openmpi-1.2.3/lib/libopen-pal.so.0 (0x00f60000) ... ========================================================================= Notice the Makefile.inc link points to the OMPI version, and that your environment uses the right mpirun/mpiCC/ompi_info commands, depending on the right OMPI libraries. The "make" procedure assumes your mkoctfile/octave-config and laminfo/ompi_info/mpiCC commands are correctly configured and working, and uses them to deduce the libraries and setup required to recompile MPITB. See Makefile.inc for details on: # # octave-config --print INCLUDEDIR OCTINCLUDEDIR # octave-config --print OCTLIBDIR # # mkoctfile --print INCFLAGS CXXFLAGS CXXPICFLAG XTRA_CXXFLAGS # mkoctfile --print DL_LDFLAGS LFLAGS # mkoctfile --print LIBS FLIBS OCTAVE_LIBS BLAS_LIBS FFTW_LIBS # # laminfo/ompi_info -path incdir -parsable | cut -d: -f3 # laminfo/ompi_info -path libdir -parsable | cut -d: -f3 # # mpiCC --showme:compile # mpiCC --showme:link -------------------------------- - Configuring Octave's startup - -------------------------------- MPITB comes with an .octaverc file linked to startups/startup_MPITB.m. If you run octave from ~/octave/mpitb, it will correctly source the provided startup run-command .octaverc. -------------------------------- $ cd ~/octave/mpitb $ ls -la .octaverc # make sure the link is there $ octave ... Minimal test: MPI_Init, [a b]=MPI_Initialized, MPI_Finalize, [a b]=MPI_Finalized Examples of environment variables to affect MPI behaviour LAM/MPI SSI interface: putenv('LAM_MPI_SSI_rpi', 'lamd'), MPI_Init OpenMPI MCA interface: putenv('OMPI_MCA_btl_base_debug','2'), MPI_Init Help on MPI: help mpi octave:1> -------------------------------- If you run octave from somewhere else, you can simply create a similar link there. For instance, if you run octave from ~/octave, simply do: -------------------------------- $ cd ~/octave $ ln -s mpitb/startups/startup_MPITB.m .octaverc $ ls -la .octaverc # make sure you did it right -------------------------------- If you already had a .octaverc which you must keep, source the provided startup code from yours, like this: -------------------------------- $ cd ~/octave $ ls -la .octaverc # make sure you have one $ ln -s mpitb/startups/startup_MPITB.m . # link startup_MPITB here, same name $ vi .octaverc # add these lines to yours ... % MPI Toolbox startup M-file, if it exists. if exist('startup_MPITB.m','file') startup_MPITB end -------------------------------- Now run octave. If everything is right you will read those help lines about MPI: -------------------------------- $ octave ... Minimal test: MPI_Init, [a b]=MPI_Initialized, MPI_Finalize, [a b]=MPI_Finalized Examples of environment variables to affect MPI behaviour LAM/MPI SSI interface: putenv('LAM_MPI_SSI_rpi', 'lamd'), MPI_Init OpenMPI MCA interface: putenv('OMPI_MCA_btl_base_debug','2'), MPI_Init Help on MPI: help mpi octave:1> -------------------------------- ----------------------------------- - Configuring users' shell script - ----------------------------------- The provided run-command startup_MPITB.m uses an environment variable ($MPITB_HOME) to discover where MPITB has been installed. If it is not defined, it tries the suggested ~/octave/mpitb location, so if you install MPITB there, you will not need any .bashrc customization (apart from the previous LAM/OMPI and Octave setup you might have). Please see startup_MPITB.m (and BASHRC_stub) for details. If you choose a different location, or want to set MPITB_HOME anyways, you can edit your shell-run-command to include the definition. MPITB comes with a BASHRC_stub file with suggestions on how you could setup your .bashrc file, just in case LAM/OMPI and/or Octave were not already included in your PATH and LD_LIBRARY_PATH. tcsh users should remember their syntax is different # export MPITB_HOME=~/octave/mpitb # for bash # setenv MPITB_HOME ~/octave/mpitb # for tcsh --------------------- - About recompiling - --------------------- Notice that several precompiled versions might be included in the compressed tarball, re-made in separate subdirs customarily named DLD-<Octave.version>-<LAM/OMPI.version>-gcc<version>-<arch32/64>. If some precompiled version matches your setup, you might want to relink DLD there and give it a try, saving you from recompiling: -------------------------------- $ cd ~/octave/mpitb $ rm DLD # ignore suggested version $ ln -s DLD-oct2.9.12-lam_7.1.3-gcc4.1.2-i386 DLD # choose preferred one $ ls -lad DLD* # make sure you did it right $ octave # .octaverc will stick to it Minimal test: MPI_Init, [a b]=MPI_Initialized, MPI_Finalize, [a b]=MPI_Finalized Examples of environment variables to affect MPI behaviour LAM/MPI SSI interface: putenv('LAM_MPI_SSI_rpi', 'lamd'), MPI_Init OpenMPI MCA interface: putenv('OMPI_MCA_btl_base_debug','2'), MPI_Init Help on MPI: help mpi octave:1> MPI_Initialized # working? ans = 0 # great! octave:2> -------------------------------- If no precompiled version works in your setup, you must recompile. The Makefiles are in $MPITB_HOME/src. You simply cd there, choose the right Makefile.include (linking it to Makefile.inc) and type "make". It will recompile the .cc files found there to .oct files in $MPITB_HOME/DLD. New include files will be developed to accomodate for other MPI implementations. As of Octave-2.9.12, MPITB has been recompiled against LAM/MPI and Open-MPI without problems using the provided include files: -------------------------------- $ cd $MPITB_HOME/src $ ls -la Make* # basically 2, Makefile[.inc] -rwxr-xr-x 1 <user> <user> 17808 jul 9 20:51 Makefile lrwxrwxrwx 1 <user> <user> 17 jul 10 13:14 Makefile.inc -> Makefile.inc.OMPI -rwxr-xr-x 1 <user> <user> 5408 jul 10 10:57 Makefile.inc.LAM -rwxr-xr-x 1 <user> <user> 5402 jul 10 10:56 Makefile.inc.OMPI -rwxr-xr-x 1 <user> <user> 7294 jul 9 13:02 Makefile.old-gcc-no-mkoctfile $ diff Makefile.inc.LAM Makefile.inc.OMPI # very little difference 127c127 < MPINFO = laminfo --- > MPINFO = ompi_info 131c131 < MPILIBS = -lmpi -llam --- > MPILIBS = -lmpi $ rm Makefile.inc # if you want the other, $ ln -s Makefile.inc.LAM Makefile.inc # simply choose (link) it $ ls -la Make* # double-check you did it right $ make # overwrite or create MPITB DLD # Oops, you are about to overwrite the currently linked MPITB DLD # or create the DLD subdir if it does not exist. You might instead $ ls -lad ../DLD* # double-check what's there right now $ cd .. # go there and $ rm DLD # remove the link (if it's a link)... # right now, "make" would create a new DLD $ mkdir DLD-oct... # ...and create a new subdir $ ln -s DLD-oct... DLD # ...and link it to DLD so that $ cd src # when you type make $ make # the new MPITB goes there -------------------------------- "make" creates the DLD subdir if required. Existence of the DLD subdir is an "order-only" prerequisite (see GNU Make manual, sect.4.3 "Types of prerequisites"), and as such it is added after | in the prerequisite list: -------------------------------- #$(INSTDLDS): $(INSTDIR)/%: % $(INSTDIR) $(INSTDLDS): $(INSTDIR)/%: % | $(INSTDIR) -------------------------------- Some users have remarked that | is a too recent feature of GNU Make and their make fails to build MPITB. The older-style, normal prerequisite version is commented above the newer order-only syntax. Remove the newer line and uncomment the older if your make won't support order-only prerequisites. Thanks to Dennis van Dok for pointing this out. Michael Creel and Thomas Weber also contributed to previous versions of the Makefile. Breanndán Ó Nualláin also helped solving bugs in the sources (see acknowledgement in MPI_Unpack.cc). As told above, the "make" procedure assumes your mkoctfile/octave- config and laminfo/ompi_info/mpiCC commands are correctly configured and working. You might need to edit/fix your .bashrc run-command as suggested in BASHRC_stub and logout/login for changes to take effect, so that all those commands work correctly before invoking make. -------------------------------------- - LAM default hostfile (boot schema) - -------------------------------------- - vs. Open-MPI -H & -hostfile syntax - -------------------------------------- You need a bhost file to boot a LAM (Local Area Multicomputer). Recent LAM versions will pick a ./lam-bhost.def file if it exists in your current directory, instead of just relying on a $LAMBHOST environment variable pointing to the file. This greatly simplifies configuration, since most users will never use $LAMBHOST. Simply create ./lam-bhost.def and add some hostnames on which you have an account. See "man bhost" if you require special syntax for multiprocessors or for different username on those hosts. Then boot the LAM: -------------------------------- $ cd ~/octave/mpitb $ mkdir myproject $ cd myproject $ cp ../startups/startup_MPITB.m .octaverc # copy-edit startup $ vi .octaverc # & customize it if you want # ... # % customize the <myappname>-marked sections # ... # % q = [p '/<myappname>']; # % if ~exist(q, 'dir'), clear p q, error('<my> not found'), end # % addpath(q); $ vi lam-bhost.def # define your LAM # host1 # host2 user=guest # host3 cpu=2 $ lamboot # boot it $ lamnodes # check it works ok $ octave # now from Octave Minimal test: MPI_Init, [a b]=MPI_Initialized, MPI_Finalize, [a b]=MPI_Finalized Examples of environment variables to affect MPI behaviour LAM/MPI SSI interface: putenv('LAM_MPI_SSI_rpi', 'lamd'), MPI_Init OpenMPI MCA interface: putenv('OMPI_MCA_btl_base_debug','2'), MPI_Init Help on MPI: help mpi octave:1> MPI_Init # working? ans = 0 # great! octave:2> -------------------------------- If you don't get error messages, there you go. If you get errors, you must solve them before using MPITB under Octave. LAM must be already booted when you use MPI_Init from inside Octave (ok, system("lamboot") would have worked too, agreed :-) For LAM-related errors, read the LAM FAQ since most probably somebody already had exactly the same problem, and the answer is already there. http://www.lam-mpi.org/ ("Getting Help" & "FAQ" buttons) http://www.lam-mpi.org/using/support/ http://www.lam-mpi.org/faq/ Open-MPI is even easier. Just MPI_Init from octave. It should work. If you get messages about LAM, make sure your DLD subdir corresponds to an OMPI-recompiled toolbox, enter octave and MPI_Init again. For OMPI-related errors, same instructions as above. http://www.open-mpi.org/ http://www.open-mpi.org/community/help/ http://www.open-mpi.org/faq/ Open-MPI lets users to specify hosts to an mpirun command with the -H (-host) and -hostfile switches, this way: -------------------------------- $ cd $MPITB_HOME/Hello $ ls Hello.m lam-bhost.def README startup_MPITB.m $ mpirun -H h1,h2,h3 octave -q --eval Hello Hello, MPI_COMM_world! I'm rank 0/3 (h1) Hello, MPI_COMM_world! I'm rank 2/3 (h3) Hello, MPI_COMM_world! I'm rank 1/3 (h2) $ cat lam-bhost.def h1 h2 h3 $ mpirun -c 3 -hostfile lam-bhost.def octave -q --eval Hello Hello, MPI_COMM_world! I'm rank 0/3 (h1) Hello, MPI_COMM_world! I'm rank 1/3 (h2) Hello, MPI_COMM_world! I'm rank 2/3 (h3) -------------------------------- When using LAM/MPI, the lam daemon must be started on each computer to be considered part of the "multicomputer". You can use the -c <#copies> syntax to LAM's mpirun to launch a parallel SPMD application. In old MPITB versions when we recommended using MPI_Comm_spawn from an interactive Octave session to launch parallel applications, care should be taken to not oversubscribe the master node (on which the interactive Octave was already running) with an additional spawned octave process. LAM requires the master node to be part of the multicomputer, and the daemon must be already launched when you MPI_Init, so the order of nodes in the round-robin host-list is already defined by when you MPI_Comm_spawn. Thus we recommended placing the master node last in the bhost file. That way, spawning an increasing number of octave processes would place them on other cluster nodes before starting to oversubscribe the master node where you were already running an octave process. Open-MPI's MPI_Comm_spawn uses the MPI_Info hint to MPI_Comm_spawn to indicate the "slave" nodes, but if you try to MPI_Comm_spawn from a normal octave interactive session, you'll get the error -------------------------------------------------------------------------- Some of the requested hosts are not included in the current allocation for the application: xterm The requested hosts were: h8,h5,h4,h2 Verify that you have mapped the allocated resources properly using the --host specification. -------------------------------------------------------------------------- so for OMPI we recommend either not using MPI_Comm_spawn or starting the first (master) interactive octave session with -------------------------------------------------------------------------- $ mpirun -c 1 -H h1,h8,h5,h4,h2 octave --interactive -------------------------------------------------------------------------- that is, including all slave nodes after the master node (where the interactive octave is to be run) so all slave nodes are in master_node_list by when you use MPI_Comm_spawn from the interactive octave session. Launching the parallel application using mpirun avoids these considerations. That's the procedure we are recommending now. Having a bhost file is required for LAM operation (lamboot), and convenient for OMPI operation using the -c switch "mpirun -c 3 -hostfile bhost" instead of repeatedly writing the hostnames as in "mpirun -H h1,h2..." --------------------------------------- - Utility scripts ($MPITB_HOME/utils) - --------------------------------------- MPITB_Clean can be used after MPI_Finalize to allow a new MPI epoch. In normal use (as per standard), after MPI_Finalize you cannot use any other MPI routine again. You can however clean all MPI DLDs from Octave memory, and start over from the beginning. That's the purpose of MPITB_Clean. It works for both OMPI and LAM. Most of the remaining utilities were developed for LAM/MPI, and are still being adapted to OMPI. Expect rough spots if you try MPI_Comm_spawn with OMPI. # ------------------------------------------------------------------ Note to selves: only MPITB_Detect_mpi.m and MPITB_Init/OMPI_Init for now. # ------------------------------------------------------------------ Although the user could conceivably have a different bhost file in each project subdir, in normal use he will be always using the same multicomputer or cluster nodes. Conceivably, an utility script could be customized to boot a new multicomputer on those always-the-same computers. Or take advantage of an already booted LAM, if it meets the required number of hosts. That is the role of $MPITB_HOME/utils/LAM_Init.m. If you are planning to use the utilities provided in the $MPITB_HOME/utils subdir, you might want to edit the LAM_Init.m file so that the builtin HOSTS variable describes your cluster. Or perhaps get used to define a global HOSTS in your code before invoking LAM_Init. Octave_Spawn is another interesting utility to simplify the tasks involved in using MPI_Comm_spawn() to start slave Octave instances from a master Octave session. It assumes that slave Octaves will source the provided startup_mergeParent.m (or they will merge with the parent communicator somehow else). Another startup_bcastSlv.m is provided for applications in which the command to execute will be decided at runtime (so you cannot use mpirun), but all slaves will run the same command. A much more elaborated protocol is available with the startup_ncmdsSlv.m, together with the NumCmds_Init and NumCmds_Send utilities. According to this "protocol", the master instance sends an initial message indicating the number of commands that will be sent to slave instances. The slaves then enter a loop in which they repeatedly wait for a new command and evaluate it. If the number of commands is "0", slaves repeat the loop forever... until a "break", "quit" or "exit" command is sent, of course. -------------------------------- - What now? TUTORIAL or demos? - -------------------------------- Do skip this indented section ||| To start getting acquainted to MPI, one should open an xterm window, ||| "lamboot" a multicomputer and then start "octave". Then, one might want ||| to open another xterm/editor from where to copy/paste lines from the ||| tutorial file to the octave window. In the past we have expressed our ||| opinion that this is the fastest way to learn MPI, and that you won't ||| regret reproducing the whole tutorial, or at least the sections dealing ||| on MPI routines you are interested in. ||| ||| But the fact is most users will benefit from the basic six, i.e., they ||| will just use MPI_Init/_Finalize, MPI_Send/_Recv, MPI_Comm_rank/_size. ||| No tutorial is required to learn those. In fact most users will have ||| already figured out the utility and use of the basic six just after ||| reading the names. ||| ||| System administration is steadily becoming more and more complex, ||| more and more academically disregarded, and exceedingly difficult ||| for novice users to simply understand howto spawn a simple xterm ||| on a slave (X-client) computer and send the window back to the master ||| (X-server) computer. ssh tunneling only obscures the issue yet more, ||| since the encripted tunnel is lost for lamd after successful lamboot, ||| and spawned processes inherit precisely _that_ lamd environment. ||| iptables standard paranoid settings add fuel to the bonfire. And ||| recently we have seen clusters with xterm setuid root again and ||| puzzling miss of environment variables on xterms. Must give up. ||| Temptative MPITB users don't deserve this nightmare. Learning to ||| interactively use and debug MPI was so cool, but it's a lost battle ||| when there are no helpful sysadmins around. ||| ||| So now... we are recommending the use of demos to start learning MPITB. In increasing difficulty order, they are (for now): - Hello - Pi - PingPong - Mandelbrot - NPB/EP - Spawn -> These demos are not mpirun-nable - Wavelets -> they still use the utils subdir Simply change into the respective subdirs, read the README files, the code (specially the help sections) and try to run them. Most of the demos have been re-designed to be mpirun'nable, instead of using MPI_Comm_spawn from a master Octave to start slave Octaves. A couple of older demos may still hang around for those still interested in the utilities subdir. If you suceed with the demos, you can safely ignore the TUTORIAL file and the rest of this file. With respect to this file, it finishes with some explanations about the tutorial and the latter old-style demos, Spawn and Wavelets. ---------------------- - The tutorial story - ---------------------- If in a future parallel application of yours you have doubts about howto use a particular MPITB command, search in the TUTORIAL for that command name to see its typical usage, even if you do not plan to try it out interactively. That is the only reason that the TUTORIAL file has not been removed from MPITB yet. Due to its _tutorial_, interactive nature, the tutorial uses MPI_Comm_spawn to start and finish new slaves as required for each exercise. Not only Octave slaves are dynamically spawned, but they are run on top of an xterm window, so the user can see the xterms pop-up, and write MPITB commands on them... and see how MPI_Recv gets hung until a matching MPI_Send is issued, and cause a fatal error and see the slave xterm die and the master Octave hung on an awaiting recv... and by the way, learn how to debug a parallel Octave application. Experience has shown us that this approach is an error, since the interest of most users on MPITB dies when they learn they cannot get an xterm back in their GUI. Reasons for that are usually: - std firewall rules on personal PC/ws (blocking all X traffic back) learning iptables just to be able to use MPITB is an overkill - standard SSH tunneling ability which obscures the problem. "xterm problem? Which problem? I can launch all xterms I want, it's just MPITB who won't start them" - standard SSH configuration on headed nodes (using localhost:10) the skills to convince sysadmins are yet harder than iptables - no direct connectivity from headless nodes (usually in 192.168.x.y) no way back to user's PC, since nobody really sits at the headed node. Efforts to include explanations and workarounds for these problems has proven to be not only a lost battle, but worse a tactical error, since users will forever remember that they had problems _with_MPITB_, not with X11, sshd, firewalls, 192.168.x.y routing or lack of helpful local sysadmin advice. The "lamprep" and "proxy" shell-scripts are kept in the "$MPITB_HOME/stdlones" subdir as romantic, silent milestones (statues to the unknown soldier if we continue the analogy), to never forget this was a lost battle. -------------------------------- - The Spawn and Wavelets demos - -------------------------------- The Spawn demos make sense with DEBUG set to true, in order to see the xterms running Octave on each slave computer. They have evolved from a simple exercise on the 'lam_spawn_sched_round_robin' LAM/MPI_Info key (now stored in Spawn_slv_bcast/Spawn_staggrd.m) to a full set of introductory demos for the utilities subdir. # ------------------------------------------------------------------ Note to selves: Remove this remark when they also work with OMPI There is a working Spawn_slv_nostartup/Spawn_ompi.m demo for now # ------------------------------------------------------------------ In increasing complexity order, the Spawn demos are: - Spawn_slv_nostartup: includes Spawn_xterms Spawn_octaves - Spawn_slv_merge: incl. Spawn_oct_man Spawn_oct_auto Spawn_oct_bcast - Spawn_slv_bcast: incl. Spawn_bcast Spawn_staggrd - Spawn_slv_ncmds: incl. Spawn_inter Spawn_auto - Spawn_slv_eval: incl. the second demo of all the previous subdirs They are grouped depending on the slave startup protocol used. The final Spawn demo illustrates howto the --eval switch can be used to avoid a fixed slave startup in the startup_MPITB.m file, and thus allow for different applications with different startup protocols in the same subdir. The Wavelets demo is really complex. Read the Contents.m file to get a "roadmap" of the different files included. It's a step-by- step whole example of how would one go through parallelizing a sequential octave application. It could probably be still improved to get better speedups, but we wanted to keep it didactical, so much more time has been spent in documenting the initial, premeditate wrong design decisions than in obtaining good speedups. The data subdir stores the instrumentation data collected using the annotation/timestamp utilities init_stamp, time_stamp, last_stamp, clas_instr, save_instr, load_instr, look_instr, lkup_label You might find them useful to timestamp your own parallel applications and easily discover where the time is mostly spent. -------------- - What then? - -------------- If you want us to add some MPITB application of yours to the MPITB web page, or mention a paper or publication in a congress which uses MPITB, or link to some document you prepared for your research fellows/students dealing on MPITB, drop us an e-mail. We'd certainly like to see the "MPITB papers" list growing :-) ************************************* * Thanks for your interest in MPITB * ************************************* Granada, SPAIN, 17/Jul/2007