/shuffile

Move (shuffle) files on local-storage associated with MPI ranks when re-running MPI jobs that cause different mapping of ranks to nodes

Primary LanguageCMIT LicenseMIT

Shuffile

build-only

Overview

This module lets one associate a set of files with a process name. Currently, the name is implied to be the rank within MPI_COMM_WORLD. In the event that a set of distributed processes are moved, for example when restarting an MPI job, functions will migrate files from their original locations to the new locations where the processes are running.

Usage is documented in src/shuffile.h and the User API docs.

Building

To build shuffile, first build the KVTree dependency:

git clone git@github.com:LLNL/KVTree.git KVTree.git

mkdir build
mkdir install

cd build
cmake -DCMAKE_INSTALL_PREFIX=../install -DMPI=ON ../KVTree.git
make clean
make
make install

To build shuffile:

cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_KVTREE_PREFIX=`pwd`/install .
make

Testing

Some simple test programs exist in the test directory that are run by the test target.

make test

Release

Copyright (c) 2018, Lawrence Livermore National Security, LLC. Produced at the Lawrence Livermore National Laboratory.
Copyright (c) 2018, UChicago Argonne LLC, operator of Argonne National Laboratory.

For release details and restrictions, please read the LICENSE and NOTICE files.

LLNL-CODE-751725 OCEC-18-060