This repository provides Spack configuration files, example job scripts, and notes about building and running Mochi-based codes on various platforms. Please refer to the subdirectory for your platform of interest for more information.
Each platform subdirectory in this repository provides a spack.yaml file.
A spack.yaml file fully describes a Spack environment, including
system-provided packages and compilers. It does so independently of any
compilers.yaml or packages.yaml files installed in ~/.spack, thereby
preventing interference with user-specific spack configurations as much as
possible.
You may use spack.yaml files to create a
Spack environment
in which Mochi packages will be installed.
If you don't have Spack installed on your platform, clone it and set it up as follows.
$ git clone https://github.com/spack/spack.git
$ . spack/share/spack/setup-env.sh
Remember that the second line needs to be executed every time you open a new terminal; it may be helpful to create an alias in your bashrc file as a shortcut.
You will then need to clone mochi-spack-packages, which contains the Mochi packages.
$ git clone https://github.com/mochi-hpc/mochi-spack-packages.git
$ spack repo add mochi-spack-packages
Now clone the present repository and cd into the subdirectory relevant
to your platform. For example:
$ git clone https://github.com/mochi-hpc-experiments/platform-configurations.git
$ cd platform-configurations/ANL/Bebop
Edit the path to mochi-spack-packages in the repos field of the spack.yaml file to
match your installation.
Then, execute the following command (changing myenv into an appropriate name for your environment).
$ spack env create myenv spack.yaml
Change to a directory outside of the platform-configurations folders
and activate the environment as follows.
$ spack env activate myenv
You may now add specs to your environment. For instance if you want to install Margo, execute the following command.
$ spack add mochi-margo
If the spack.yaml file provides multiple compilers and you want
to use another than the default one, specify the compiler explicitely,
for example:
$ spack add mochi-margo %gcc@8.2.0
Note that the spack.yaml file you used may already have a spec
added as an example (usually mochi-margo). You can remove it as
follows.
$ spack rm mochi-margo
Once you have added the specs you need in your environment, install everything by executing the following command.
$ spack install
You may add more specs later on. For more information on how to manage Spack environments, please refer to the Spack documentation.
Should you want to contribute a spack.yaml for a particular machine,
please submit a merge request with it, and ensure the following.
- The
spack.yamlfile should contain the compiler(s) that have been tested and confirmed to work with Mochi packages. - The
spack.yamlfile should try to list system-provided packages, in particular packages used for building (cmake,autoconf, etc.), and relevant system-provided MPI implementations.- Note that this must be done manually. Spack provides a
spack external findcommand that can be used to locate a subset of system packages, but it does not populate thespack.yamlfile.
- Note that this must be done manually. Spack provides a
- The
spack.yamlfile should contain the relevant variants for packages, in particular the transport methods to use withlibfabric. - The path to the
spack.yamlfile should be of the form<institution>/<platform>/spack.yaml. - Please make sure that your
spack.yamlis a reliable way to work with Mochi on the target platform, other people will rely on it!
You can also contribute changes to existing spack.yaml files, in particular
to add working compilers, system packages, etc. As always, please test that
new setups work before creating a merge request.