===================
clMAGMA README FILE
===================
* Further documentation is provided in docs/html/index.html.
* To INSTALL clMAGMA, modify the make.inc file to indicate your C/C++ compiler,
Fortran compiler, and where AMD's clBLAS, CPU BLAS, and LAPACK are installed on your
system. Examples are given in make.inc.acml, make.inc.atlas, make.inc.macos,
make.inc.mkl-*, make.inc.openblas showing how to link to ACML, ATLAS, MacOS
veclib, MKL, and OpenBLAS, respectively.
All the make.inc files assume $CUDADIR is set in your environment.
For bash (sh), put in ~/.bashrc (with your system's path):
export CUDADIR=/usr/loca/cuda
For csh/tcsh, put in ~/.cshrc:
setenv CUDADIR /usr/local/cuda
MAGMA is tested with CUDA >= 5.5. Some functionality requires a newer version.
The MKL make.inc files assume $MKLROOT is set in your environment.
For bash (sh), put in ~/.bashrc (with your system's path):
source /opt/intel/composerxe/mkl/bin/mklvars.sh intel64
For csh/tcsh, put in ~/.cshrc:
source /opt/intel/composerxe/mkl/bin/mklvars.csh intel64
clMAGMA is tested with MKL 11.2.3 (2015), both LP64 and ILP64;
other versions may work.
The ACML make.inc file assumes $ACMLDIR is set in your environment.
For bash (sh), put in ~/.bashrc (with your system's path):
export ACMLDIR=/opt/acml-5.3.1
For csh/tcsh, put in ~/.cshrc:
setenv ACMLDIR /opt/acml-5.3.1
clMAGMA is tested with ACML 5.3.1; other versions may work.
See comments in make.inc.acml regarding ACML 4;
a couple testers fail to compile with ACML 4.
The ATLAS make.inc file assumes $ATLASDIR and $LAPACKDIR are set in your environment.
If not installed, install LAPACK from http://www.netlib.org/lapack/
For bash (sh), put in ~/.bashrc (with your system's path):
export ATLASDIR=/opt/atlas
export LAPACKDIR=/opt/LAPACK
For csh/tcsh, put in ~/.cshrc:
setenv ATLASDIR /opt/atlas
setenv LAPACKDIR /opt/LAPACK
The OpenBLAS make.inc file assumes $OPENBLASDIR is set in your environment.
For bash (sh), put in ~/.bashrc (with your system's path):
export OPENBLASDIR=/opt/openblas
For csh/tcsh, put in ~/.cshrc:
setenv OPENBLASDIR /opt/openblas
* Compiling OpenCL kernels at compile time or at run time
By default, clMAGMA compiles its kernels at compile time and saves them in
lib/clmagma_kernels.co. At runtime, it searchs CLMAGMA_PATH or LD_LIBRARY_PATH
for this file.
Alternatively, if you do not wish to compile kernels at compile time -- for
instance, if you don't know what GPU will be used at runtime -- install a copy
of the clmagmablas directory and include this directory in CLMAGMA_PATH or
LD_LIBRARY_PATH.
TODO add make targets to do each of these.
* Building without Fortran
clMAGMA can be built without Fortran by commenting out FORT in the make.inc file.
However, some testers will not be able to check their results.
* Building a Shared Library
By default now, all make.inc files add the -fPIC option to CFLAGS, FFLAGS,
F90FLAGS, and NVCCFLAGS, required for building a shared library. Note in
NVCCFLAGS that -fPIC is passed via the -Xcompiler option. Running:
make
or
make lib
make test
will create a shared library lib/libmagma.so, static library lib/libmagma.a,
and testing drivers in 'testing'.
(The current exception is for ATLAS, in make.inc.atlas, which in our
install is a static library, thus requiring clMAGMA to be a static library.)
* Building a Static Library
Alternatively, comment out FPIC in the make.inc file to compile only a static
library. Running:
make
or
make lib
make test
will create a static library lib/libmagma.a, and testing drivers in 'testing'.
* Install
To install libraries and include files in a given prefix, run:
make install prefix=/usr/local/magma
The default prefix is /usr/local/clmagma. You can also set prefix in make.inc.
* Environment variables
For multi-GPU functions, set $MAGMA_NUM_GPUS to set the number of GPUs to use.
For multi-core BLAS libraries, set $OMP_NUM_THREADS or $MKL_NUM_THREADS or
$VECLIB_MAXIMUM_THREADS to set the number of CPU threads, depending on your
BLAS library.
* A short standalone EXAMPLE is provided in directory 'example'. This is
intended to show the minimum needed to start using clMAGMA, without all the
extra Makefiles, headers, and libraries used in testing. You must edit
example/Makefile to reflect your make.inc, or use pkg-config, as described in
example/README.txt.
* To TEST clMAGMA, go to directory 'testing'. Provided are a number of
drivers testing different routines. These drivers are also useful
as examples on how to use clMAGMA, as well as to benchmark the performance.
The testers print "ok" or "failed" for whether the error passes the tolerance.
In some cases, the tolerance may be too strict, so a test may "fail" even
though it is only slightly above the tolerance. Error values around 1e-15 for
double and double-complex, and 1e-7 for single and single-complex, are
generally acceptable. Values larger than 1e-4 are very suspicious.
* To TUNE clMAGMA, you can modify the blocking factors for the algorithms of
interest in file 'control/get_nb_tahiti.cpp'. The default values are tuned for
AMD Radeon 7970 (Tahiti) GPUs. You can also compare your performance to
what we get, given in file
'testing/results_clmagma.txt', as an easy check for your installation.
* To autotune clBLAS, set the CLBLAS_STORAGE_PATH environment variable
to a working directory and run clBLAS tune. Subsequent clMAGMA runs will
use the optimized routines found in CLBLAS_STORAGE_PATH.
* To use a GPU on a server, disable X forwarding when you ssh to the server,
using 'ssh -x hostname'. To see whether OpenCL finds your GPU, use 'clinfo'.
For more INFORMATION, please refer to the MAGMA homepage and user forum:
http://icl.cs.utk.edu/magma/
http://icl.cs.utk.edu/magma/forum/
The MAGMA project supports the package in the sense that reports of
errors or poor performance will gain immediate attention from the
developers. Such reports, descriptions of interesting applications,
and other comments should be posted on the MAGMA user forum.