Find static MKL LAPACK and ScaLAPACK libraries on Linux

[adam-sim-dev]

Dear Michael,

I am wondering how can I find and link static MKL LAPACK and ScaLAPACK libraries on Linux. These two modules actually find and link static lib on Windows.

I am now testing on Ubuntu 20.04 with oneAPI 2022.

Best,
Adam

[scivision]

This is initially implemented in f0c32f4.
This feature for Intel MKL requires CMake ≥ 3.24 for LINK_GROUP as per Intel link line advisor

cmake -Bbuild -Dfind_static=on

[scivision]

The default for MKL finding on Windows is by default static, but on MacOS and Linux the default is shared as you observed.

[adam-sim-dev]

Thanks, Michael, I just tested your latest commit and it works as expected. Actually, I also want to use the FindLAPACK.cmake and FindSCALAPACK.cmake modules to find lapack and scalapack, without the lapack.cmake and scalapack.cmake option files. In this case, the -Dfind_static=on option does not work. Is it much better to put this option in the corresponding FindXXX.cmake files?

[adam-sim-dev]

There is a BLA_STATIC option in the FindLAPACK.cmake module file distributed by default with CMake (3.24.0 I am using). There is no FindScaLAPACK.cmake with CMake. That is why I want to use your CMake modules.

[adam-sim-dev]

I workaround the problem by setting as follows in my own CMakeLists.txt: (taken from your latest commit, thanks)

if(find_static)
  set(_orig_suff ${CMAKE_FIND_LIBRARY_SUFFIXES})
  set(CMAKE_FIND_LIBRARY_SUFFIXES ${CMAKE_STATIC_LIBRARY_SUFFIX})
endif()
find_package(SCALAPACK REQUIRED)
if(find_static)
  set(CMAKE_FIND_LIBRARY_SUFFIXES ${_orig_suff})
endif()

[adam-sim-dev]

Dear Michael,

I still have difficulty linking the SCALAPACK_LIBRARIES found by FindSCALAPACK.cmake. I do not know what exactly the SCALAPACK_LIBRARIES are. Does SCALAPACK_LIBRARIES only include mkl_scalapack_lp64, and mkl_blacs_xxx?

Intel oneAPI Math Kernel Library Link Line Advisor shows one should use the following link line:

${MKLROOT}/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group ${MKLROOT}/lib/intel64/libmkl_intel_lp64.a ${MKLROOT}/lib/intel64/libmkl_sequential.a ${MKLROOT}/lib/intel64/libmkl_core.a ${MKLROOT}/lib/intel64/libmkl_blacs_intelmpi_lp64.a -Wl,--end-group -lpthread -lm -ldl

I mean, are mkl_core and mkl_intel_lp64 missing from SCALAPACK_LIBRARIES? I am not sure.

I suggest that could you please add the Result Variables you declared in FindLAPACK.cmake, FindSCALAPACK.cmake, and maybe other CMake modules to the CMake cache. CMake beginners like me can see the results using CMake-GUI or search the CMakeCache file.

Thanks,
Adam

[scivision]

Yes I chose to omit the mkl_core and mkl_intel_lp64 from Scalapack_libraries to avoid link order issues. For the larger projects where I use Scalapack with lengthy link commands, the transitive resolver in CMake would put the libraries out of order and fail to link.
This is because I had projects where Scalapack was optional. If I was more careful in those other projects to link only either Lapack or Scalapack it wouldn't be an issue.

[scivision]

I think the practice is to not put *_LIBRARIES variables in the cache to avoid confusing behavior on project reconfigure. The practice is generally to only cache variables that are "expensive" to obtain like any of the find_*() commands. The derived result variables like *_LIBRARIES are almost zero cost.

[scivision]

Yes what I would do for the Find*.cmake is to make the "static" option a COMPONENT to avoid arbitrary variable setting. I think that's a more robust way to do it .

[adam-sim-dev]

I think the practice is to not put *_LIBRARIES variables in the cache to avoid confusing behavior on project reconfigure. The practice is generally to only cache variables that are "expensive" to obtain like any of the find_*() commands. The derived result variables like *_LIBRARIES are almost zero cost.

I am not so familiar about cmake. Just want to see the results more clearly. How about show them with message?

[scivision]

I can make them print as message(VERBOSE "...") which is optionally revealed (to avoid excess printing by default in large projects). The message would be revealed by:

cmake -B build --log-level=VERBOSE

[scivision]

this VERBOSE log is now in d5b621f

[scivision]

there is now a STATIC component in FindLapack.cmake and FindScalapack.cmake as used in cmake/lapack.cmake and cmake/scalapack.cmake

[adam-sim-dev]

there is now a STATIC component in FindLapack.cmake and FindScalapack.cmake as used in cmake/lapack.cmake and cmake/scalapack.cmake

Thanks. I will test them soon.

[adam-sim-dev]

Hi Michael,

When I link to LAPACK_LIBRARIES found by

find_package(LAPACK COMPONENTS STATIC OpenMP)

the following error occurs:

ld: /opt/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_core.a(xzgeqrf.o): in function mkl_lapack_xzgeqrf': xzgeqrf_gen.f:(.text+0x240): undefined reference to mkl_lapack_zgeqrf_pfnr'
ld: /opt/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_core.a(xzgbtrs.o): in function mkl_lapack_xzgbtrs': xzgbtrs_gen.f:(.text+0xc45): undefined reference to mkl_blas_ztrmm'
ld: xzgbtrs_gen.f:(.text+0x1532): undefined reference to mkl_blas_ztrmm' ld: xzgbtrs_gen.f:(.text+0x1ff6): undefined reference to mkl_blas_ztrmm'
ld: /opt/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_core.a(xzgbtrf.o): in function mkl_lapack_xzgbtrf': xzgbtrf_gen.f:(.text+0x994): undefined reference to mkl_blas_izamax'
ld: /opt/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_core.a(xdgeqrf.o): in function mkl_lapack_xdgeqrf': xdgeqrf_gen.f:(.text+0x23c): undefined reference to mkl_lapack_dgeqrf_pfnr'
ld: /opt/intel/oneapi/mkl/2022.1.0/lib/intel64/libmkl_core.a(zlarft.o): in function mkl_lapack_zlarft': zlarft_gen.f:(.text+0xb71): undefined reference to mkl_blas_ztrmm'

Does it work on your side?