Intel MKL FATAL ERROR: Cannot load libmkl_avx512.so or libmkl_def.so
Closed this issue · 4 comments
I follow your advice and create virtual envirment by conda create -n trefide python=3.6
and I have installed with pip successfully.
However when I ran the code in demo file Demo PMD Compression & Denoising.ipynb
in the line
spatial_thresh, temporal_thresh = determine_thresholds((fov_height, fov_width, num_frames),
(block_height, block_width),
consec_failures, max_iters_main,
max_iters_init, tol,
d_sub, t_sub, 5, True)
the iPython crashed down and restarted. The error in console is
Intel MKL FATAL ERROR: Cannot load libmkl_avx512.so or libmkl_def.so
I follow the advice in https://stackoverflow.com/questions/36659453/intel-mkl-fatal-error-cannot-load-libmkl-avx2-so-or-libmkl-def-so by entering the commond conda install nomkl numpy scipy scikit-learn numexpr conda remove mkl mkl-service, but it didn't work.
As I have install the MTK, I tried to locate the libmkl_avx512.so, which is under the path /opt/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64_lin
So, I modified the makefile as
CXX = icpc
CXXFLAGS = -mkl=sequential -qopenmp -O3 -fPIC -I$(shell pwd)/proxtv -I$(shell pwd)/glmgen/include/ -D NOMATLAB=1 #-mkl=parallel
LDFLAGS = -shared
PROXTV = proxtv
LIBPROXTV = $(PROXTV)/libproxtv.so
GLMGEN = glmgen
MKLLIB = /opt/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64
LIBGLMGEN = $(GLMGEN)/lib/libglmgen.so
LIBTREFIDE = libtrefide.so
LIBS = -lmkl_intel_lp64 -lmkl_core -lm -lmkl_intel_thread -liomp5
#LIBS = /opt/intel/compilers_and_libraries_2019.1.144/linux/mkl/lib/intel64
SRCS = utils/welch.cpp proxtf/wpdas.cpp proxtf/line_search.cpp proxtf/utils.cpp proxtf/l1tf/ipm.cpp proxtf/admm.cpp pmd/pmd.cpp pmd/decimation.cpp
OBJS =$(patsubst %.cpp,%.o,$ (SRCS))
.PHONY: all
all:$(LIBPROXTV) $ (LIBGLMGEN)$(LIBTREFIDE) $ (MKLLIB)
$(LIBPROXTV):
cd $(PROXTV); make;
$(LIBGLMGEN):
cd $(GLMGEN); make;
$(LIBTREFIDE): $ (OBJS)
$(CXX) $ (CXXFLAGS)$(LDFLAGS) -o $ @$^ $ (LIBS)
$(SRCS:.cpp=.d) : %.d : %.cpp
$(CXX) $ (CXXFLAGS) -o$@ $ ^ $(MKLLIB)
.PHONY : clean
clean:
rm -f$(LIBTREFIDE) $ (OBJS) $(SRCS:.cpp=.d)
cd $(PROXTV); make clean;
cd $(GLMGEN); make clean;
It can be installed without error, but still crashed when run the demo.
I have tried everything, could you please help me out?
I'm running into this error as well. I followed all of the instructions in the Math Kernel Library Install and subsequent trefide build and install, but this error seems to persist.
Based on my copy-paste error googling, it seems as if this is a problem with other programs as well that use MKL. I tried most of the solutions people have proposed, but still get library linking errors. @RenieWell perhaps you can try explicitly loading the libraries like below. It seemed to work for some people.
$ LD_PRELOAD="/opt/intel/mkl/lib/intel64/libmkl_def.so:/opt/intel/mkl/lib/intel64/libmkl_avx.so" jupyter-lab --port 4242
symbol lookup error: /opt/intel/mkl/lib/intel64/libmkl_avx.so: undefined symbol: mkl_sparse_optimize_bsr_trsm_i8
I had the same issue. With preloading of libmkl_core.so and libmkl_sequential.so it worked and I can run the demo notebooks. I found this solution here.
LD_PRELOAD=$CONDA_PREFIX/libmkl_core.so:$CONDA_PREFIX/libmkl_sequential.so
I had the same issue. With preloading of libmkl_core.so and libmkl_sequential.so it worked and I can run the demo notebooks. I found this solution here.
LD_PRELOAD=$CONDA_PREFIX/libmkl_core.so:$CONDA_PREFIX/libmkl_sequential.so
Thanks jtoloe!
I solved it by using the new updated conda solution.
the following combination of packages (got from $ conda list) worked for me:
# Name # Version # Build
blas 1.0 mkl
mkl 2021.3.0 h06a4308_520
mkl-service 2.4.0 py36h7f8727e_0
mkl_fft 1.3.0 py36h42c9631_2
mkl_random 1.2.2 py36h51133e4_0
numpy 1.19.5 pypi_0
following installs everything required:
conda install mkl_fft -y☕ ✌️