ImportError of libmkl_intel_lp64.so.2 during compiling
Zhaoxian-Wu opened this issue · 9 comments
Zhaoxian-Wu commented
Description
Recently, I followed the instructions proposed by @jubueche in issue #545. Even though it worked before, I found I failed toda because of an import error of libmkl_intel_lp64.so.2.
How to reproduce
I exactly followed the following process
conda create -n aihwkit-cuda-dev python=3.10 -y
conda activate aihwkit-cuda-dev
git clone https://github.com/IBM/aihwkit.git ; cd aihwkit
pip install -r requirements.txt
conda install mkl mkl-include -y
export CXX=/usr/bin/g++
export CC=/usr/bin/gcc
export MKLROOT=$CONDA_PREFIX
export CMAKE_PREFIX_PATH=$CONDA_PREFIX/lib
export CUDA_VERSION=11.3
export CUDA_HOME=/usr/local/cuda-${CUDA_VERSION}
export CUDA_TOOLKIT_ROOT_DIR=${CUDA_HOME}
export CUDA_LIB_PATH=${CUDA_HOME}/lib64
export CUDA_INCLUDE_DIRS=${CUDA_HOME}/include
export PATH=${CUDA_HOME}/bin:${PATH}
make build_inplace_cudaThe compiling succeeded, but I got the error on libmkl_intel_lp64.so.2 when I tried to run:
(aihwkit-cuda-dev) zhaoxian@server:~/Desktop/aihwkit$ PYTHONPATH=src/ python examples/01_simple_layer.py
Traceback (most recent call last):
File "/home/zhaoxian/Desktop/aihwkit/examples/01_simple_layer.py", line 25, in <module>
from aihwkit.nn import AnalogLinear
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/nn/__init__.py", line 17, in <module>
from aihwkit.nn.modules.container import AnalogSequential, AnalogWrapper
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/nn/modules/container.py", line 23, in <module>
from aihwkit.nn.modules.base import AnalogLayerBase
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/nn/modules/base.py", line 22, in <module>
from aihwkit.simulator.tiles.module import TileModule
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/simulator/tiles/__init__.py", line 17, in <module>
from aihwkit.simulator.tiles.analog import AnalogTile
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/simulator/tiles/analog.py", line 21, in <module>
from aihwkit.simulator.tiles.rpucuda import RPUCudaSimulatorTileWrapper
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/simulator/tiles/rpucuda.py", line 24, in <module>
from aihwkit.simulator.parameters.base import RPUConfigGeneric
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/simulator/parameters/__init__.py", line 31, in <module>
from .training import UpdateParameters
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/simulator/parameters/training.py", line 20, in <module>
from .helpers import _PrintableMixin
File "/home/zhaoxian/Desktop/aihwkit/src/aihwkit/simulator/parameters/helpers.py", line 20, in <module>
from aihwkit.simulator import rpu_base
ImportError: libmkl_intel_lp64.so.2: cannot open shared object file: No such file or directoryOther information
I ensure the MKL is correctly installed, and the lib is correctly found during compiling.
(aihwkit-cuda-dev) zhaoxian@server:~/Desktop/aihwkit$ ls $MKLROOT/lib
engines-3 libgcc_s.so libmkl_avx2.so.2 libmkl_pgi_thread.so libomp-fallback-cstring.o libsqlite3.so.0 libubsan.so.1
itcl4.2.2 libgcc_s.so.1 libmkl_avx512.so.2 libmkl_pgi_thread.so.2 libomp-fallback-cstring.spv libsqlite3.so.0.8.6 libubsan.so.1.0.0
libasan.so libgomp.so libmkl_avx.so.2 libmkl_rt.so libomp-spirvdevicertl-optional.o libssl.a libuuid.a
libasan.so.6 libgomp.so.1 libmkl_blacs_intelmpi_ilp64.so libmkl_rt.so.2 libomp-spirvdevicertl-required.o libssl.so libuuid.so
libasan.so.6.0.0 libgomp.so.1.0.0 libmkl_blacs_intelmpi_ilp64.so.2 libmkl_scalapack_ilp64.so libomptarget-opencl-optional.bc libssl.so.3 libuuid.so.1
libatomic.so libhistory.a libmkl_blacs_intelmpi_lp64.so libmkl_scalapack_ilp64.so.2 libomptarget-opencl-required.bc libstdc++.so libuuid.so.1.3.0
libatomic.so.1 libhistory.so libmkl_blacs_intelmpi_lp64.so.2 libmkl_scalapack_lp64.so libomptarget.rtl.level0.so libstdc++.so.6 libz.a
libatomic.so.1.2.0 libhistory.so.8 libmkl_blacs_openmpi_ilp64.so libmkl_scalapack_lp64.so.2 libomptarget.rtl.opencl.so libstdc++.so.6.0.29 libz.so
libbz2.a libhistory.so.8.2 libmkl_blacs_openmpi_ilp64.so.2 libmkl_sequential.so libomptarget.rtl.x86_64.so libtbbmalloc_proxy.so.2 libz.so.1
libbz2.so libiomp5_db.so libmkl_blacs_openmpi_lp64.so libmkl_sequential.so.2 libomptarget.so libtbbmalloc_proxy.so.2.8 libz.so.1.2.13
libbz2.so.1.0 libiomp5.so libmkl_blacs_openmpi_lp64.so.2 libmkl_tbb_thread.so libomptarget.sycl.wrap.so libtbbmalloc.so.2 mkl_msg.cat
libbz2.so.1.0.8 libiompstubs5.so libmkl_cdft_core.so libmkl_tbb_thread.so.2 libpanel.a libtbbmalloc.so.2.8 ossl-modules
libcrypto.a libitm.so libmkl_cdft_core.so.2 libmkl_vml_avx2.so.2 libpanel.so libtbb.so.12 pkgconfig
libcrypto.so libitm.so.1 libmkl_core.so libmkl_vml_avx512.so.2 libpanel.so.6 libtbb.so.12.8 python3.1
libcrypto.so.3 libitm.so.1.0.0 libmkl_core.so.2 libmkl_vml_avx.so.2 libpanel.so.6.4 libtcl8.6.so python3.10
libffi.7.so liblsan.so libmkl_def.so.2 libmkl_vml_cmpt.so.2 libpanelw.a libtclstub8.6.a sqlite3.36.0
libffi.8.so liblsan.so.0 libmkl_gf_ilp64.so libmkl_vml_def.so.2 libpanelw.so libtinfo.a tcl8
libffi.a liblsan.so.0.0.0 libmkl_gf_ilp64.so.2 libmkl_vml_mc3.so.2 libpanelw.so.6 libtinfo.so tcl8.6
libffi.so liblzma.a libmkl_gf_lp64.so libmkl_vml_mc.so.2 libpanelw.so.6.4 libtinfo.so.6 tclConfig.sh
libffi.so.7 liblzma.so libmkl_gf_lp64.so.2 libncurses++.a libpython3.10.so libtinfo.so.6.4 tclooConfig.sh
libffi.so.8 liblzma.so.5 libmkl_gnu_thread.so libncurses.a libpython3.10.so.1.0 libtinfow.a tdbc1.1.3
libffi.so.8.1.2 liblzma.so.5.4.5 libmkl_gnu_thread.so.2 libncurses.so libpython3.so libtinfow.so tdbcmysql1.1.3
libform.a libmenu.a libmkl_intel_ilp64.so libncurses.so.6 libquadmath.so libtinfow.so.6 tdbcodbc1.1.3
libform.so libmenu.so libmkl_intel_ilp64.so.2 libncurses.so.6.4 libquadmath.so.0 libtinfow.so.6.4 tdbcpostgres1.1.3
libform.so.6 libmenu.so.6 libmkl_intel_lp64.so libncurses++w.a libquadmath.so.0.0.0 libtk8.6.so terminfo
libform.so.6.4 libmenu.so.6.4 libmkl_intel_lp64.so.2 libncursesw.a libreadline.a libtkstub8.6.a thread2.8.7
libformw.a libmenuw.a libmkl_intel_thread.so libncursesw.so libreadline.so libtsan.so tk8.6
libformw.so libmenuw.so libmkl_intel_thread.so.2 libncursesw.so.6 libreadline.so.8 libtsan.so.0 tkConfig.sh
libformw.so.6 libmenuw.so.6 libmkl_mc3.so.2 libncursesw.so.6.4 libreadline.so.8.2 libtsan.so.0.0.0
libformw.so.6.4 libmenuw.so.6.4 libmkl_mc.so.2 libomp-device-svml.o libsqlite3.so libubsan.so- Pytorch version: 2.1.2+cu121
- Package version: 0.8.0
- OS: Ubuntu 20.04.2
- Python version: Python 3.10
- Conda version (or N/A): conda 23.10.0
Compiling Information.
make build_inplace_mkl flags="-DUSE_CUDA=ON "
make[1]: Entering directory '/home/zhaoxian/Desktop/aihwkit'
make build_inplace flags="-DRPU_BLAS=MKL -DINTEL_MKL_DIR=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev -DUSE_CUDA=ON "
make[2]: Entering directory '/home/zhaoxian/Desktop/aihwkit'
python setup.py build_ext -j8 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE --inplace -DRPU_BLAS=MKL -DINTEL_MKL_DIR=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev -DUSE_CUDA=ON
--------------------------------------------------------------------------------
-- Trying 'Ninja' generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool.
-- Configuring incomplete, errors occurred!
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying 'Ninja' generator - failure
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
-- Trying 'Unix Makefiles' generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.
-- The C compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is GNU 9.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (0.3s)
-- Generating done (0.0s)
-- Build files have been written to: /home/zhaoxian/Desktop/aihwkit/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying 'Unix Makefiles' generator - success
--------------------------------------------------------------------------------
Configuring Project
Working directory:
/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build
Command:
/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/cmake/data/bin/cmake /home/zhaoxian/Desktop/aihwkit -G 'Unix Makefiles' --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-install -DPYTHON_VERSION_STRING:STRING=3.10.13 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/include/python3.10 -DPYTHON_LIBRARY:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libpython3.10.so -DPython_EXECUTABLE:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/bin/python -DPython_ROOT_DIR:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/include/python3.10 -DPython_NumPy_INCLUDE_DIRS:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/bin/python -DPython3_ROOT_DIR:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/include/python3.10 -DPython3_NumPy_INCLUDE_DIRS:PATH=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/numpy/core/include -DCMAKE_BUILD_TYPE=Debug -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DRPU_BLAS=MKL -DINTEL_MKL_DIR=/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev -DUSE_CUDA=ON
Not searching for unused variables given on the command line.
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Invoking cmake through scikit-build
-- The BLAS backend of choice:MKL
-- MKL_THREADING = OMP
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Checking for [mkl_intel_lp64 - mkl_gnu_thread - mkl_core - gomp - pthread - m - dl]
-- Library mkl_intel_lp64: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_intel_lp64.so
-- Library mkl_gnu_thread: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_gnu_thread.so
-- Library mkl_core: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_core.so
-- Library gomp: -fopenmp
-- Library pthread: /usr/lib/x86_64-linux-gnu/libpthread.so
-- Library m: /usr/lib/x86_64-linux-gnu/libm.so
-- Library dl: /usr/lib/x86_64-linux-gnu/libdl.so
-- Looking for cblas_sgemm
-- Looking for cblas_sgemm - found
-- MKL library found
-- Performing Test C_HAS_AVX_1
-- Performing Test C_HAS_AVX_1 - Failed
-- Performing Test C_HAS_AVX_2
-- Performing Test C_HAS_AVX_2 - Success
-- Performing Test C_HAS_AVX2_1
-- Performing Test C_HAS_AVX2_1 - Failed
-- Performing Test C_HAS_AVX2_2
-- Performing Test C_HAS_AVX2_2 - Success
-- Performing Test CXX_HAS_AVX_1
-- Performing Test CXX_HAS_AVX_1 - Failed
-- Performing Test CXX_HAS_AVX_2
-- Performing Test CXX_HAS_AVX_2 - Success
-- Performing Test CXX_HAS_AVX2_1
-- Performing Test CXX_HAS_AVX2_1 - Failed
-- Performing Test CXX_HAS_AVX2_2
-- Performing Test CXX_HAS_AVX2_2 - Success
-- AVX compiler support found
-- MKL include for RPU is /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_intel_lp64.so;/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_gnu_thread.so;/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_core.so;-fopenmp;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/x86_64-linux-gnu/libdl.so;/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/libmkl_avx2.so.2
-- Found Python3: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/bin/python (found version "3.10.13") found components: Interpreter Development Development.Module Development.Embed
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/pybind11/include (found version "2.11.1")
-- Found Python: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/bin/python (found version "3.10.13") found components: Interpreter
-- Found Torch: /home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/torch/include;/home/zhaoxian/anaconda3/envs/aihwkit-cuda-dev/lib/python3.10/site-packages/torch/include/torch/csrc/api/include
-- Set _GLIBCXX_USE_CXX11_ABI=0
-- The CUDA compiler identification is NVIDIA 11.3.109
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda-11.3/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found CUDAToolkit: /usr/local/cuda-11.3/targets/x86_64-linux/include (found version "11.3.109")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Configuring done (4.7s)
-- Generating done (0.0s)
-- Build files have been written to: /home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build
make[3]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[4]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
[ 1%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rng.cpp.o
[ 3%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_chopped_transfer_device.cpp.o
[ 3%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_buffered_transfer_device.cpp.o
[ 5%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/math_util.cpp.o
[ 6%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu.cpp.o
[ 7%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/dense_bit_line_maker.cpp.o
[ 10%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_dynamic_transfer_device.cpp.o
[ 8%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_constantstep_device.cpp.o
[ 11%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_expstep_device.cpp.o
[ 12%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_forward_backward_pass.cpp.o
[ 13%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_hidden_device.cpp.o
[ 15%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_linearstep_device.cpp.o
[ 16%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_mixedprec_device.cpp.o
[ 17%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_mixedprec_device_base.cpp.o
[ 18%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_mixedprec_int_device.cpp.o
[ 20%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_onesided_device.cpp.o
[ 21%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_piecewisestep_device.cpp.o
[ 22%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_powstep_device.cpp.o
[ 24%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_powstep_reference_device.cpp.o
[ 25%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_pulsed.cpp.o
[ 26%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_pulsed_device.cpp.o
[ 27%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_pulsed_meta_parameter.cpp.o
[ 29%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_simple_device.cpp.o
[ 30%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_softbounds_reference_device.cpp.o
[ 31%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_transfer_device.cpp.o
[ 32%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_vector_device.cpp.o
[ 34%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/rpu_weight_updater.cpp.o
[ 35%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/sparse_bit_line_maker.cpp.o
[ 36%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/utility_functions.cpp.o
[ 37%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/weight_clipper.cpp.o
[ 39%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/weight_drifter.cpp.o
[ 40%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/weight_modifier.cpp.o
[ 41%] Building CXX object CMakeFiles/RPU_CPU.dir/src/rpucuda/weight_remapper.cpp.o
[ 43%] Linking CXX static library libRPU_CPU.a
make[5]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
[ 43%] Built target RPU_CPU
make[5]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
[ 45%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/cuda_buffer.cu.o
[ 45%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/chopped_weight_output.cu.o
[ 46%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/cuda_math_util.cu.o
[ 48%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/io_manager.cu.o
[ 49%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/bit_line_maker.cu.o
[ 50%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/maximizer.cu.o
[ 53%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/forward_backward_pass.cu.o
[ 53%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/cuda_util.cu.o
[ 54%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/noise_manager.cu.o
[ 55%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/pulsed_weight_updater.cu.o
[ 56%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda.cu.o
[ 58%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_buffered_transfer_device.cu.o
[ 59%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_chopped_transfer_device.cu.o
[ 60%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_constantstep_device.cu.o
[ 62%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_dynamic_transfer_device.cu.o
[ 63%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_expstep_device.cu.o
[ 64%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_hidden_device.cu.o
[ 65%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_linearstep_device.cu.o
[ 67%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_mixedprec_device.cu.o
[ 68%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_mixedprec_device_base.cu.o
[ 69%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_mixedprec_int_device.cu.o
[ 70%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_onesided_device.cu.o
[ 72%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_piecewisestep_device.cu.o
[ 73%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_powstep_device.cu.o
[ 74%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_powstep_reference_device.cu.o
[ 75%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_pulsed.cu.o
[ 77%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_pulsed_device.cu.o
[ 78%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_simple_device.cu.o
[ 79%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_softbounds_reference_device.cu.o
[ 81%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_transfer_device.cu.o
[ 82%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/rpucuda_vector_device.cu.o
[ 83%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/test_helper.cu.o
[ 84%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/update_management_helper.cu.o
[ 86%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/weight_clipper_cuda.cu.o
[ 87%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/weight_drifter_cuda.cu.o
[ 88%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/weight_modifier_cuda.cu.o
[ 89%] Building CUDA object CMakeFiles/RPU_GPU.dir/src/rpucuda/cuda/weight_remapper_cuda.cu.o
[ 91%] Linking CUDA device code CMakeFiles/RPU_GPU.dir/cmake_device_link.o
[ 92%] Linking CXX static library libRPU_GPU.a
make[5]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
[ 92%] Built target RPU_GPU
make[5]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
make[5]: Entering directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
[ 93%] Building CXX object src/aihwkit/simulator/CMakeFiles/rpu_base.dir/rpu_base_src/rpu_base_devices.cpp.o
[ 94%] Building CXX object src/aihwkit/simulator/CMakeFiles/rpu_base.dir/rpu_base_src/rpu_base.cpp.o
[ 97%] Building CXX object src/aihwkit/simulator/CMakeFiles/rpu_base.dir/rpu_base_src/rpu_base_tiles.cpp.o
[ 97%] Building CXX object src/aihwkit/simulator/CMakeFiles/rpu_base.dir/rpu_base_src/rpu_base_utils.cpp.o
[ 98%] Building CXX object src/aihwkit/simulator/CMakeFiles/rpu_base.dir/rpu_base_src/rpu_base_tiles_cuda.cpp.o
[100%] Linking CXX shared module rpu_base.cpython-310-x86_64-linux-gnu.so
Processed 1 modules
Generated ./src/aihwkit/simulator/__init__.pyi
make[5]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
[100%] Built target rpu_base
make[4]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
Install the project...
-- Install configuration: "Debug"
-- Installing: /home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-install/src/aihwkit/simulator/rpu_base.cpython-310-x86_64-linux-gnu.so
-- Set non-toolchain portion of runtime path of "/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-install/src/aihwkit/simulator/rpu_base.cpython-310-x86_64-linux-gnu.so" to "$ORIGIN"
make[3]: Leaving directory '/home/zhaoxian/Desktop/aihwkit/_skbuild/linux-x86_64-3.10/cmake-build'
copying _skbuild/linux-x86_64-3.10/cmake-install/src/aihwkit/simulator/rpu_base.cpython-310-x86_64-linux-gnu.so -> src/aihwkit/simulator/rpu_base.cpython-310-x86_64-linux-gnu.so
running build_ext
make[2]: Leaving directory '/home/zhaoxian/Desktop/aihwkit'
make[1]: Leaving directory '/home/zhaoxian/Desktop/aihwkit'kkvtran commented
Would you try with export CMAKE_PREFIX_PATH=$CONDA_PREFIX?
Zhaoxian-Wu commented
Would you try with export CMAKE_PREFIX_PATH=$CONDA_PREFIX?
I tried it today but I got the same error. And I guess the mkl is not related to the CMAKE_PREFIX_PATH?
jubueche commented
Could you start completely from scratch and follow my instructions exactly? Because for me it is working fine.
jubueche commented
Any updates on this? I see that you are building this in your Desktop folder. Maybe try home instead. It's pretty hard for me to reproduce this.
Zhaoxian-Wu commented
Any updates on this? I see that you are building this in your Desktop folder. Maybe try home instead. It's pretty hard for me to reproduce this.
I tried it again in home ~/, but the same thing happened. Is it because I am missing any environment variable about MKL?
Since the compile process works well but the running process doesn't, I guess it's because I forgot to set some environment. Do you have any idea where the Python program finds libmkl_intel_lp64.so.2?
Zhaoxian-Wu commented
I found there is $COMPILER variable in your $PATH (#545) . It's an empty string on my machine so I detele it. Could you check what the variable is on your machine?
kkvtran commented
Since you saw this problem at runtime, I wonder if you need to set LD_LIBRARY_PATH=$CONDA_PREFIX/lib
Honestly, I know LD_LIBRARY_PATH is applicable on Linux. I do not know if Mac OS has the same environment variable.
Zhaoxian-Wu commented
I managed to run it. The reason is I didn't specify the variable LD_LIBRARY_PATH as question #485 suggests. After adding export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH the issue is solved. The following is the step-by-step operation
conda create -n aihwkit-cuda-dev python=3.10 -y
conda activate aihwkit-cuda-dev
git clone https://github.com/IBM/aihwkit.git ; cd aihwkit
pip install -r requirements.txt
conda install mkl mkl-include -y
export CXX=/usr/bin/g++
export CC=/usr/bin/gcc
export MKLROOT=$CONDA_PREFIX
export CMAKE_PREFIX_PATH=$CONDA_PREFIX
export CUDA_VERSION=11.1
export CUDA_HOME=/usr/local/cuda-${CUDA_VERSION}
export CUDA_TOOLKIT_ROOT_DIR=${CUDA_HOME}
export CUDA_LIB_PATH=${CUDA_HOME}/lib64
export CUDA_INCLUDE_DIRS=${CUDA_HOME}/include
export PATH=${CUDA_HOME}/bin:${PATH}
export LD_LIBRARY_PATH=$CONDA_PREFIX/lib:$LD_LIBRARY_PATH
make build_inplace_cudaI guess it would be helpful for the developers if these tricky details are mentioned on the documents :D
Zhaoxian-Wu commented
Since you saw this problem at runtime, I wonder if you need to set LD_LIBRARY_PATH=$CONDA_PREFIX/lib Honestly, I know LD_LIBRARY_PATH is applicable on Linux. I do not know if Mac OS has the same environment variable.
Thanks for your command! It works perfectly.