uxlfoundation/scikit-learn-intelex

Building oneDAL with icpx Compiler Fails to Build scikit-learn-intelex Patch

rakshithgb-fujitsu opened this issue · 5 comments

rakshithgb-fujitsu commented

Description:

I'm encountering an issue when building this intelex patch when using oneDAL built with icpx compiler.

Environment:

Compiler: icpx
Python version: 3.10
Platform: Linux (x86_64)

Steps to Reproduce:

Use the following make command to build oneDAL:

make -f makefile daal oneapi_c BACKEND_CONFIG=mkl ARCH=32e PLAT=lnx32e COMPILER=icx -j$(nproc)

Attempt to build the scikit-learn-intelex patch using:

python3.10 setup.py install --single-version-externally-managed --record=record.txt

Expected Result:

The scikit-learn-intelex patch should build successfully without any errors.

Actual Result:

The build process fails with the following error messages:

[100%] Linking CXX shared module _onedal_py_dpc.cpython-310-x86_64-linux-gnu.so
/usr/bin/ld: cannot find -l:libonedal_dpc.so.2: No such file or directory
/usr/bin/ld: cannot find -l:libonedal_parameters_dpc.so.2: No such file or directory
icpx: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[2]: *** [CMakeFiles/_onedal_py_dpc.dir/build.make:513: _onedal_py_dpc.cpython-310-x86_64-linux-gnu.so] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/_onedal_py_dpc.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

Additional Information:

  • The error indicates missing libraries: libonedal_dpc.so.2 and libonedal_parameters_dpc.so.2. These libraries are not generated during the oneDAL build process.
  • The patch builds successfully when using the GNU compiler.
ethanglaser commented

Hi @rakshithgb-fujitsu, there are 2 options based on your situation:

  1. You intend to build full scope (or at least dpc backend) of sklearnex: in this case the OneDAL build command is not sufficient because onedal dpc libs are required - you can modify make -f makefile daal oneapi_c BACKEND_CONFIG=mkl ARCH=32e PLAT=lnx32e COMPILER=icx -j$(nproc) to instead be make -f makefile onedal_dpc BACKEND_CONFIG=mkl ARCH=32e PLAT=lnx32e COMPILER=icx -j$(nproc)
  2. You intend to only build host backend for sklearnex: in this case there may be a change from our side required to allow for sklearnex build without dpc backend

Please let me know if the second one applies - if so I can look into reproducing and providing a conditional in sklearnex that will not require dpc.

rakshithgb-fujitsu commented

@ethanglaser thanks for the inputs, yes building with onedal_dpc option works. It would be nice to have the second option implemented as well, I see a new pull request for this. Will wait for the fix in meantime.

rakshithgb-fujitsu commented

@ethanglaser though build step works, during runtime I get import errors:

from daal4py._daal4py import *
ImportError: libsvml.so: cannot open shared object file: No such file or directory
Alexsandruss commented

Building oneDAL with icx leads to dependency on some compiler shared libs. This dependency is inherited by daal4py shared library. I guess it will be fixed by linking options in oneDAL and/or daal4py.