Unable to compile
Closed this issue · 4 comments
ggoupy commented
Hello.
I am trying to build the Python interface but I run across a compilation error :
Scanning dependencies of target spyker
[ 92%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/backprop.cpp.o
[ 92%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/canny.cpp.o
[ 93%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/fire.cpp.o
[ 93%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/code.cpp.o
[ 93%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/conv.cpp.o
[ 93%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/dog.cpp.o
[ 93%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/fc.cpp.o
[ 93%] Building CXX object CMakeFiles/spyker.dir/src/spyker/cpu/inhibit.cpp.o
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/fc.cpp: In constructor ‘Spyker::Core::CPU::FC::FC(Spyker::Len3, Spyker::Len2, Spyker::Len3, Spyker::Type)’:
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/fc.cpp:36:23: error: ‘desc’ is not a member of ‘dnnl::matmul’
36 | dnnl::matmul::desc desc(input, kernel, output);
| ^~~~
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/fc.cpp:37:43: error: ‘desc’ was not declared in this scope
37 | dnnl::matmul::primitive_desc prim(desc, onednn_static->engine);
| ^~~~
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp: In constructor ‘Spyker::Core::CPU::Conv::Conv(Spyker::Len4, Spyker::Len4, Spyker::Len4, Spyker::Len2, Spyker::Len4, Spyker::Type)’:
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:38:36: error: ‘desc’ is not a member of ‘dnnl::convolution_forward’
38 | dnnl::convolution_forward::desc desc(dnnl::prop_kind::forward_inference, //
| ^~~~
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:40:63: error: expected primary-expression before ‘,’ token
40 | input, kernel, {}, output, {_stride.y, _stride.x}, //
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:40:73: error: expected primary-expression before ‘{’ token
40 | input, kernel, {}, output, {_stride.y, _stride.x}, //
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:40:95: error: expected primary-expression before ‘,’ token
40 | input, kernel, {}, output, {_stride.y, _stride.x}, //
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:41:46: error: expected primary-expression before ‘{’ token
41 | {_pad.t, _pad.z}, {_pad.y, _pad.x});
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:41:62: error: expected primary-expression before ‘,’ token
41 | {_pad.t, _pad.z}, {_pad.y, _pad.x});
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:41:64: error: expected primary-expression before ‘{’ token
41 | {_pad.t, _pad.z}, {_pad.y, _pad.x});
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:41:80: error: expected primary-expression before ‘)’ token
41 | {_pad.t, _pad.z}, {_pad.y, _pad.x});
| ^
/tmp/pip-req-build-lhj6rq17/src/spyker/cpu/conv.cpp:42:56: error: ‘desc’ was not declared in this scope
42 | dnnl::convolution_forward::primitive_desc prim(desc, onednn_static->engine);
| ^~~~
make[2]: *** [CMakeFiles/spyker.dir/build.make:128: CMakeFiles/spyker.dir/src/spyker/cpu/fc.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [CMakeFiles/spyker.dir/build.make:102: CMakeFiles/spyker.dir/src/spyker/cpu/conv.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:289: CMakeFiles/spyker.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-lhj6rq17/setup.py", line 99, in <module>
setup(
File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3/dist-packages/wheel/bdist_wheel.py", line 223, in run
self.run_command('build')
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/tmp/pip-req-build-lhj6rq17/setup.py", line 39, in run
self.build_extension(ext)
File "/tmp/pip-req-build-lhj6rq17/setup.py", line 96, in build_extension
subprocess.check_call(['cmake', '--build', '.'] + build_args, cwd=self.build_temp)
File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.', '--', '-j8']' returned non-zero exit status 2.
error
ERROR: Failed building wheel for spyker
Running setup.py clean for spyker
Running command /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-lhj6rq17/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-lhj6rq17/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
running clean
removing 'build/temp.linux-x86_64-3.8' (and everything under it)
removing 'build/lib.linux-x86_64-3.8' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.8' does not exist -- can't clean it
removing 'build'
Failed to build spyker
Cleaning up...
Removing source in /tmp/pip-req-build-lhj6rq17
Removed build tracker: '/tmp/pip-req-tracker-5vwbnst5'
ERROR: Failed to build one or more wheels
Exception information:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_internal/cli/base_command.py", line 186, in _main
status = self.run(options, args)
File "/usr/lib/python3/dist-packages/pip/_internal/commands/wheel.py", line 188, in run
raise CommandError(
pip._internal.exceptions.CommandError: Failed to build one or more wheels
Here is the config :
['cmake', '/tmp/pip-req-build-lhj6rq17', '-DENABLE_PYTHON=ON', '-DENABLE_TESTS=OFF', '-DENABLE_EXAMPLES=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-req-build-lhj6rq17/build/lib.linux-x86_64-3.8/spyker/', '-DPYTHON_EXECUTABLE=/usr/bin/python3', '-DCMAKE_BUILD_TYPE=Release']
-- The CXX compiler identification is GNU 9.4.0
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Looking for cblas_saxpy
-- Looking for cblas_saxpy - found
-- Found CBLAS: /usr/lib/x86_64-linux-gnu/libopenblas.so
-- Looking for cheev_
-- Looking for cheev_ - found
-- A library with LAPACK API found.
-- Looking for LAPACKE_dgels
-- Looking for LAPACKE_dgels - not found
-- Found LAPACKE: /usr/lib/x86_64-linux-gnu/liblapacke.so
CMake Warning (dev) at CMakeLists.txt:83 (set):
implicitly converting 'LIST' to 'STRING' type.
This warning is for project developers. Use -Wno-dev to suppress it.
-- The C compiler identification is GNU 9.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- DNNL_TARGET_ARCH: X64
-- DNNL_LIBRARY_NAME: dnnl
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Could NOT find Doxyrest (missing: DOXYREST_EXECUTABLE)
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.10", minimum required is "2.7")
-- Could NOT find Sphinx (missing: SPHINX_EXECUTABLE)
-- Found Git: /usr/bin/git (found version "2.25.1")
-- Enabled workload: INFERENCE
-- Enabled primitives: CONVOLUTION;MATMUL
-- Enabled primitive CPU ISA: ALL
-- Enabled primitive GPU ISA: ALL
-- Primitive cache is enabled
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - NOTFOUND
-- Found OpenMP_CXX: -fopenmp
-- Found OpenMP: TRUE
-- pybind11 v2.11.0 dev1
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.8.10", minimum required is "3.6")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.8.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
--
-- Summary:
-- Optimization Flags: ------------ -march=native
-- CUDA Archutecutre List: --------
-- Python Interface: -------------- ON
-- CUDA: -------------------------- OFF
-- CUDNN: ------------------------- OFF
-- DNNL: -------------------------- ON
-- BLAS: -------------------------- ON
--
-- Configuring done
Do you know what the cause of the issue could be ?
ShahriarRezghi commented
Hi @ggoupy
Thanks for opening this issue. The error says that error: ‘desc’ is not a member of ‘dnnl::matmul’, but this is not the case. This leads me to believe that you are using an incompatible version of dnnl, and not the version that is a git submodule. How did you obtain the source code? did you clone the repo recursively?
ggoupy commented
First of all, the command git clone --recursive git@github.com:Mathific/Spyker.git to clone the project in the installation guide does not work for me : public key denied.
So I cloned the repo using git clone --recursive https://github.com/ShahriarRezghi/Spyker but I obtained the same error for the 3rd party library blasw :
Cloning into '../Spyker/3rd/blasw'...
git@github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:Mathific/Blasw.git' into submodule path '../Spyker/3rd/blasw' failed
Failed to clone '3rd/blasw' a second time, aborting
I ended up cloning manually each 3rd party library.
I suppose the issue is that I use the latest version of oneDNN and not the one from this commit f35a62f9b3c1db5ce8a2704e530e050b2f4b1807.
EDIT :
Indeed, I successfully compiled but I have an error when importing spyker (import spyker) :
~/.local/lib/python3.8/site-packages/spyker/__init__.py in <module>
----> 1 from spyker.spyker_plugin.control import *
2 from spyker.spyker_plugin import version, device, tensor, sparse_tensor
3 from spyker.utils import create_tensor, wrap_array, copy_array, to_tensor, to_numpy, to_torch, read_mnist
4 from spyker.module import *
5 import spyker.sparse
/home/<name>/.local/lib/python3.8/site-packages/spyker/spyker_plugin.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN6Spyker4Core17cuda_conv_optionsElll
even if I compiled the package without CUDA.
ShahriarRezghi commented
This was a bug which I fixed. It should work now.
ggoupy commented
It works fine now. Thank you for quickly resolving the issue, keep up the good work!