make test error 1
jnghhk opened this issue · 2 comments
TensorFlow installed from (source or binary): pip (conda env)
TensorFlow version (use command below): 1.8.0
Python version: 2.7
GCC/Compiler version (if compiling from source):5.4.0
CUDA/cuDNN version:9/7.1.2
GPU model and memory: GTX1060 6G
Exact command to reproduce: make test
cd ../benchmark && bash run_all_unittests.bash # unit tests
sparse_res_block_tests (unittest.loader.ModuleImportFailure) ... ERROR
reduce_mask_tests (unittest.loader.ModuleImportFailure) ... ERROR
sparse_conv_tests (unittest.loader.ModuleImportFailure) ... ERROR
sparse_scatter_tests (unittest.loader.ModuleImportFailure) ... ERROR
sparse_gather_tests (unittest.loader.ModuleImportFailure) ... ERROR
test_calc_out_size (tf_conv_dims_tests.CalcOutSizeDeconvTests) ... ok
test_session (tf_conv_dims_tests.CalcOutSizeDeconvTests)
Returns a TensorFlow Session for use in executing tests. ... ok
test_calc_out_size (tf_conv_dims_tests.CalcOutSizeTests) ... ok
test_session (tf_conv_dims_tests.CalcOutSizeTests)
Returns a TensorFlow Session for use in executing tests. ... ok
test_calc_padding (tf_conv_dims_tests.CalcPaddingTests) ... ok
test_calc_padding_err_ksize_list (tf_conv_dims_tests.CalcPaddingTests) ... ok
test_calc_padding_err_strides_list (tf_conv_dims_tests.CalcPaddingTests) ... ok
test_calc_padding_err_strides_tensor (tf_conv_dims_tests.CalcPaddingTests) ... ok
test_calc_padding_stride (tf_conv_dims_tests.CalcPaddingTests) ... ok
test_calc_padding_valid (tf_conv_dims_tests.CalcPaddingTests) ... ok
test_session (tf_conv_dims_tests.CalcPaddingTests)
Returns a TensorFlow Session for use in executing tests. ... ok
======================================================================
ERROR: sparse_res_block_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_res_block_tests
Traceback (most recent call last):
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_res_block_tests.py", line 32, in
from sparse_conv_lib import _get_offset_array
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
======================================================================
ERROR: reduce_mask_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: reduce_mask_tests
Traceback (most recent call last):
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/reduce_mask_tests.py", line 25, in
from sparse_conv_lib import convert_mask_to_indices, convert_mask_to_indices_custom
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
======================================================================
ERROR: sparse_conv_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_conv_tests
Traceback (most recent call last):
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_tests.py", line 29, in
from sparse_conv_lib import _get_offset_array
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
======================================================================
ERROR: sparse_scatter_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_scatter_tests
Traceback (most recent call last):
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_scatter_tests.py", line 26, in
from sparse_conv_lib import sbnet_module
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
======================================================================
ERROR: sparse_gather_tests (unittest.loader.ModuleImportFailure)
ImportError: Failed to import test module: sparse_gather_tests
Traceback (most recent call last):
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 254, in _find_tests
module = self._get_module_from_name(name)
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/unittest/loader.py", line 232, in _get_module_from_name
import(name)
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_gather_tests.py", line 26, in
from sparse_conv_lib import convert_mask_to_block_indices, convert_mask_to_indices_custom
File "/home/jnghhk/PycharmProjects/py2/sbnet/sbnet_tensorflow/benchmark/sparse_conv_lib.py", line 74, in
sbnet_module = tf.load_op_library('../sbnet_ops/libsbnet.so')
File "/home/jnghhk/anaconda3/envs/py2/lib/python2.7/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
NotFoundError: ../sbnet_ops/libsbnet.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
Ran 16 tests in 4.775s
FAILED (errors=5)
Makefile:14: recipe for target 'test' failed
make: *** [test] Error 1
Hmm.. Looks like an ABI mismatch problem. Possibly your tensorflow wheel was built with a different gcc version from what you are using to build sbnet custom ops.
tensorflow/tensorflow#9137
In this TF issue they are missing the same exact symbol and the solution appears to be using
-D_GLIBCXX_USE_CXX11_ABI=0 during build for both the TF pip and the custom op.