Symbol not found, but versions are compatible according to the website
koenichiwa opened this issue · 4 comments
koenichiwa commented
Hey I just did a fresh install of miniconda, and used pip to install tensorflow and tensorflow_decision_forests Some of the packages that are installed now are
tensorboard 2.13.0
tensorboard-data-server 0.7.1
tensorflow 2.13.0
tensorflow-decision-forests 1.5.0
tensorflow-estimator 2.13.0
tensorflow-io-gcs-filesystem 0.34.0
I opened my notebook and when I imported both packages I get the following error:
WARNING:root:Failure to load the inference.so custom c++ tensorflow ops. This error is likely caused the version of TensorFlow and TensorFlow Decision Forests are not compatible. Full error:dlopen(/Users/koen/miniconda3/envs/kaggle/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/koen/miniconda3/envs/kaggle/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
The weird thing is is that I have macos 11.2.3. It seems that it doesn't build the right version
rstz commented
Hi, that's odd. I'm not very familiar with the conda environment, and I don't have a mac with macos 11.2.3 so debugging is going to be a bit hard. Could you please try with TF-DF 2.14. (and TF 2.12), as well as TF-DF 2.16.0rc0 and TF 2.14.0rc0 and report if that works?
koenichiwa commented
Hi thanks for the fast reply!
I'm not very familiar with the conda environment
I only used it to create an environment with python 3.11 and pip, so that I could remove everything if the install went wrong (I've been trying to install this for 3 days now, first I did it with my older, more heavy duty iMac, but now I'm on my macbook)
I was mostly confused by the line which was built for Mac OS X 12.3 as I'm on 11.2.3.
Anyhow, I'm tried the other versions. This is the full output of 1.16.0rc0/2.14.0rc0
WARNING:root:TensorFlow Decision Forests 1.6.0rc0 is compatible with the following TensorFlow Versions: ['2.14.0rc0', '2.14.0rc1', '2.14.0rc2', '2.14.0']. However, TensorFlow 2.14.0-rc0 was detected. This can cause issues with the TF API and symbols in the custom C++ ops. See the TF and TF-DF compatibility table at https://github.com/tensorflow/decision-forests/blob/main/documentation/known_issues.md#compatibility-table.
WARNING:root:Failure to load the inference.so custom c++ tensorflow ops. This error is likely caused the version of TensorFlow and TensorFlow Decision Forests are not compatible. Full error:dlopen(/Users/koenvanwel/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/koenvanwel/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
Cell In[2], line 2
1 import tensorflow as tf
----> 2 import tensorflow_decision_forests as tfdf
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/__init__.py:64
60 from tensorflow_decision_forests.tensorflow import check_version
62 check_version.check_version(__version__, compatible_tf_versions)
---> 64 from tensorflow_decision_forests import keras
65 from tensorflow_decision_forests.component import py_tree
66 from tensorflow_decision_forests.component.builder import builder
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/keras/__init__.py:53
15 """Decision Forest in a Keras Model.
16
17 Usage example:
(...)
48 ```
49 """
51 from typing import Callable, List
---> 53 from tensorflow_decision_forests.keras import core
54 from tensorflow_decision_forests.keras import wrappers
56 # Utility classes
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/keras/core.py:61
59 from tensorflow_decision_forests.component.inspector import inspector as inspector_lib
60 from tensorflow_decision_forests.component.tuner import tuner as tuner_lib
---> 61 from tensorflow_decision_forests.keras import core_inference
62 from tensorflow_decision_forests.tensorflow import core as tf_core
63 from tensorflow_decision_forests.tensorflow import tf1_compatibility
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/keras/core_inference.py:36
34 from tensorflow_decision_forests.tensorflow import core_inference as tf_core
35 from tensorflow_decision_forests.tensorflow import tf_logging
---> 36 from tensorflow_decision_forests.tensorflow.ops.inference import api as tf_op
37 from yggdrasil_decision_forests.learner import abstract_learner_pb2
38 from yggdrasil_decision_forests.learner.multitasker import multitasker_pb2
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/api.py:179
177 from tensorflow_decision_forests.component.inspector import inspector as inspector_lib
178 from tensorflow_decision_forests.tensorflow import tf1_compatibility
--> 179 from tensorflow_decision_forests.tensorflow.ops.inference import op
180 from yggdrasil_decision_forests.dataset import data_spec_pb2
181 from yggdrasil_decision_forests.model import abstract_model_pb2
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/op.py:15
1 # Copyright 2021 Google LLC.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 15 from tensorflow_decision_forests.tensorflow.ops.inference.op_dynamic import *
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/op_dynamic.py:24
22 except Exception as e:
23 check_version.info_fail_to_load_custom_op(e, "inference.so")
---> 24 raise e
26 # Importing all the symbols.
27 module = sys.modules[__name__]
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/op_dynamic.py:21
18 import sys
20 try:
---> 21 ops = tf.load_op_library(resource_loader.get_path_to_datafile("inference.so"))
22 except Exception as e:
23 check_version.info_fail_to_load_custom_op(e, "inference.so")
File ~/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow/python/framework/load_library.py:54, in load_op_library(library_filename)
31 @tf_export('load_op_library')
32 def load_op_library(library_filename):
33 """Loads a TensorFlow plugin, containing custom ops and kernels.
34
35 Pass "library_filename" to a platform-specific mechanism for dynamically
(...)
52 RuntimeError: when unable to load the library or get the python wrappers.
53 """
---> 54 lib_handle = py_tf.TF_LoadLibrary(library_filename)
55 try:
56 wrappers = _pywrap_python_op_gen.GetPythonWrappers(
57 py_tf.TF_GetOpList(lib_handle))
NotFoundError: dlopen(/Users/koenvanwel/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/koenvanwel/miniconda3/envs/tf/lib/python3.11/site-packages/tensorflow_decision_forests/tensorflow/ops/inference/inference.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
pip install tensorflow==2.12.0 tensorflow_decision_forests==1.4.0
Threw the following error:
ERROR: Ignored the following versions that require a different python version: 0.28.0 Requires-Python >=3.7, <3.11; 1.21.2 Requires-Python >=3.7,<3.11; 1.21.3 Requires-Python >=3.7,<3.11; 1.21.4 Requires-Python >=3.7,<3.11; 1.21.5 Requires-Python >=3.7,<3.11; 1.21.6 Requires-Python >=3.7,<3.11
ERROR: Could not find a version that satisfies the requirement tensorflow-macos~=2.12.0 (from tensorflow-decision-forests) (from versions: none)
ERROR: No matching distribution found for tensorflow-macos~=2.12.0
I even tried to compile from source
ERROR: Could not find a version that satisfies the requirement tensorflow==mac-intel-crosscompile (from versions: 2.12.0rc0, 2.12.0rc1, 2.12.0, 2.12.1, 2.13.0rc0, 2.13.0rc1, 2.13.0rc2, 2.13.0, 2.14.0rc0, 2.14.0rc1)
I'm currently trying v1.0.1/2.10.1 but the step Building wheel for grpcio (setup.py) is taking a long time
Could I help with any other information?
koenichiwa commented
Ah 1.0.1 throws the following, similar error over a different .so file
2023-09-16 01:05:50.207620: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING:root:TensorFlow Decision Forests 1.0.1 is compatible with the following TensorFlow Versions: ['2.10.0']. However, TensorFlow 2.10.1 was detected. This can cause issues with the TF API and symbols in the custom C++ ops. See the TF and TF-DF compatibility table at https://github.com/tensorflow/decision-forests/blob/main/documentation/known_issues.md#compatibility-table.
WARNING:root:Failure to load the training.so custom c++ tensorflow ops. This error is likely caused the version of TensorFlow and TensorFlow Decision Forests are not compatible. Full error:dlopen(/Users/koenvanwel/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/training.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/koenvanwel/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/training.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
Cell In[1], line 2
1 import tensorflow as tf
----> 2 import tensorflow_decision_forests as tfdf
3 import pandas as pd
4 import seaborn as sns
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/__init__.py:63
59 from tensorflow_decision_forests.tensorflow import check_version
61 check_version.check_version(__version__, compatible_tf_versions)
---> 63 from tensorflow_decision_forests import keras
64 from tensorflow_decision_forests.component import py_tree
65 from tensorflow_decision_forests.component.builder import builder
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/keras/__init__.py:54
15 """Decision Forest in a Keras Model.
16
17 Usage example:
(...)
49
50 """
52 from typing import Callable, List
---> 54 from tensorflow_decision_forests.keras import core
55 from tensorflow_decision_forests.keras import wrappers
57 # Utility classes
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/keras/core.py:59
57 from tensorflow_decision_forests.component.inspector import inspector as inspector_lib
58 from tensorflow_decision_forests.component.tuner import tuner as tuner_lib
---> 59 from tensorflow_decision_forests.keras import core_inference
60 from tensorflow_decision_forests.tensorflow import core as tf_core
61 from tensorflow_decision_forests.tensorflow import tf1_compatibility
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/keras/core_inference.py:31
29 from tensorflow_decision_forests.component.inspector import inspector as inspector_lib
30 from tensorflow_decision_forests.tensorflow import core_inference as tf_core
---> 31 from tensorflow_decision_forests.tensorflow import tf_logging
32 from tensorflow_decision_forests.tensorflow.ops.inference import api as tf_op
33 from yggdrasil_decision_forests.learner import abstract_learner_pb2
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/tf_logging.py:26
23 from typing import Union, Any, List
24 from absl import logging
---> 26 from tensorflow_decision_forests.tensorflow.ops.training import op as training_op
28 # Background
29 # ==========
30 #
(...)
52 # the redirection is setup; false positive). If you face one of those
53 # situations, please ping us.
54 REDIRECT_YGGDRASIL_CPP_OUTPUT_TO_PYTHON_OUTPUT = "auto"
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/op.py:15
1 # Copyright 2021 Google LLC.
2 #
3 # Licensed under the Apache License, Version 2.0 (the "License");
(...)
12 # See the License for the specific language governing permissions and
13 # limitations under the License.
---> 15 from tensorflow_decision_forests.tensorflow.ops.training.op_dynamic import *
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/op_dynamic.py:24
22 except Exception as e:
23 check_version.info_fail_to_load_custom_op(e, "training.so")
---> 24 raise e
26 # Importing all the symbols.
27 module = sys.modules[__name__]
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/op_dynamic.py:21
18 import sys
20 try:
---> 21 ops = tf.load_op_library(resource_loader.get_path_to_datafile("training.so"))
22 except Exception as e:
23 check_version.info_fail_to_load_custom_op(e, "training.so")
File ~/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow/python/framework/load_library.py:54, in load_op_library(library_filename)
31 @tf_export('load_op_library')
32 def load_op_library(library_filename):
33 """Loads a TensorFlow plugin, containing custom ops and kernels.
34
35 Pass "library_filename" to a platform-specific mechanism for dynamically
(...)
52 RuntimeError: when unable to load the library or get the python wrappers.
53 """
---> 54 lib_handle = py_tf.TF_LoadLibrary(library_filename)
55 try:
56 wrappers = _pywrap_python_op_gen.GetPythonWrappers(
57 py_tf.TF_GetOpList(lib_handle))
NotFoundError: dlopen(/Users/koenvanwel/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/training.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/koenvanwel/miniconda3/envs/tfdf/lib/python3.10/site-packages/tensorflow_decision_forests/tensorflow/ops/training/training.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
rstz commented
Hi, one more thing I need to debug this: Are you using an Intel mac or an Arm64 (M1 / M2) machine? I just checked and everything works well on my arm64 machine, but I don't have an Intel machine available to debug on that. Many packages have been cross-compiled. With this information, I might also be able to guide you through self-compilation.