Colab Notebook : NotFoundError: metrics/_text_similarity_metric_ops.so: undefined symbol:

[mmcs-work]

Running the first code block in t5-trivia notebook, would lead to this error.

To Reproduce
Steps to reproduce the behavior:

1. Execute the notebook first cell. (where GCS set up is done)

Additional context
Here is the detailed error message:

Installing dependencies...
---------------------------------------------------------------------------
NotFoundError                             Traceback (most recent call last)
<ipython-input-3-4f9ae8526295> in <module>()
     12 import tensorflow_datasets as tfds
     13 
---> 14 import t5
     15 import t5.models
     16 import seqio

9 frames
/usr/local/lib/python3.7/dist-packages/t5/__init__.py in <module>()
     15 """Import API modules."""
     16 
---> 17 import t5.data
     18 import t5.evaluation
     19 

/usr/local/lib/python3.7/dist-packages/t5/data/__init__.py in <module>()
     15 """Import data modules."""
     16 # pylint:disable=wildcard-import,g-bad-import-order
---> 17 from t5.data.dataset_providers import *
     18 from t5.data.glue_utils import *
     19 import t5.data.postprocessors

/usr/local/lib/python3.7/dist-packages/t5/data/dataset_providers.py in <module>()
     26 import re
     27 
---> 28 import seqio
     29 from t5.data import utils
     30 import tensorflow.compat.v2 as tf

/usr/local/lib/python3.7/dist-packages/seqio/__init__.py in <module>()
     16 # pylint:disable=wildcard-import,g-bad-import-order
     17 
---> 18 from seqio.dataset_providers import *
     19 from seqio import evaluation
     20 from seqio import experimental

/usr/local/lib/python3.7/dist-packages/seqio/dataset_providers.py in <module>()
     33 from seqio import utils
     34 from seqio.feature_converters import FeatureConverter
---> 35 from seqio.vocabularies import PassThroughVocabulary
     36 from seqio.vocabularies import Vocabulary
     37 import tensorflow.compat.v2 as tf

/usr/local/lib/python3.7/dist-packages/seqio/vocabularies.py in <module>()
     21 from absl import logging
     22 import tensorflow.compat.v2 as tf
---> 23 import tensorflow_text as tf_text
     24 
     25 from sentencepiece import sentencepiece_model_pb2

/usr/local/lib/python3.7/dist-packages/tensorflow_text/__init__.py in <module>()
     19 # pylint: disable=wildcard-import
     20 from tensorflow_text.python import keras
---> 21 from tensorflow_text.python import metrics
     22 from tensorflow_text.python.ops import *
     23 

/usr/local/lib/python3.7/dist-packages/tensorflow_text/python/metrics/__init__.py in <module>()
     18 
     19 # pylint: disable=wildcard-import
---> 20 from tensorflow_text.python.metrics.text_similarity_metric_ops import *
     21 
     22 # Public symbols in the "tensorflow_text.metrics" package.

/usr/local/lib/python3.7/dist-packages/tensorflow_text/python/metrics/text_similarity_metric_ops.py in <module>()
     26 from tensorflow.python.framework import load_library
     27 from tensorflow.python.platform import resource_loader
---> 28 gen_text_similarity_metric_ops = load_library.load_op_library(resource_loader.get_path_to_datafile('_text_similarity_metric_ops.so'))
     29 
     30 

/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/load_library.py in load_op_library(library_filename)
     56     RuntimeError: when unable to load the library or get the python wrappers.
     57   """
---> 58   lib_handle = py_tf.TF_LoadLibrary(library_filename)
     59   try:
     60     wrappers = _pywrap_python_op_gen.GetPythonWrappers(

NotFoundError: /usr/local/lib/python3.7/dist-packages/tensorflow_text/python/metrics/_text_similarity_metric_ops.so: undefined symbol: _ZN10tensorflow15OpKernelContext15allocate_outputEN4absl14lts_2020_09_2311string_viewERKNS_11TensorShapeEPPNS_6TensorE

I have noticed a similar issue in #319 and #826 (Installing the specified versions or the recent versions of gcs_config didn't solve this issue. If there is any specific version that worked for you, would like to know.)

[agemagician]

Could you please, double-check again, because for me it is running normally without any problem

Block:

print("Installing dependencies...")
%tensorflow_version 2.x
!pip install -q t5

import functools
import os
import time
import warnings
warnings.filterwarnings("ignore", category=DeprecationWarning)

import tensorflow.compat.v1 as tf
import tensorflow_datasets as tfds

import t5
import t5.models
import seqio

BASE_DIR = "gs://google.com" #@param { type: "string" }
if not BASE_DIR or BASE_DIR == "gs://":
  raise ValueError("You must enter a BASE_DIR.")
DATA_DIR = os.path.join(BASE_DIR, "data")
MODELS_DIR = os.path.join(BASE_DIR, "models")
ON_CLOUD = True


if ON_CLOUD:
  print("Setting up GCS access...")
  import tensorflow_gcs_config
  from google.colab import auth
  # Set credentials for GCS reading/writing from Colab and TPU.
  TPU_TOPOLOGY = "v2-8"
  try:
    tpu = tf.distribute.cluster_resolver.TPUClusterResolver()  # TPU detection
    TPU_ADDRESS = tpu.get_master()
    print('Running on TPU:', TPU_ADDRESS)
  except ValueError:
    raise BaseException('ERROR: Not connected to a TPU runtime; please see the previous cell in this notebook for instructions!')
  auth.authenticate_user()
  tf.enable_eager_execution()
  tf.config.experimental_connect_to_host(TPU_ADDRESS)
  tensorflow_gcs_config.configure_gcs_from_colab_auth()

tf.disable_v2_behavior()

# Improve logging.
from contextlib import contextmanager
import logging as py_logging

if ON_CLOUD:
  tf.get_logger().propagate = False
  py_logging.root.setLevel('INFO')

@contextmanager
def tf_verbosity_level(level):
  og_level = tf.logging.get_verbosity()
  tf.logging.set_verbosity(level)
  yield
  tf.logging.set_verbosity(og_level)

Result:

Installing dependencies...
Setting up GCS access...
Running on TPU: grpc://10.52.181.66:8470
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:101: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/compat/v2_compat.py:101: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term

[agemagician]

I had this problem before when I was using different versions of TensorFlow and TensorFlow text.
I just checked Colab and it seems they upgraded both of them to version 2.6. It should work fine now.

[mmcs-work]

@agemagician Yes since today it's working most likely because of the issue you mentioned. Thanks. (I kept it open to have more insight. Makes sense now. )