Failed to convert LookupTableFindV2 with IntegerLookup and StringLookup Keras layers
gombru opened this issue · 6 comments
gombru commented
I'm trying to convert a Keras model with IntegerLookup and StringLookup layers.
I've seen #1500 and #1219, but still getting errors.
This is the procedure I'm following:
pip install git+https://github.com/onnx/tensorflow-onnx
pip install onnxruntime-extensions
import tf2onnx
import onnxruntime_extensions
from onnx import helper
input_signature = [{'user_id': tf.TensorSpec([1], tf.int64, name='user_id'),
'user_country': tf.TensorSpec([1], tf.string, name='user_country')}]
onnx_model, _ = tf2onnx.convert.from_keras(keras_model, input_signature, opset=13, extra_opset=[helper.make_opsetid("ai.onnx.contrib", 1)])
The model has as input a dict of features with different types.
And this is the error log:
Cannot infer shape for user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential/embedding/embedding_lookup/Identity_1: user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential/embedding/embedding_lookup/Identity_1:0
Cannot infer shape for user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_1/embedding_1/embedding_lookup/Identity_1: user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_1/embedding_1/embedding_lookup/Identity_1:0
Failed to convert node 'user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_3/string_lookup_1/None_Lookup/LookupTableFindV2' (fct=<bound method LookupTableFind.version_8 of <class 'tf2onnx.custom_opsets.onnx_ml.LookupTableFind'>>)
'OP=LookupTableFindV2\nName=user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_3/string_lookup_1/None_Lookup/LookupTableFindV2\nInputs:\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_40:0=Identity, [], 7\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_30:0=Identity, [1], 8\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_41:0=Const, [], 7\nOutpus:\n\tuser_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_3/string_lookup_1/None_Lookup/LookupTableFindV2:0=None, 7'
Traceback (most recent call last):
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/tfonnx.py", line 292, in tensorflow_onnx_mapping
func(g, node, **kwargs, initialized_tables=initialized_tables, dequantize=dequantize)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/custom_opsets/onnx_ml.py", line 34, in version_8
utils.make_sure(shared_name is not None, "Could not determine table shared name for node %s", node.name)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/utils.py", line 264, in make_sure
raise ValueError("make_sure failure: " + error_msg % args)
ValueError: make_sure failure: Could not determine table shared name for node user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_3/string_lookup_1/None_Lookup/LookupTableFindV2
Failed to convert node 'user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_1/integer_lookup_1/None_Lookup/LookupTableFindV2' (fct=<bound method LookupTableFind.version_8 of <class 'tf2onnx.custom_opsets.onnx_ml.LookupTableFind'>>)
'OP=LookupTableFindV2\nName=user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_1/integer_lookup_1/None_Lookup/LookupTableFindV2\nInputs:\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_37:0=Identity, [], 7\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_29:0=Identity, [1], 7\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_38:0=Const, [], 7\nOutpus:\n\tuser_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_1/integer_lookup_1/None_Lookup/LookupTableFindV2:0=[1], 7'
Traceback (most recent call last):
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/tfonnx.py", line 292, in tensorflow_onnx_mapping
func(g, node, **kwargs, initialized_tables=initialized_tables, dequantize=dequantize)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/custom_opsets/onnx_ml.py", line 34, in version_8
utils.make_sure(shared_name is not None, "Could not determine table shared name for node %s", node.name)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/utils.py", line 264, in make_sure
raise ValueError("make_sure failure: " + error_msg % args)
ValueError: make_sure failure: Could not determine table shared name for node user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_1/integer_lookup_1/None_Lookup/LookupTableFindV2
Failed to convert node 'user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_2/string_lookup/None_Lookup/LookupTableFindV2' (fct=<bound method LookupTableFind.version_8 of <class 'tf2onnx.custom_opsets.onnx_ml.LookupTableFind'>>)
'OP=LookupTableFindV2\nName=user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_2/string_lookup/None_Lookup/LookupTableFindV2\nInputs:\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_31:0=Identity, [], 7\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_28:0=Identity, [1], 8\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_32:0=Const, [], 7\nOutpus:\n\tuser_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_2/string_lookup/None_Lookup/LookupTableFindV2:0=None, 7'
Traceback (most recent call last):
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/tfonnx.py", line 292, in tensorflow_onnx_mapping
func(g, node, **kwargs, initialized_tables=initialized_tables, dequantize=dequantize)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/custom_opsets/onnx_ml.py", line 34, in version_8
utils.make_sure(shared_name is not None, "Could not determine table shared name for node %s", node.name)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/utils.py", line 264, in make_sure
raise ValueError("make_sure failure: " + error_msg % args)
ValueError: make_sure failure: Could not determine table shared name for node user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential_2/string_lookup/None_Lookup/LookupTableFindV2
Failed to convert node 'user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential/integer_lookup/None_Lookup/LookupTableFindV2' (fct=<bound method LookupTableFind.version_8 of <class 'tf2onnx.custom_opsets.onnx_ml.LookupTableFind'>>)
'OP=LookupTableFindV2\nName=user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential/integer_lookup/None_Lookup/LookupTableFindV2\nInputs:\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_34:0=Identity, [], 7\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_26:0=Identity, [1], 7\n\tFunc/user_model/StatefulPartitionedCall/StatefulPartitionedCall/input/_35:0=Const, [], 7\nOutpus:\n\tuser_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential/integer_lookup/None_Lookup/LookupTableFindV2:0=[1], 7'
Traceback (most recent call last):
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/tfonnx.py", line 292, in tensorflow_onnx_mapping
func(g, node, **kwargs, initialized_tables=initialized_tables, dequantize=dequantize)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/custom_opsets/onnx_ml.py", line 34, in version_8
utils.make_sure(shared_name is not None, "Could not determine table shared name for node %s", node.name)
File "/local_disk0/.ephemeral_nfs/envs/pythonEnv-dd74afdc-32d7-4f4d-ae84-dd9e6a74cb29/lib/python3.8/site-packages/tf2onnx/utils.py", line 264, in make_sure
raise ValueError("make_sure failure: " + error_msg % args)
ValueError: make_sure failure: Could not determine table shared name for node user_model/StatefulPartitionedCall/StatefulPartitionedCall/user_encoding/sequential/integer_lookup/None_Lookup/LookupTableFindV2
LongTwig commented
Did anyone find a solution to this, experiencing similar issue using LookupTableFindV2 with IntegerLookup
ponspack9 commented
Same problem here, with both IntegerLookup and StringLookup. They seem to be a problem also with TFLite
andrey-klochkov-liftoff commented
I'm having the same issue and here's a colab that reproduces it:
https://colab.research.google.com/drive/1Duzoue2IJmtCK4x-KoO-qQ0cBj3_S2YH#scrollTo=Jw1bIuR9cTaP
fatcat-z commented
I'm having the same issue and here's a colab that reproduces it: https://colab.research.google.com/drive/1Duzoue2IJmtCK4x-KoO-qQ0cBj3_S2YH#scrollTo=Jw1bIuR9cTaP
Could you please allow jayzhang6666@gmail.com to access it?
fatcat-z commented
Could you please try my suggestions in this issue to see if it resolves this?