tensorflow/tfx

tfjs_next_page_prediction example failed

Bbean opened this issue · 1 comments

Bbean commented

System information

  • Have I specified the code to reproduce the issue: Yes
  • Environment in which the code is executed : docker tfx latest(1.14.0.dev20230619)
  • TensorFlow version:2.11.0
  • TFX Version:1.12.0
  • Python version:3.7
  • Python dependencies (from pip freeze output):
    absl-py==1.4.0
    aiohttp @ file:///home/conda/feedstock_root/build_artifacts/aiohttp_1663850771047/work
    aiohttp-cors==0.7.0
    aiorwlock==1.3.0
    aiosignal @ file:///home/conda/feedstock_root/build_artifacts/aiosignal_1667935791922/work
    ansiwrap==0.8.4
    anyio @ file:///home/conda/feedstock_root/build_artifacts/anyio_1685486400275/work/dist
    apache-beam==2.46.0
    argon2-cffi @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi_1640817743617/work
    argon2-cffi-bindings @ file:///home/conda/feedstock_root/build_artifacts/argon2-cffi-bindings_1649500320262/work
    astunparse==1.6.3
    async-timeout @ file:///home/conda/feedstock_root/build_artifacts/async-timeout_1640026696943/work
    asynctest==0.13.0
    attrs==21.4.0
    Babel==2.12.1
    backcall @ file:///home/conda/feedstock_root/build_artifacts/backcall_1592338393461/work
    backoff==2.2.1
    backports.functools-lru-cache @ file:///home/conda/feedstock_root/build_artifacts/backports.functools_lru_cache_1618230623929/work
    beatrix-jupyterlab @ file:///home/kbuilder/miniconda3/conda-bld/dlenv-tf-2-11-gpu_1686189831411/work/packages/beatrix_jupyterlab-2023.62.145605.tar.gz#sha256=dc0618129624f08938e5de9508880ef81dba8b272d097fd0b0c61f77af4da3f5
    beautifulsoup4 @ file:///home/conda/feedstock_root/build_artifacts/beautifulsoup4_1680888073205/work
    bleach @ file:///home/conda/feedstock_root/build_artifacts/bleach_1674535352125/work
    blessed==1.20.0
    brotlipy==0.7.0
    cached-property==1.5.2
    cachetools==4.2.4
    certifi==2023.5.7
    cffi @ file:///home/conda/feedstock_root/build_artifacts/cffi_1666183775483/work
    charset-normalizer @ file:///home/conda/feedstock_root/build_artifacts/charset-normalizer_1661170624537/work
    chex==0.1.5
    click==7.1.2
    cloud-tpu-client==0.10
    cloud-tpu-profiler==2.4.0
    cloudpickle==2.2.1
    colorama @ file:///tmp/build/80754af9/colorama_1607707115595/work
    colorful==0.5.5
    conda==22.9.0
    conda-content-trust @ file:///tmp/build/80754af9/conda-content-trust_1617045594566/work
    conda-package-handling @ file:///home/conda/feedstock_root/build_artifacts/conda-package-handling_1669907009957/work
    conda_package_streaming @ file:///home/conda/feedstock_root/build_artifacts/conda-package-streaming_1685101166527/work
    crcmod==1.7
    cryptography @ file:///home/conda/feedstock_root/build_artifacts/cryptography_1666563371538/work
    cycler==0.11.0
    Cython==0.29.35
    dacite==1.8.1
    db-dtypes==1.1.1
    debugpy @ file:///home/conda/feedstock_root/build_artifacts/debugpy_1660619049122/work
    decorator @ file:///home/conda/feedstock_root/build_artifacts/decorator_1641555617451/work
    defusedxml @ file:///home/conda/feedstock_root/build_artifacts/defusedxml_1615232257335/work
    Deprecated==1.2.14
    dill==0.3.1.1
    distlib==0.3.6
    dm-tree==0.1.8
    docker==4.4.4
    docopt==0.6.2
    entrypoints @ file:///home/conda/feedstock_root/build_artifacts/entrypoints_1643888246732/work
    etils==0.9.0
    exceptiongroup @ file:///home/conda/feedstock_root/build_artifacts/exceptiongroup_1678703645500/work
    explainable-ai-sdk==1.3.3
    fastapi==0.96.0
    fastavro==1.7.4
    fasteners==0.18
    fastjsonschema @ file:///home/conda/feedstock_root/build_artifacts/python-fastjsonschema_1684761244589/work/dist
    filelock==3.12.0
    flatbuffers==2.0.7
    flax==0.6.4
    flit_core @ file:///home/conda/feedstock_root/build_artifacts/flit-core_1684084314667/work/source/flit_core
    fonttools==4.38.0
    frozenlist @ file:///home/conda/feedstock_root/build_artifacts/frozenlist_1659556953511/work
    fsspec==2023.1.0
    gast==0.4.0
    gcsfs==2023.1.0
    gitdb==4.0.10
    GitPython==3.1.31
    google-api-core==1.32.0
    google-api-python-client==1.8.0
    google-apitools==0.5.31
    google-auth==1.35.0
    google-auth-httplib2==0.1.0
    google-auth-oauthlib==0.4.6
    google-cloud-aiplatform==1.17.1
    google-cloud-artifact-registry==1.8.1
    google-cloud-bigquery==2.34.4
    google-cloud-bigquery-storage==2.16.2
    google-cloud-bigtable==1.7.3
    google-cloud-core @ file:///home/conda/feedstock_root/build_artifacts/google-cloud-core_1658759191372/work
    google-cloud-datastore @ file:///home/conda/feedstock_root/build_artifacts/google-cloud-datastore_1661336072891/work
    google-cloud-dlp==3.9.2
    google-cloud-language==1.3.2
    google-cloud-monitoring==2.15.0
    google-cloud-pubsub==2.13.11
    google-cloud-pubsublite==1.6.0
    google-cloud-recommendations-ai==0.7.1
    google-cloud-resource-manager==1.6.3
    google-cloud-spanner==3.26.0
    google-cloud-storage==2.9.0
    google-cloud-videointelligence==1.16.3
    google-cloud-vision==3.1.4
    google-crc32c==1.5.0
    google-pasta==0.2.0
    google-resumable-media==2.5.0
    googleapis-common-protos @ file:///home/conda/feedstock_root/build_artifacts/googleapis-common-protos-feedstock_1657690297642/work
    gpustat==1.0.0
    greenlet==2.0.2
    grpc-google-iam-v1==0.12.6
    grpcio @ file:///home/conda/feedstock_root/build_artifacts/grpc-split_1662408129152/work
    grpcio-status @ file:///home/conda/feedstock_root/build_artifacts/grpcio-status_1662108958711/work
    gviz-api==1.10.0
    Gymnasium==0.26.3
    gymnasium-notices==0.0.1
    h11==0.14.0
    h5py==3.8.0
    hdfs==2.7.0
    htmlmin==0.1.12
    httplib2==0.21.0
    idna @ file:///home/conda/feedstock_root/build_artifacts/idna_1663625384323/work
    ImageHash==4.3.1
    imageio==2.31.0
    importlib-metadata @ file:///home/conda/feedstock_root/build_artifacts/importlib-metadata_1653252814274/work
    importlib-resources @ file:///home/conda/feedstock_root/build_artifacts/importlib_resources_1676919000169/work
    iniconfig==2.0.0
    ipykernel @ file:///home/conda/feedstock_root/build_artifacts/ipykernel_1666723258080/work
    ipython @ file:///home/conda/feedstock_root/build_artifacts/ipython_1651240553635/work
    ipython-genutils==0.2.0
    ipython-sql==0.5.0
    ipywidgets==7.7.5
    jaraco.classes==3.2.3
    jax==0.3.25
    jaxlib==0.3.25
    jedi @ file:///home/conda/feedstock_root/build_artifacts/jedi_1669134318875/work
    jeepney==0.8.0
    Jinja2 @ file:///home/conda/feedstock_root/build_artifacts/jinja2_1654302431367/work
    joblib==1.2.0
    json5==0.9.14
    jsonschema @ file:///home/conda/feedstock_root/build_artifacts/jsonschema-meta_1669810440410/work
    jupyter-http-over-ws==0.0.8
    jupyter-server @ file:///home/conda/feedstock_root/build_artifacts/jupyter_server_1671591499479/work
    jupyter-server-mathjax==0.2.6
    jupyter-server-proxy==3.2.2
    jupyter_client @ file:///home/conda/feedstock_root/build_artifacts/jupyter_client_1673615989977/work
    jupyter_core @ file:///home/conda/feedstock_root/build_artifacts/jupyter_core_1658332345782/work
    jupyterlab==3.4.8
    jupyterlab-git==0.41.0
    jupyterlab-pygments @ file:///home/conda/feedstock_root/build_artifacts/jupyterlab_pygments_1649936611996/work
    jupyterlab-widgets==1.1.4
    jupyterlab_server==2.22.1
    jupytext==1.14.6
    keras==2.11.0
    keras-tuner==1.3.5
    keyring==23.13.1
    keyrings.google-artifactregistry-auth==1.1.2
    kiwisolver==1.4.4
    kt-legacy==1.0.5
    kubernetes==12.0.1
    libclang==16.0.0
    llvmlite==0.39.1
    lz4==4.3.2
    Markdown==3.4.3
    markdown-it-py==2.2.0
    MarkupSafe==2.0.1
    matplotlib==3.5.3
    matplotlib-inline @ file:///home/conda/feedstock_root/build_artifacts/matplotlib-inline_1660814786464/work
    mdit-py-plugins==0.3.5
    mdurl==0.1.2
    mistune @ file:///home/conda/feedstock_root/build_artifacts/mistune_1675771498296/work
    ml-metadata==1.12.0
    ml-pipelines-sdk==1.12.0
    more-itertools==9.1.0
    msgpack==1.0.5
    multidict @ file:///home/conda/feedstock_root/build_artifacts/multidict_1648882415996/work
    multimethod==1.9.1
    nb-conda @ file:///home/conda/feedstock_root/build_artifacts/nb_conda_1654442778977/work
    nb-conda-kernels @ file:///home/conda/feedstock_root/build_artifacts/nb_conda_kernels_1636999991206/work
    nbclassic @ file:///home/conda/feedstock_root/build_artifacts/nbclassic_1683202085119/work
    nbclient @ file:///home/conda/feedstock_root/build_artifacts/nbclient_1665125402713/work
    nbconvert @ file:///home/conda/feedstock_root/build_artifacts/nbconvert-meta_1683636231832/work
    nbdime==3.2.0
    nbformat @ file:///home/conda/feedstock_root/build_artifacts/nbformat_1679336765223/work
    nest-asyncio @ file:///home/conda/feedstock_root/build_artifacts/nest-asyncio_1664684991461/work
    networkx==2.6.3
    notebook @ file:///home/conda/feedstock_root/build_artifacts/notebook_1680870634737/work
    notebook-executor @ file:///home/kbuilder/miniconda3/conda-bld/dlenv-tf-2-11-gpu_1686189831411/work/packages/notebook_executor
    notebook_shim @ file:///home/conda/feedstock_root/build_artifacts/notebook-shim_1682360583588/work
    numba==0.56.4
    numpy==1.21.6
    nvidia-ml-py==11.495.46
    oauth2client==4.1.3
    oauthlib==3.2.2
    objsize==0.6.1
    onnx==1.12.0
    opencensus==0.11.2
    opencensus-context==0.1.3
    opentelemetry-api==1.18.0
    opentelemetry-exporter-otlp==1.18.0
    opentelemetry-exporter-otlp-proto-common==1.18.0
    opentelemetry-exporter-otlp-proto-grpc==1.18.0
    opentelemetry-exporter-otlp-proto-http==1.18.0
    opentelemetry-proto==1.18.0
    opentelemetry-sdk==1.18.0
    opentelemetry-semantic-conventions==0.39b0
    opt-einsum==3.3.0
    optax==0.1.4
    orbax==0.1.0
    orjson==3.9.0
    overrides==6.5.0
    packaging==20.9
    pandas==1.3.5
    pandas-profiling==3.6.6
    pandocfilters @ file:///home/conda/feedstock_root/build_artifacts/pandocfilters_1631603243851/work
    papermill==2.4.0
    parso @ file:///home/conda/feedstock_root/build_artifacts/parso_1638334955874/work
    patsy==0.5.3
    pexpect @ file:///home/conda/feedstock_root/build_artifacts/pexpect_1667297516076/work
    phik==0.12.3
    pickleshare @ file:///home/conda/feedstock_root/build_artifacts/pickleshare_1602536217715/work
    Pillow==9.5.0
    pkgutil_resolve_name @ file:///home/conda/feedstock_root/build_artifacts/pkgutil-resolve-name_1633981968097/work
    platformdirs==3.5.1
    plotly==5.14.1
    pluggy==1.0.0
    portpicker==1.5.2
    prettytable==3.7.0
    prometheus-client @ file:///home/conda/feedstock_root/build_artifacts/prometheus_client_1684970902412/work
    promise==2.3
    prompt-toolkit @ file:///home/conda/feedstock_root/build_artifacts/prompt-toolkit_1677600924538/work
    proto-plus==1.22.2
    protobuf==3.19.6
    psutil @ file:///home/conda/feedstock_root/build_artifacts/psutil_1666155398032/work
    ptyprocess @ file:///home/conda/feedstock_root/build_artifacts/ptyprocess_1609419310487/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
    py-spy==0.3.14
    pyarrow==6.0.1
    pyasn1==0.4.8
    pyasn1-modules==0.2.7
    pycosat @ file:///home/conda/feedstock_root/build_artifacts/pycosat_1666656960991/work
    pycparser @ file:///tmp/build/80754af9/pycparser_1636541352034/work
    pydantic==1.10.9
    pydot==1.4.2
    pyfarmhash==0.3.2
    Pygments @ file:///home/conda/feedstock_root/build_artifacts/pygments_1681904169130/work
    PyJWT==2.7.0
    pymongo==3.13.0
    pyOpenSSL @ file:///home/conda/feedstock_root/build_artifacts/pyopenssl_1685514481738/work
    pyparsing==3.0.9
    pyrsistent @ file:///home/conda/feedstock_root/build_artifacts/pyrsistent_1649013358450/work
    PySocks @ file:///tmp/build/80754af9/pysocks_1594394576006/work
    pytest==7.4.0
    python-dateutil @ file:///home/conda/feedstock_root/build_artifacts/python-dateutil_1626286286081/work
    pytz==2023.3
    pyu2f @ file:///home/conda/feedstock_root/build_artifacts/pyu2f_1604248910016/work
    PyWavelets==1.3.0
    PyYAML==5.4.1
    pyzmq @ file:///home/conda/feedstock_root/build_artifacts/pyzmq_1663830492333/work
    ray==2.4.0
    ray-cpp==2.4.0
    regex==2023.6.3
    requests @ file:///home/conda/feedstock_root/build_artifacts/requests_1684774241324/work
    requests-oauthlib==1.3.1
    retrying==1.3.3
    rich==13.4.1
    rsa @ file:///home/conda/feedstock_root/build_artifacts/rsa_1658328885051/work
    ruamel-yaml-conda @ file:///tmp/build/80754af9/ruamel_yaml_1616016701961/work
    scikit-image==0.19.3
    scikit-learn==1.0.2
    scipy==1.7.3
    seaborn==0.12.2
    SecretStorage==3.3.3
    Send2Trash @ file:///home/conda/feedstock_root/build_artifacts/send2trash_1682601222253/work
    Shapely==1.8.5.post1
    simpervisor==0.4
    six @ file:///tmp/build/80754af9/six_1644875935023/work
    smart-open==6.3.0
    smmap==5.0.0
    sniffio @ file:///home/conda/feedstock_root/build_artifacts/sniffio_1662051266223/work
    soupsieve @ file:///home/conda/feedstock_root/build_artifacts/soupsieve_1658207591808/work
    SQLAlchemy==2.0.15
    sqlparse==0.4.4
    starlette==0.27.0
    statsmodels==0.13.5
    tabulate==0.9.0
    tangled-up-in-unicode==0.2.0
    tenacity==8.2.2
    tensorboard==2.11.2
    tensorboard-data-server==0.6.1
    tensorboard-plugin-profile==2.13.0
    tensorboard-plugin-wit==1.8.1
    tensorboardX==2.6
    tensorflow==2.11.0
    tensorflow-cloud==0.1.16
    tensorflow-data-validation==1.12.0
    tensorflow-datasets==4.8.2
    tensorflow-decision-forests==1.2.0
    tensorflow-estimator==2.11.0
    tensorflow-hub==0.12.0
    tensorflow-io==0.29.0
    tensorflow-io-gcs-filesystem==0.29.0
    tensorflow-metadata==1.12.0
    tensorflow-model-analysis==0.43.0
    tensorflow-probability==0.19.0
    tensorflow-serving-api==2.11.0
    tensorflow-transform==1.12.0
    tensorflowjs==4.4.0
    tensorstore==0.1.28
    termcolor==2.3.0
    terminado @ file:///home/conda/feedstock_root/build_artifacts/terminado_1670253674810/work
    textwrap3==0.9.2
    tf2onnx==1.14.0
    tfx==1.12.0
    tfx-bsl==1.12.0
    threadpoolctl==3.1.0
    tifffile==2021.11.2
    tinycss2 @ file:///home/conda/feedstock_root/build_artifacts/tinycss2_1666100256010/work
    toml==0.10.2
    tomli==2.0.1
    toolz @ file:///home/conda/feedstock_root/build_artifacts/toolz_1657485559105/work
    tornado @ file:///home/conda/feedstock_root/build_artifacts/tornado_1656937818679/work
    tqdm @ file:///opt/conda/conda-bld/tqdm_1647339053476/work
    traitlets @ file:///home/conda/feedstock_root/build_artifacts/traitlets_1675110562325/work
    typeguard==2.13.3
    typer==0.9.0
    typing_extensions @ file:///home/conda/feedstock_root/build_artifacts/typing_extensions_1685704949284/work
    uritemplate==3.0.1
    urllib3 @ file:///home/conda/feedstock_root/build_artifacts/urllib3_1678635778344/work
    uvicorn==0.22.0
    virtualenv==20.21.0
    visions==0.7.5
    wcwidth @ file:///home/conda/feedstock_root/build_artifacts/wcwidth_1673864653149/work
    webencodings==0.5.1
    websocket-client @ file:///home/conda/feedstock_root/build_artifacts/websocket-client_1684708148666/work
    Werkzeug==2.1.2
    widgetsnbextension==3.6.4
    witwidget==1.8.1
    wordcloud==1.9.2
    wrapt==1.15.0
    wurlitzer==3.0.3
    yarl @ file:///home/conda/feedstock_root/build_artifacts/yarl_1648966511831/work
    ydata-profiling==4.2.0
    zipp @ file:///home/conda/feedstock_root/build_artifacts/zipp_1677313463193/work
    zstandard @ file:///home/conda/feedstock_root/build_artifacts/zstandard_1655887611100/work

Describe the current behavior

python tfjs_next_page_prediction_e2e_test.py
get error:
FAIL: testTFJSPagePredictionPipeline (main.TFJSNextPagePredictionPipelineEndToEndTest)
TFJSNextPagePredictionPipelineEndToEndTest.testTFJSPagePredictionPipeline

Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 813, in _convert_tf_saved_model
resource_ids_maps = _get_resource_ids_maps(model, concrete_func, resource_initializer_concrete_func)
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 680, in _get_resource_ids_maps
captured_input_index = resource_id_to_captured_input_index[_id]
KeyError: 3207

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/Documents/tfx/tfx/examples/tfjs_next_page_prediction/tfjs_next_page_prediction_e2e_test.py", line 94, in testTFJSPagePredictionPipeline
LocalDagRunner().run(pipeline)
File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/portable/tfx_runner.py", line 124, in run
return self.run_with_ir(pipeline_pb, run_options=run_options_pb, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/local/local_dag_runner.py", line 109, in run_with_ir
component_launcher.launch()
File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/portable/launcher.py", line 573, in launch
executor_output = self._run_executor(execution_info)
File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/portable/launcher.py", line 448, in _run_executor
executor_output = self._executor_operator.run_executor(execution_info)
File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/portable/python_executor_operator.py", line 135, in run_executor
return run_with_executor(execution_info, executor)
File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/portable/python_executor_operator.py", line 59, in run_with_executor
execution_info.exec_properties)
File "/opt/conda/lib/python3.7/site-packages/tfx/components/trainer/executor.py", line 178, in Do
run_fn(fn_args)
File "/home/Documents/tfx/tfx/examples/tfjs_next_page_prediction/tfjs_next_page_prediction_util.py", line 198, in run_fn
rewriter.ModelType.TFJS_MODEL)
File "/opt/conda/lib/python3.7/site-packages/tfx/components/trainer/rewriting/converters.py", line 131, in rewrite_saved_model
dst_model_type)
File "/opt/conda/lib/python3.7/site-packages/tfx/components/trainer/rewriting/converters.py", line 47, in _invoke_rewriter
rewriter_inst.perform_rewrite(original_model, rewritten_model)
File "/opt/conda/lib/python3.7/site-packages/tfx/components/trainer/rewriting/rewriter.py", line 106, in perform_rewrite
self._rewrite(original_model, rewritten_model)
File "/opt/conda/lib/python3.7/site-packages/tfx/components/trainer/rewriting/tfjs_rewriter.py", line 91, in _rewrite
_ensure_str(rewritten_model.path))
File "/opt/conda/lib/python3.7/site-packages/tfx/components/trainer/rewriting/tfjs_rewriter.py", line 29, in _convert_tfjs_model
saved_model_path, destination_path
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/converter.py", line 818, in convert
weight_shard_size_bytes, metadata_map)
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/converter.py", line 539, in _dispatch_converter
metadata=metadata_map)
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 988, in convert_tf_saved_model
frozen_graph_dir=frozen_graph_dir)
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 820, in _convert_tf_saved_model
output_node_names)
File "/opt/conda/lib/python3.7/site-packages/tensorflowjs/converters/tf_saved_model_conversion_v2.py", line 434, in _freeze_saved_model_v1
sess, meta_graph_def, output_node_names)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 357, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/graph_util_impl.py", line 322, in convert_variables_to_constants
variable_names_denylist=variable_names_blacklist)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/convert_to_constants.py", line 1288, in convert_variables_to_constants_from_session_graph
variable_names_denylist=variable_names_denylist))
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/convert_to_constants.py", line 943, in init
graph_def = graph_util.extract_sub_graph(graph_def, output_node_names)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 357, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/graph_util_impl.py", line 251, in extract_sub_graph
_assert_nodes_are_present(name_to_node, dest_nodes)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/graph_util_impl.py", line 204, in _assert_nodes_are_present
assert d in name_to_node, "%s is not in graph" % d
AssertionError: Identity is not in graph

Describe the expected behavior

tfjs testcase pass

Standalone code to reproduce the issue

tfx/tfx/examples/tfjs_next_page_prediction/tfjs_next_page_prediction_e2e_test.py

Any reply is welcome, thanks

@Bbean,

This error looks like the pipeline fails at the tf model to tfjs model conversion step. I was able to replicate the issue. Both test case and pipeline fails with "AssertionError: Identity is not in graph" error.

Please find the attached gist. @briron, Could you please have a look why the tfjs_next_page_prediction example fails. Thanks