google-ai-edge/model-explorer

AttributeError: 'int' object has no attribute 'dtype'

mat10599 opened this issue · 8 comments

mat10599 commented

Hello,
I am trying to run your graph visualization on the vision transformer from the CoCa model.
Here is the code I run and the full traceback I get.

Code

model,_, processor = open_clip.create_model_and_transforms(
            model_name="coca_ViT-L-14",
            pretrained="mscoco_finetuned_laion2B-s13B-b90k",
            device=device)
inputs = (torch.rand([1, 3, 224, 224]),)
ep = torch.export.export(model.visual, inputs) 
model_explorer.visualize_pytorch('visionnet', exported_program=ep,
                                port=8081)

Traceback

Converting pytorch model to model explorer graphs...
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[24], line 1
----> 1 model_explorer.visualize_pytorch('visionnet', exported_program=ep,
      2                                 port=8081)

File /opt/conda/lib/python3.10/site-packages/model_explorer/apis.py:82, in visualize_pytorch(name, exported_program, host, port, extensions, colab_height, settings)
     80 # Construct config.
     81 cur_config = config()
---> 82 cur_config.add_model_from_pytorch(
     83     name, exported_program=exported_program, settings=settings
     84 )
     86 # Start server.
     87 server.start(
     88     host=host, port=port, config=cur_config, colab_height=colab_height, extensions=extensions
     89 )

File /opt/conda/lib/python3.10/site-packages/model_explorer/config.py:85, in ModelExplorerConfig.add_model_from_pytorch(self, name, exported_program, settings)
     83 print('Converting pytorch model to model explorer graphs...')
     84 adapter = PytorchExportedProgramAdapterImpl(exported_program, settings)
---> 85 graphs = adapter.convert()
     86 graphs_index = len(self.graphs_list)
     87 self.graphs_list.append(graphs)

File /opt/conda/lib/python3.10/site-packages/model_explorer/pytorch_exported_program_adater_impl.py:237, in PytorchExportedProgramAdapterImpl.convert(self)
    236 def convert(self) -> ModelExplorerGraphs:
--> 237   return {'graphs': [self.create_graph()]}

File /opt/conda/lib/python3.10/site-packages/model_explorer/pytorch_exported_program_adater_impl.py:233, in PytorchExportedProgramAdapterImpl.create_graph(self)
    231 graph = Graph(id='graph', nodes=[])
    232 for node in self.gm.graph.nodes:
--> 233   graph.nodes.append(self.create_node(node))
    234 return graph

File /opt/conda/lib/python3.10/site-packages/model_explorer/pytorch_exported_program_adater_impl.py:227, in PytorchExportedProgramAdapterImpl.create_node(self, fx_node)
    225 self.add_incoming_edges(fx_node, node)
    226 self.add_node_attrs(fx_node, node)
--> 227 self.add_outputs_metadata(fx_node, node)
    228 return node

File /opt/conda/lib/python3.10/site-packages/model_explorer/pytorch_exported_program_adater_impl.py:200, in PytorchExportedProgramAdapterImpl.add_outputs_metadata(self, fx_node, node)
    198 if val is None:
    199   continue
--> 200 dtype = str(val.dtype)
    201 shape = json.dumps(val.shape)
    202 metadata.attrs.append(KeyValue(key='tensor_shape', value=dtype + shape))

AttributeError: 'int' object has no attribute 'dtype'

Any help appreciated,

Thanks,
Mathieu

pkgoogle commented

Hi @mat10599,

I was not able to replicate your issue ... so this likely means this is an environmental issue:
image

Here's my environment:
Python 3.11
MacOS M1

$ pip show ai-edge-model-explorer
Name: ai-edge-model-explorer
Version: 0.1.7
Summary: A modern model graph visualizer and debugger
Home-page: 
Author: 
Author-email: Google LLC <opensource@google.com>
License: 
Location: xxxxxxxx/envs/model-explorer/lib/python3.11/site-packages
Requires: ai-edge-model-explorer-adapter, flask, ipython, numpy, packaging, portpicker, requests, termcolor, torch, typing-extensions
Required-by: 
$ pip show torch
Name: torch
Version: 2.3.1
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: xxxxxxxxx/envs/model-explorer/lib/python3.11/site-packages
Requires: filelock, fsspec, jinja2, networkx, sympy, typing-extensions
Required-by: ai-edge-model-explorer, open_clip_torch, timm, torchvision
$ pip show open_clip_torch
Name: open_clip_torch
Version: 2.26.1
Summary: Open reproduction of consastive language-image pretraining (CLIP) and related.
Home-page: https://github.com/mlfoundations/open_clip
Author: Gabriel Ilharco, Mitchell Wortsman, Romain Beaumont
Author-email: Ross Wightman <ross@huggingface.co>
License: MIT
Location: xxxxxxxx/envs/model-explorer/lib/python3.11/site-packages
Requires: ftfy, huggingface-hub, regex, timm, torch, torchvision, tqdm
Required-by:

Here's the script I ran:

import model_explorer
import open_clip
import torch


device = "cpu"

model,_, processor = open_clip.create_model_and_transforms(
            model_name="coca_ViT-L-14",
            pretrained="mscoco_finetuned_laion2B-s13B-b90k",
            device=device)
inputs = (torch.rand([1, 3, 224, 224]),)
ep = torch.export.export(model.visual, inputs) 
model_explorer.visualize_pytorch('visionnet', exported_program=ep,
                                port=8081)

Here's how to recreate my conda environment:

conda create -n model-explorer python=3.11
conda activate model-explorer
pip install ai-edge-model-explorer open_clip_torch

While conda is not necessary, I do recommend you have a way of separating your environments/starting fresh ones. Is your environment similar? Are there any discrepancies you can see? Are you using a GPU/Cuda? Thanks.

mat10599 commented

Hello,

Thank you for your quick reply,

The error happens both when I load the model on cuda and cpu.

Here is what i get when i run pip show ai-edge-model-explorer

Name: ai-edge-model-explorer
Version: 0.1.7
Summary: A modern model graph visualizer and debugger
Home-page: 
Author: 
Author-email: Google LLC <opensource@google.com>
License: 
Location: /opt/conda/lib/python3.10/site-packages
Requires: ai-edge-model-explorer-adapter, flask, ipython, numpy, packaging, portpicker, requests, termcolor, torch, typing-extensions

pip show torch

Name: torch
Version: 2.2.0+cu121
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: packages@pytorch.org
License: BSD-3
Location: /opt/conda/lib/python3.10/site-packages
Requires: filelock, fsspec, jinja2, networkx, nvidia-cublas-cu12, nvidia-cuda-cupti-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-runtime-cu12, nvidia-cudnn-cu12, nvidia-cufft-cu12, nvidia-curand-cu12, nvidia-cusolver-cu12, nvidia-cusparse-cu12, nvidia-nccl-cu12, nvidia-nvtx-cu12, sympy, triton, typing-extensions
Required-by: ai-edge-model-explorer, clip-benchmark, open-clip-torch, pytorch-ranger, timm, torch-optimizer, torchvision

pip show open_clip_torch

Name: open-clip-torch
Version: 2.24.0
Summary: OpenCLIP
Home-page: https://github.com/mlfoundations/open_clip
Author: 
Author-email: 
License: UNKNOWN
Location: /opt/conda/lib/python3.10/site-packages
Requires: ftfy, huggingface-hub, protobuf, regex, sentencepiece, timm, torch, torchvision, tqdm
Required-by: clip-benchmark

I am using vertex ai workbench VMs and torch 2.2 (torch 2.3 is not available yet) is installed by default during VM creation.
The problem is most probably due to the torch version. It's hard for me to check in local with torch 2.3 as I am on windows and on the office computer...
I am on python 3.10 which might be another cause ...
Mathieu

pkgoogle commented

Hi @mat10599, do you have access to WSL or can you install/use WSL on your windows machine? Just to get more data... I will see if I can setup a vertex AI workbench to try.

pkgoogle commented

Ok I have setup a new vertex AI workbench... in a fairly standard configuration (I kept the jupyterlab3 image... did you?), It already has conda installed, so I just went to the terminal and did this:

pip install ai-edge-model-explorer open_clip_torch

This effectively installed torch 2.3.1 in my case.. how did you create your vertex AI workbench instance?

mat10599 commented

Hi @mat10599, do you have access to WSL or can you install/use WSL on your windows machine? Just to get more data... I will see if I can setup a vertex AI workbench to try.

I can free myself some time to try this over the weekend but visualisation is more a nice to have for me than a key component.

Ok I have setup a new vertex AI workbench... in a fairly standard configuration (I kept the jupyterlab3 image... did you?), It already has conda installed, so I just went to the terminal and did this:

pip install ai-edge-model-explorer open_clip_torch

This effectively installed torch 2.3.1 in my case.. how did you create your vertex AI workbench instance?

Regarding you second comment, when I create an environment, tensorflow/pytorch is pre-installed during vm creation and setup wrt the gpus. I have been advised to apply this method to avoid driver gpu issues.

(see screenshot below for the pre-installed environment that are available, torch 2.2 is the latest)
image

pkgoogle commented

Ok, I'm guessing my account type is different so I have different options... do you know if you have a clean environment you can select? I'm thinking you can just do the installation steps on a cleaner environment so your software will use the right versions. Alternatively, have you just tried manually updating PyTorch?

With GPU & conda:

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

or
With GPU & pip:

pip3 install torch torchvision torchaudio
github-actions commented

Marking this issue as stale since it has been open for 7 days with no activity. This issue will be closed if no further activity occurs.

github-actions commented

This issue was closed because it has been inactive for 14 days. Please post a new issue if you need further assistance. Thanks!