Issue with parsing ONNX models from onnx model zoo

[ssatyendras]

Hi,
I am trying to use zigzag to parse the onnx models. While I was able to parse some nets, the following nets are not being parsed (with the respective errors reported as listed below):

Faster R-CNN R-50-FPN,
Mask R-CNN R-50-FPN:

File /imec/scratch/dtpatha/sahoo98/tools/conda/envs/sss_default/lib/python3.10/site-packages/zigzag/classes/io/onnx/conv.py:79, in ConvParser.generate_layer_node_for_conv..get_input_output_weight_data_type(node, model)
     76 # w_name = node.input[3]
     77 # Get the weight data type through the graph initializers
     78 initializer_names = [i.name for i in model.graph.initializer]
---> 79 w_data_type = model.graph.initializer[initializer_names.index(w_name)].data_type
     81 return ia_elem_type, oa_elem_type, w_data_type

ValueError: '2' is not in list

FCN ResNet-50,
SSD-MobilenetV1-12,
YOLOv3-12:

File /imec/scratch/dtpatha/sahoo98/tools/conda/envs/sss_default/lib/python3.10/site-packages/zigzag/classes/workload/layer_node.py:341, in LayerNode.extract_layer_info(self)
    339 operand_data_reuse: Dict[str, float] = {}
    340 for operand, size_in_elem in operand_size_elem.items():
--> 341     operand_data_reuse[operand] = total_MAC_count/size_in_elem
    342 self.operand_data_reuse = operand_data_reuse

ZeroDivisionError: division by zero

BERT-Squad:

File /imec/scratch/dtpatha/sahoo98/tools/conda/envs/sss_default/lib/python3.10/site-packages/zigzag/classes/io/onnx/matmul.py:64, in MatMulParser.generate_layer_node_for_matmul(self)
     60     return d
     62 ia_dimension_shape, oa_dimension_shape = get_node_input_output_dimension_shapes(self.node, self.onnx_model)
---> 64 assert len(ia_dimension_shape) == len(oa_dimension_shape) == 2  # First element is batch size, second is input/output channel
     65 assert ia_dimension_shape[0] == oa_dimension_shape[0]  # Batch size should be the same for input and output
     66 # If the batch size is 0, we discard it by setting it to 1 internally inside ZigZag

AssertionError: 

[asyms]

Hi.

Did you run shape inference on the models? You can find more info here: https://zigzag-project.github.io/zigzag/workload.html#inferring-an-onnx-model-s-shapes

I suspect this wil solve some, but not all issues. Please let me know.

Kind regards,
Arne

[ssatyendras]

Hi,
Thanks for the response. I did run shape inference on the models before the parsing.
The parser worked fine on the following models from the model zoo of onnx:

• "AlexNet"
• "CaffeNet"
• "DenseNet-121-12"
• "EfficientNet-Lite4"
• "GoogleNet"
• "Inception-1"
• "MNIST-12"
• "MobileNet v2-1.0"
• "ResNet50"
• "ShuffleNet-v2"
• "SqueezeNet 1.0"
• "VGG 16"
• "ZFNet-512"
• "ResNet101_DUC_HDC-12"
• "SSD"

The errors were for the following models that were not parsed as detailed in my previous comment:

• "BERT-Squad"
• "Faster R-CNN R-50-FPN"
• "FCN ResNet-50"
• "Mask R-CNN R-50-FPN"
• "SSD-MobilenetV1-12"
• "YOLOv3-12"

Thanks & Regards,
Siva

[fpedd]

@asyms FYI: BERT not passing these checks is something I saw on MobileBERT too. I removed the checks on our internal version

[asyms]

I didn't find the time to check all the different models you list, but these checks will fail because there is some information lacking in the ONNX graph. Either there is shape information missing or some of the dimensions of the shapes are dynamic.

If you have fixed one the models you listed above, please feel free to submit a pull request with the model here: https://github.com/KULeuven-MICAS/zigzag/tree/master/zigzag/inputs/examples/workload

Kind regards,
Arne

[KarelPeeters]

It's possible at least the first case was fixed by #10, there were a lot of edge cases around getting tensor information by name.

[asyms]

Closing this due to inactivity. Feel free to re-open if there's any specific networks that still aren't working.