SSDMobilenetV1 providing lesser accuracy with tflite parser

[MohitB9]

Hello there!
I would like to start with a little bit of background related with my experimentation.
My aim is to run ssdmobilenetV1 by using arm-nn tflite parser.

1. Conversion process followed for ssdmobilenetV1.pb to ssdmobilenetV1.tflite:
   As suggested, I converted the default Pb file(with 4 output nodes) into an intermediate form. I used 'export_tflite_ssd_graph.py' script available at Tensorflow/models/research/object_detection/.
   Then finally I converted the *.pb file into *.tflite file by using :
   converter = tf.lite.TFLiteConverter.from_frozen_graph(graph_def_file, input_arrays, output_arrays) tflite_model=converter.convert() open("converted_model.tflite","wb").write(tflite_model)
   Please note that this model is in float32 format. I am not using any quantisation method.

2. Validation with native tflite APIs:
   I tested this model with coco2017 validation dataset with tflite python code. I got accuracy of 23.0 mAP.

3. Actual issue with ARMNN-tflite:
   Lesser accuracy of 13.6 mAP is obtained after using the same model with armnn-tflite code.
   Observation is, bounding box coordinates are different among native tflite and armnn-tflite.
   I am aware that similar issue has been filed and solved a couple of weeks ago [https://github.com//issues/214] I am using latest versions 19.08 of both ARMNN and ACL.

Following attachments are available with this issue:

• sample armnn-cpp functions which are being used.
• python script of tflite which I used to compare
• tflite model
• sample outputs of first 10 images from coco2017 dataset using both armnn and tflite in text files. The data in text files can be read in following format:
  class, confidence, bounding box coordinates[left,top,right ,bottom] or [x1,y1,x2,y2].

Kindly communicate in case any additional information or data will be useful from your end.
Many thanks in advance!

armnn-tflite-output.zip
tflite-output.zip
codes.zip

[MikeJKelly]

Hi MohitB9,

thanks for this, we're going to investigate this.

Yours,
Mike

[MohitB9]

Hello there,
A bug has been found in my code. In the attached code, it can be observed that while writing the output coordinates into a text file, one of the 'output0' is missing multiplier within the square brackets.
With this modification, I am able to obtain mAP value equal to that of native tensorflow lite model.
Closing the reported issue.
Thanks @petertorelli and friends at ARM

[MatthewARM]

Thanks @MohitB9 good luck!