vx-delegate and Yolov5 models

Question

vx-delegate and Yolov5 models

Opened this issue a year ago · 8 comments

Sorry if this is a duplicate. I had asked on the tim-vx github a few days ago.

We've been trying to get Yolov5 models working with the NXP imx8mp with the NPU. The models we have tested all work just fine with the CPU, and the same tflite models compiled for the Google TPU also work just fine.

The models are object detection. First started with our model exported per the NXP docs. When that didn't work we got the NXP Yolov5 demo model and script from NXP support. But that also fails in the same way.

Finally another NXP user who says he has it working with the same NXP BSP and TFlite versions provided his model. But that model also fails in the same way for us.

Using the recommended .71 NXP BSP and TFlite 2.9.1. With the NPU the models give all zeros for bounding box output, and I suspect the confidence score may be bogus as well.

Has anyone else seen this type of issue when processing Yolov5 models using the NPU? And if so how did you resolve it?

I see a custom op for yolo4 lite, but don't see the build configuration setting up to include that processing. Since that custom op appears to deal with yolo4 post processing I'm wondering if that might also apply for yolo5 models. Is there some Yocto build option that needs to be set to properly process yolov5 models with the vx-delegate and NPU?

Thanks,

Answer 1 · 2023-06-19T14:07:37.000Z

@burtbick ,

I can forward your problem to NXP team. Would you mind share your model?

Thanks

Answer 2 · 2023-06-19T14:38:29.000Z

Thanks Sven, I had a ticket open previously with NXP on this issue, but the support tech handling that ticket was not very helpful. I opened a new ticket with NXP not expecting much, but this time they sent a .diff patch for vx-delegate that is supposed to fix the problem. Not sure why they didn't know about that on the first ticket since a number of users were having the same issues we were seeing. And it's not clear why some are successful while others were not. Likely somehow the success stories already had the patch applied. This failed with all the yolov5 models we tried including the example one provided by NXP so not specific to our model, which is fairly simple. So we are going to test with the NXP supplied patch which was applied to op_map.cc And a brief examination appears to me like it will fix the issue. Thanks again, Burt

…

________________________________ From: Sven ***@***.***> Sent: Monday, June 19, 2023 10:07 AM To: VeriSilicon/tflite-vx-delegate ***@***.***> Cc: burt burtbicksler.com ***@***.***>; Mention ***@***.***> Subject: Re: [VeriSilicon/tflite-vx-delegate] vx-delegate and Yolov5 models (Issue #189) @burtbick<https://github.com/burtbick> , I can forward your problem to NXP team. Would you mind share your model? Thanks — Reply to this email directly, view it on GitHub<#189 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACYT7RMZ25H6IG6XFOFNOI3XMBMLHANCNFSM6AAAAAAZKKR2NA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 3 · 2023-06-20T14:18:41.000Z

Hi @burtbick ,
I am sorry NXP support does not provided an acceptable solution for you.

Can you please share the NXP community link in order match these issues and provide the help?

Best Regards
Robert

Answer 4 · 2023-06-20T15:48:30.000Z

Thanks Robert, I'm waiting to hear if the patch from NXP actually fixes the Yolov5 processing issues. I'm traveling so waiting for our main developer to test the patch. Once I have verified that the supplied patch works I'll post the results and the patch here. But it may be a couple of days. The patch was a response to a support ticket and not via the NXP community forums. Burt

…

________________________________ From: robert-kalmar ***@***.***> Sent: Tuesday, June 20, 2023 10:18 AM To: VeriSilicon/tflite-vx-delegate ***@***.***> Cc: burt burtbicksler.com ***@***.***>; Mention ***@***.***> Subject: Re: [VeriSilicon/tflite-vx-delegate] vx-delegate and Yolov5 models (Issue #189) Hi @burtbick<https://github.com/burtbick> , I am sorry NXP support does not provided an acceptable solution for you. Can you please share the NXP community link in order match these issues and provide the help? Best Regards Robert — Reply to this email directly, view it on GitHub<#189 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ACYT7RLFWWX5JDRQ2L6K3KDXMGWM3ANCNFSM6AAAAAAZKKR2NA>. You are receiving this because you were mentioned.Message ID: ***@***.***>

Answer 5 · 2023-07-02T18:24:39.000Z

Just reporting back that the patch I received from NXP appears to have resolved the problem we were having.
This was a patch to the vx-delegate op_map.cc source file and was a fairly significant patch. Not sure why they didn't have that already included in their Yocto build or notes, but at least it is fixed now.

Answer 6 · 2023-07-04T10:57:35.000Z

Great! Can you share the patch? We are running into a similar issue. Thanks!

Answer 7 · 2023-07-04T12:27:42.000Z

Here's a link to the patch file. The file as received is diff.patch and needs to be applied to the op_map.cc vx-delegate file.

Of course be sure to verify that the patch applies properly, and this is for TFLIte version 2.9.1 which is what we have, and it applied and resolved the issue with no problems.
Here's the patch file..

diff.patch.tar.gz

Answer 8 · 2024-08-27T19:04:20.000Z

Here's a link to the patch file. The file as received is diff.patch and needs to be applied to the op_map.cc vx-delegate file.

Of course be sure to verify that the patch applies properly, and this is for TFLIte version 2.9.1 which is what we have, and it applied and resolved the issue with no problems. Here's the patch file..

diff.patch.tar.gz

I'm running into what appears to be the exact same issue as you, except with YOLOv8. Did you have to rebuild the vx-delegate from source? If so, how? I'm having difficulty with that.