Use export.py to generate yolov5s.onnx will get a negative number.
cmdbug opened this issue · 74 comments
❔Question
Use export.py to generate yolov5s.onnx will get a negative number.
This is the code that executes the onnx part
session = onnxruntime.InferenceSession('./weights/yolov5s.onnx')
batch_size = session.get_inputs()[0].shape[0]
img_size_h = session.get_inputs()[0].shape[2]
img_size_w = session.get_inputs()[0].shape[3]
image_src = Image.open(image_path)
resized = letterbox_image(image_src, (img_size_w, img_size_h))
img_in = np.transpose(resized, (2, 0, 1)).astype(np.float32) # HWC -> CHW
img_in = np.expand_dims(img_in, axis=0)
img_in /= 255.0
input_name = session.get_inputs()[0].name
# output, output_exist = session.run(['decoder.output_conv', 'lane_exist.linear2'], {"input.1": image_np})
outputs = session.run(None, {input_name: img_in})There are already negative numbers in the outputs. Then after the result is processed, it will appear that part of it is correct, such as car in the figure. However, the top/bottom of the bicycle is right, the left/right is wrong, the left/right of the dog is right, and the top/bottom is wrong. What might cause this problem? Thanks.
Additional context
torch:1.5.1
torchvision:0.6.1
onnxruntime:1.3.0
Hello @WZTENG, thank you for your interest in our work! Please visit our Custom Training Tutorial to get started, and see our Jupyter Notebook 
If this is a bug report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.
If this is a custom model or data training question, please note that Ultralytics does not provide free personal support. As a leader in vision ML and AI, we do offer professional consulting, from simple expert advice up to delivery of fully customized, end-to-end production solutions for our clients, such as:
- Cloud-based AI systems operating on hundreds of HD video streams in realtime.
- Edge AI integrated into custom iOS and Android apps for realtime 30 FPS video inference.
- Custom data training, hyperparameter evolution, and model exportation to any destination.
For more information please visit https://www.ultralytics.com.
Have you solved the problem? I have the same question here
Have you solved the problem? I have the same question here
@dlawrences cool
Hi both,
It very much seems that the above script generates the results as per the three raw output layers:
(1, 3, 20, 20, 85)(1, 3, 40, 40, 85)(1, 3, 80, 80, 85)
These results are not final. In the detect.py script these are also processed during inference:
Lines 29 to 36 in a1c8406
You have two options:
- Change export.py to include the
Detectlayer:
Line 28 in a1c8406
Above needs to be changed to false.
I have done this for my own experiments. The ONNX export seems to work, however the CoreML one doesn't.
- Create logic to replicate inference steps in
Detectlayer
You could replicate the same logic that's referenced above using numpy (i.e. pass the results through sigmoid and do all the handling).
In both cases, you do miss the following:
- filtering results with objectness lower than some threshold
- NMS
- conversion from
xc, yc, w, htox1, y1, x2, y2
These are currently done as part of the following function:
Lines 549 to 554 in a1c8406
Hope this is useful. Good luck!
thanks!
@dlawrences model.model[-1].export = False
After modifying it to false, there is still a problem with generating .onnx parsing. Can you refer to the code for parsing onnx? Thank you!
@WZTENG what is the error you are encountering?
model.model[-1].export = False
After modifying it to false, the result is still problematic, and it feels worse than before.
I understand why you're saying it feels worse than before, but that it is because you are now missing any NMS step, as specified above. Could you please answer/check on the following points?
- I see you are training the COCO dataset: have you generated your own custom anchors or are you running on those provided by @glenn-jocher ?
- Please share the training results of your yolo5s model; I am fairly interested in how many epochs you have trained for
- Please share the hyperparams you have used during training
- Please run the same image through detect.py and show the result
Also, I would like to have a look at the .onnx file, just to make sure there's nothing wrong with you. Would you please attach it here?
Thanks
Use the official yolov5s.pt file to convert it. Not trained by myself.
model.model[-1].export = False ---> yolov5s.onnx
The result of detect.py operation is correct.
before:
anchors = [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]] # 5s
after:
anchors = [[116, 90, 156, 198, 373, 326], [30, 61, 62, 45, 59, 119], [10, 13, 16, 30, 33, 23]] # 5s
After modifying the anchors sequence, I found that some parts are normal, but there are still problems with other pictures.
Finally succeeded. It is estimated that some of the internal processing methods are different from what I thought before. If I have time, I will carefully check the internal processing process. The current successful way is to set model.model[-1].export = False, and use the output [0] and call the official NMS to display it correctly. Previously, I used the results with 3 outputs and processed the relevant content myself. However, thank you for providing useful information.
Hi @WZTENG
Great news, happy you have managed to do it! I think it would be really useful for others to create a mini-documentation containing your findings. Would you be willing to put together this info?
Tip: It should however be possible to process the output of the three feature maps independently from the Detect layer by replicating all those operations in NumPy/any other framework though. I have managed to do it myself using CoreML ops.
Cheers
Additional info: I am not sure what you mean by "and use the output [0]", but if you are only consuming the results as per the higher level feature map (80x80), then you are missing on some results.
Please consider that the Detect layer, at least as per my memory, produces outputs scaled to the dimensions of the input image (i.e. the original image may be 1920x1080, but you have trained using 640x640 inputs, so this is the dimension space). In detect.py, there is some logic that handles this. Namely:
-
results from the three feature maps are scaled back to the initial image shape (
im0); this also takes into account any letterboxing and handles the right padding
Lines 83 to 85 in a040500
-
results are converted to x, y, w, h and normalized:
Line 95 in a040500
These operations are not done in the Detect layer, but as part of the post-processing (even after NMS).
It is only successful now, but I am still not sure what caused it. If it can be solved, I will send it out.
It feels a bit different from the data I have seen before, but the output is no problem. The following is the method I implemented, although there are more, but at least I understand how to deal with it. I hope that those who have encountered this problem can refer to it.
This is the analysis method I wrote myself. Please note that the parameters of model.model[-1].export = BOOL have a big difference during export.
def detect_onnx(official=True, image_path=None):
num_classes = 80
anchors = [[116, 90, 156, 198, 373, 326], [30, 61, 62, 45, 59, 119], [10, 13, 16, 30, 33, 23]] # 5s
session = onnxruntime.InferenceSession('./weights/yolov5s.onnx')
# print("The model expects input shape: ", session.get_inputs()[0].shape)
batch_size = session.get_inputs()[0].shape[0]
img_size_h = session.get_inputs()[0].shape[2]
img_size_w = session.get_inputs()[0].shape[3]
# input
image_src = Image.open(image_path)
resized = letterbox_image(image_src, (img_size_w, img_size_h))
img_in = np.transpose(resized, (2, 0, 1)).astype(np.float32) # HWC -> CHW
img_in = np.expand_dims(img_in, axis=0)
img_in /= 255.0
# print("Shape of the image input shape: ", img_in.shape)
# inference
input_name = session.get_inputs()[0].name
outputs = session.run(None, {input_name: img_in})
batch_detections = []
if official and len(outputs) == 4: # model.model[-1].export = boolean ---> True:3 False:4
# model.model[-1].export = False ---> outputs[0] (1, xxxx, 85)
# official
batch_detections = torch.from_numpy(np.array(outputs[0]))
batch_detections = non_max_suppression(batch_detections, conf_thres=0.4, iou_thres=0.5, agnostic=False)
else:
# model.model[-1].export = False ---> outputs[1]/outputs[2]/outputs[2]
# model.model[-1].export = True ---> outputs
# (1, 3, 20, 20, 85)
# (1, 3, 40, 40, 85)
# (1, 3, 80, 80, 85)
# myself (from yolo.py Detect)
boxs = []
a = torch.tensor(anchors).float().view(3, -1, 2)
anchor_grid = a.clone().view(3, 1, -1, 1, 1, 2)
if len(outputs) == 4:
outputs = [outputs[1], outputs[2], outputs[3]]
for index, out in enumerate(outputs):
out = torch.from_numpy(out)
batch = out.shape[1]
feature_w = out.shape[2]
feature_h = out.shape[3]
# Feature map corresponds to the original image zoom factor
stride_w = int(img_size_w / feature_w)
stride_h = int(img_size_h / feature_h)
conf = out[..., 4]
pred_cls = out[..., 5:]
grid_x, grid_y = np.meshgrid(np.arange(feature_w), np.arange(feature_h))
# cx, cy, w, h
pred_boxes = torch.FloatTensor(out[..., :4].shape)
pred_boxes[..., 0] = (torch.sigmoid(out[..., 0]) * 2.0 - 0.5 + grid_x) * stride_w # cx
pred_boxes[..., 1] = (torch.sigmoid(out[..., 1]) * 2.0 - 0.5 + grid_y) * stride_h # cy
pred_boxes[..., 2:4] = (torch.sigmoid(out[..., 2:4]) * 2) ** 2 * anchor_grid[index] # wh
conf = torch.sigmoid(conf)
pred_cls = torch.sigmoid(pred_cls)
output = torch.cat((pred_boxes.view(batch_size, -1, 4),
conf.view(batch_size, -1, 1),
pred_cls.view(batch_size, -1, num_classes)),
-1)
boxs.append(output)
outputx = torch.cat(boxs, 1)
# NMS
batch_detections = w_non_max_suppression(outputx, num_classes, conf_thres=0.4, nms_thres=0.3)
return batch_detectionsIf necessary, you can change all the methods used to numpy to achieve better, which is convenient for other frameworks.
@WZTENG can you share the whole code for onnx inference?
also what is w_non_max_suppression()
@WZTENG
thank you very much for your works and test.
Could you share your code for the function w_non_max_suppression?
Thank you again.
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zip
yolov3/v4
yolov5
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zip
Can it be used to test whether the converted best.onnx is OK?
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipCan it be used to test whether the converted best.onnx is OK?
Yes, but I only wrote the 5s part of the code, other models need to add the corresponding anchors part. Note the order of anchors.
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipCan it be used to test whether the converted best.onnx is OK?
Yes, but I only wrote the 5s part of the code, other models need to add the corresponding anchors part. Note the order of anchors.
I use your code te test yolov5s.onnx ,but something eror happened:
File "demo_onnx.py", line 306, in
detections = detect_onnx(official=False, image_path=image_path)
File "demo_onnx.py", line 234, in detect_onnx
pred_boxes[..., 0] = (torch.sigmoid(out[..., 0]) * 2.0 - 0.5 + grid_x) * stride_w # cx
TypeError: add(): argument 'other' (position 1) must be Tensor, not numpy.ndarray
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipCan it be used to test whether the converted best.onnx is OK?
Yes, but I only wrote the 5s part of the code, other models need to add the corresponding anchors part. Note the order of anchors.
I use your code te test yolov5s.onnx ,but something eror happened:
File "demo_onnx.py", line 306, in
detections = detect_onnx(official=False, image_path=image_path)
File "demo_onnx.py", line 234, in detect_onnx
pred_boxes[..., 0] = (torch.sigmoid(out[..., 0]) * 2.0 - 0.5 + grid_x) * stride_w # cx
TypeError: add(): argument 'other' (position 1) must be Tensor, not numpy.ndarray
It works normally for me. Did you modify the code? You can also use the conversion function for conversion.
numpy->tensor: torch.from_numpy(numpy array)
tensor->numpy: tensor array .numpy()
The code of this zip is yolov5_v1.x version, not yolov5_v2 version.
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipCan it be used to test whether the converted best.onnx is OK?
Yes, but I only wrote the 5s part of the code, other models need to add the corresponding anchors part. Note the order of anchors.
I use your code te test yolov5s.onnx ,but something eror happened:
File "demo_onnx.py", line 306, in
detections = detect_onnx(official=False, image_path=image_path)
File "demo_onnx.py", line 234, in detect_onnx
pred_boxes[..., 0] = (torch.sigmoid(out[..., 0]) * 2.0 - 0.5 + grid_x) * stride_w # cx
TypeError: add(): argument 'other' (position 1) must be Tensor, not numpy.ndarrayIt works normally for me. Did you modify the code? You can also use the conversion function for conversion.
numpy->tensor: torch.from_numpy(numpy array)
tensor->numpy: tensor array .numpy()
The code of this zip is yolov5_v1.x version, not yolov5_v2 version.
NO ,i modify nothing. Could you upload your yolov5s.onnx or other onnx so as to check whether the onnx that converted is not correct ?
@WZTENG For me it worked, I only got a Type-error for the draw rectangle input. When I casted the box coordinates to int it all worked using the latest commit.
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipCan it be used to test whether the converted best.onnx is OK?
Yes, but I only wrote the 5s part of the code, other models need to add the corresponding anchors part. Note the order of anchors.
I use your code te test yolov5s.onnx ,but something eror happened:
File "demo_onnx.py", line 306, in
detections = detect_onnx(official=False, image_path=image_path)
File "demo_onnx.py", line 234, in detect_onnx
pred_boxes[..., 0] = (torch.sigmoid(out[..., 0]) * 2.0 - 0.5 + grid_x) * stride_w # cx
TypeError: add(): argument 'other' (position 1) must be Tensor, not numpy.ndarrayIt works normally for me. Did you modify the code? You can also use the conversion function for conversion.
numpy->tensor: torch.from_numpy(numpy array)
tensor->numpy: tensor array .numpy()
The code of this zip is yolov5_v1.x version, not yolov5_v2 version.NO ,i modify nothing. Could you upload your yolov5s.onnx or other onnx so as to check whether the onnx that converted is not correct ?
It's ok now.
@WZTENG thanks for the .zip! It works well for me
@china56321 @EmilioOldenziel should we modify this script so we don't have to import the heavy torch package? I'm working to that end and can update here if we are interested.
I did get that working so ONXX no torch is possible with no torch and @WZTENG's script!
@WZTENG Thx for the script and script. I'm new to Yolo so it really cleared up a lot of confusion. But are you able to get similar results to running directly with the .pt file? I'm not able to reproduce the same results from a .pt file with your script.
I tried my own fine-tuned weight as well as the origin checkpoints weight for V2
@WZTENG Thx for the script and script. I'm new to Yolo so it really cleared up a lot of confusion. But are you able to get similar results to running directly with the .pt file? I'm not able to reproduce the same results from a .pt file with your script.
I tried my own fine-tuned weight as well as the origin checkpoints weight for V2
After looking into it more I was able to figure out the issues I was having. For some reason unless I export to onnx with size 640 the box coordinates in your script aren't correct.
But the main issue is the NMS step. You can see in the image that I've attached below that the NMS isn't performing how you'd expect. And suggestions/advice?
download demo_onnx.zip and run demo_onnx.py. It can be run in v1 version, v2 version is not verified. It is important to note the order of anchorns. The order of v2 and v1 seems to have changed.
download demo_onnx.zip and run demo_onnx.py. It can be run in v1 version, v2 version is not verified. It is important to note the order of anchorns. The order of v2 and v1 seems to have changed.
Thx. The anchors was the issue
@Jacobsolawetz , would you mind sharing your script that does nms on the onnx output without using torch please ? Did you use the output with the detect layer that give a shape (batch_size, XXXXX, number_of_classes+5) ?
torch.cat
torch.tensor
torchvision.ops.boxes.nms
torch.mm
I did get that working so ONXX no torch is possible with no torch and @WZTENG's script!
@Jacobsolawetz Could you share you modified demo_onnx.py ?
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zip
hello, do you have a cpp version for demo_onnx.py?
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.ziphello, do you have a cpp version for demo_onnx.py?
#343 (comment)
In fact, there is no difference from v4 and v3. You can directly find the previous cpp version and modify the formula and anchors.
Hi @WZTENG
Great news, happy you have managed to do it! I think it would be really useful for others to create a mini-documentation containing your findings. Would you be willing to put together this info?
Tip: It should however be possible to process the output of the three feature maps independently from the Detect layer by replicating all those operations in NumPy/any other framework though. I have managed to do it myself using CoreML ops.
Cheers
@dlawrences can you please share more info on how to do it using CoreML ops ?
download demo_onnx.zip and run demo_onnx.py. It can be run in v1 version, v2 version is not verified. It is important to note the order of anchorns. The order of v2 and v1 seems to have changed.
hello，thanks for your work! I have test by using yoru code, and check the anchors, they are the same as the yolov5! But my test results is not good by testing by the onnx model! The results are shown by the follow 2 images! I don't konw why, may the NMS? Can you give me some advice? Thanks very much!
@zzzz737
The demo only applies to v1, v2 needs to modify the anchors order, v3 has not been used. For the time being, it is clear what caused this problem. Sorry.
@zzzz737
The demo only applies to v1, v2 needs to modify the anchors order, v3 has not been used. For the time being, it is clear what caused this problem. Sorry.
Yeah, I changed the anchors order, the results are the same as the pt model! Thank you!
@zzzz737
The demo only applies to v1, v2 needs to modify the anchors order, v3 has not been used. For the time being, it is clear what caused this problem. Sorry.Yeah, I changed the anchors order, the results are the same as the pt model! Thank you!
hi，can you share the whole code for onnx inference，Thank you
@zzzz737
The demo only applies to v1, v2 needs to modify the anchors order, v3 has not been used. For the time being, it is clear what caused this problem. Sorry.Yeah, I changed the anchors order, the results are the same as the pt model! Thank you!
hi，can you share the whole code for onnx inference，Thank you
the demo is given by the yolov5 projects in github by ultralytics/yolov5
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zip
Does it only support square size (eg,640640) ? What should i do if i want to change the onnx input size(eg,640320 or other size ) ?
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipDoes it only support square size (eg,640_640) ? What should i do if i want to change the onnx input size(eg,640_320 or other size ) ?
320 is always ok！
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipDoes it only support square size (eg,640_640) ? What should i do if i want to change the onnx input size(eg,640_320 or other size ) ?
320 is always ok！
I mean 640 X 320 or other size,eg 460*860 ,not 640 X 640 ,What should i do to achieve this function ?
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipDoes it only support square size (eg,640_640) ? What should i do if i want to change the onnx input size(eg,640_320 or other size ) ?
320 is always ok！
I mean 640_320 ,not 640_640
you can try，i think it's ok
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipDoes it only support square size (eg,640_640) ? What should i do if i want to change the onnx input size(eg,640_320 or other size ) ?
320 is always ok！
I mean 640_320 ,not 640_640
you can try，i think it's ok
Yes ,i have tried it ,the box is wrong ,all are shifted, Can you sovle it ?
I did get that working so ONXX no torch is possible with no torch and @WZTENG's script!
Hi @Jacobsolawetz ,
By any chance are you willing to share your script with no torch dependency?
Regards,
E
Hi all,
@WZTENG thanks for providing the code for onnx inference. I improvised the anchors as you suggested. The code works fine for my single class model, on a yolov5m.pt model converted to onnx with model.model[-1].export = True. The detection results of .pt and .onnx inference match perfectly. Model was based on yolov5 v3.
For yolov5x based weights file, I exported onnx with model.model[-1].export = False in export.py. Also, in the demo_onnx.py file, detections should be obtained with
detections = detect_onnx(official=True, image_path=image_path)
And now my .pt and .onnx file results match perfectly for yolov5x based architecture for yolov5 v3 model ... Thanks a lot
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zip
Does your inference script work for the v3 graph? I'm having issues with the final detection boxes. I'm using the same anchors as I did with the v2 graph. But changed the input size to 416
Thx in advance
UPDATE: My problem actually seems to be with the 416 input size. But I'm not sure why the post-processing would change because of it
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zip
I have tested this demo_onnx.zip to do inference with a custom trained model based on default yolov5s, all I need to do is to modify the anchors using [[10, 13, 16, 30, 33, 23], [30, 61, 62, 45, 59, 119], [116, 90, 156, 198, 373, 326]], num_classes and the onnx model path inside the detect_onnx function. Thank you so much for your work.
Anyone help me!!When I execute the command on NVIDIA Jetson NX device:. / trtexec -- onnx = yolov5s. Pt -- saveengine = yolov5s. Onnx, I get an error.
Input filename: yspc.sim.onnx
ONNX IR version: 0.0.6
Opset version: 11
Producer name: pytorch
Producer version: 1.7
Domain:
Model version: 0
Doc string:
[01/07/2021-14:44:07] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/07/2021-14:44:07] [W] [TRT] onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[01/07/2021-14:44:07] [I] [TRT] ModelImporter.cpp:135: No importer registered for op: ScatterND. Attempting to import as plugin.
[01/07/2021-14:44:07] [I] [TRT] builtin_op_importers.cpp:3659: Searching for plugin: ScatterND, plugin_version: 1, plugin_namespace:
[01/07/2021-14:44:07] [E] [TRT] INVALID_ARGUMENT: getPluginCreator could not find plugin ScatterND version 1
ERROR: builtin_op_importers.cpp:3661 In function importFallbackPluginImporter:
[8] Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
[01/07/2021-14:44:07] [E] Failed to parse onnx file
[01/07/2021-14:44:07] [E] Parsing model failed
[01/07/2021-14:44:07] [E] Engine creation failed
[01/07/2021-14:44:07] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=yspc.sim.onnx --saveEngine=yspc.trt
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipDoes it only support square size (eg,640_640) ? What should i do if i want to change the onnx input size(eg,640_320 or other size ) ?
320 is always ok！
I mean 640_320 ,not 640_640
you can try，i think it's ok
Yes ,i have tried it ,the box is wrong ,all are shifted, Can you sovle it ?
I met the same problem. The input image after lettorbox size is (640,480), but the onnx model expected input is (640,640). Have you solved the problem?
i unzipped and ran the demo_onnx.py.
i encountered the error below:
OSError: [WinError 127] The specified procedure could not be found. Error loading "C:\Users\User\anaconda3\envs\yolov5\lib\site-packages\torch\lib\cublas64_11.dll" or one of its dependencies.
may i know what could be the problem?
Hi, i tried running the demo_onnx.py. i didn't change anything else except for the number of classes and the anchors.
below is the error i got. any idea what the issue is?
Traceback (most recent call last):
File "demo_onnx.py", line 318, in
detections = detect_onnx(official=False, image_path=image_path)
File "demo_onnx.py", line 253, in detect_onnx
output = torch.cat((pred_boxes.view(batch_size, -1, 4),
RuntimeError: Sizes of tensors must match except in dimension 2. Got 19200 and 16000 in dimension 1 (The offending index is 2)
@gohguodong sorry this code you ran is not part of the official repo code so we can't provide support for it. We are working on providing better inference examples for deployment environments in the future though!
@glenn-jocher any updates on this?
@lleye there are constant updates to export.
@glenn-jocher Is someone currently working on improving the CoreML export? It would be great if it could generate a complete object detection model (including the Detect layers feeding into CoreML's NMS layer) that then could be used easily with Apple's Vision framework. I would be happy to help with this (to the best of my abilities).
@mukul1em @imyoungyang here. download demo_onnx.zip and unzip.
demo_onnx.zipDoes it only support square size (eg,640_640) ? What should i do if i want to change the onnx input size(eg,640_320 or other size ) ?
320 is always ok！
I mean 640_320 ,not 640_640
you can try，i think it's ok
Yes ,i have tried it ,the box is wrong ,all are shifted, Can you sovle it ?
I met the same problem. The input image after lettorbox size is (640,480), but the onnx model expected input is (640,640). Have you solved the problem?
You have to shift the boxes int(pad*(1/scale)) pixels
In plot_one_box function you sould write:
pad = int(pad*(1/scale))
h, w = image.shape[:2]
if h > w:
c1, c2 = (int(x[0])-pad, int(x[1])), (int(x[2])-pad, int(x[3]))
else:
c1, c2 = (int(x[0]), int(x[1])-pad), (int(x[2]), int(x[3])-pad)
image is your orijinal image
@glenn-jocher Is someone currently working on improving the CoreML export? It would be great if it could generate a complete object detection model (including the Detect layers feeding into CoreML's NMS layer) that then could be used easily with Apple's Vision framework. I would be happy to help with this (to the best of my abilities).
Were you ever able to use an exported coreml model? I'm struggling to figure out how to export correctly.
@tylercollins590 Yes, I was. See my reply here.
@pocketpixels thank you. Does this work on V6 of Yolov5 or a previous version?
@tylercollins590 Not sure. I haven't tested with the latest version. You can find my fork with this script here. Note that I made my changes in the branch better_coreml_export.
@pocketpixels got it exported and working in my iOS app. I'm seeing very low FPS (5-7fps) with Yolov5s. Did you experience similar?
Performance will depend on the device and the image resolution you use.
@pocketpixels I am using an iphone 12 pro. iDetection from ultralytics runs Yolov5s at about 35 FPS. I am using 640x640 resolution in my exported model. Any ideas where I should focus?
I am not using it for continuous detection myself. And wouldn’t really recommend that either as iOS devices will burn through battery and will throttle down pretty quickly if you max out the GPU and/or Neural Engine and CPU.
The iDetection app uses a lower resolution.
IDetection uses 320x192. So about 7x less pixels. So the performance you are seeing seems reasonable.
@tylercollins590 did you get the script by @pocketpixels to work on v6 (or 6.1) of YOLOv5? Managed to work around some numpy syntax changes (line 59) however am running into issues with arrays shapes, a bit beyond me...
Fusing layers...
Model summary: 213 layers, 7012822 parameters, 0 gradients, 15.8 GFLOPs
terminate called after throwing an instance of 'std::runtime_error'
what(): shape '[1, 3, 1, 1, 2]' is invalid for input of size 38400
Aborted (core dumped)
@mshamash unfortunately I have not been able to get anything to run on any version past v4
@tylercollins590 Interesting, as I did get that script to work on my v5 YOLOv5s models in late-2021 (final CoreML models had NMS)
@mshamash unfortunately I have not been able to get anything to run on any version past v4
@tylercollins590 - After much effort, and trial and error, I got the .mlmodel files from the YOLOv5 (v6.1) export.py script to work within my iOS app on still images. If this is something you are also doing with your models I'd be happy to share the code.
I'll be making a new repo with an entire sample project soon, as (surprisingly) there's no good public, simple, example for using YOLOv5 with CoreML. At least none that I could find or use...