BobLiu20/YOLOv3_PyTorch

error during training

Closed this issue · 3 comments

zhanghanduo commented

Hi. During training with other dataset, I found the following error:
error

This error occurs everytime but in different steps, sometimes 400, sometimes 3800. Could you figure out what is the problem?

zhanghanduo commented

I found the problem. gj and gi (grid box indices) are theoretically within the box of in_h, in_w. But somehow occasionally gj or gi jumps out of the box which causes the error. So I added a conditional judge for get_target function in yolo_loss.py:
tes1

I didn't pull request because I am not sure whether this is correct. I need the author's opinion~~

BobLiu20 commented

@zhanghanduo Hi, I think the root case is your ground truth has the value of bbox greater than 1.0 in some image. So, You can limit your target value from 0.0 to 1.0 in this case.
image

So, In my opinion. Do not add a condition to skip good bbox and just limit it in dataset reading.
image
The pull request is welcome after your verify!

zhanghanduo commented

Thank you! Actually I thought about the reason is my data problem. But 99% of the data is correct which made me confused. I thought the special data was wrong and should be excluded. But you are right, I should not skip good bbox.