mikel-brostrom/boxmot

Why is my MOTA negative?

Closed this issue · 8 comments

Search before asking

  • I have searched the Yolov8 Tracking issues and found no similar bug report.

Question

I run val.py on my own custom dataset, then I found MOTA is negative.
Below is my gt.txt and evaluation results.
image
gt.txt

There can be many reasons: GT not following the expected format, bad detector, kalman filter over/under-shooting...

image
Would you like to help me see if my gt.txt is correct? I compared it with the format of the mot17 dataset. I didn't find the error. I also uploaded gt.txt above.

image
I validated the detector I trained, map is not good, but does it result in a negative MOTA?

Negative MOTA does not necessarily indicate a problem with the evaluation. MOTA is negative when FN + FP > GT (described in the HOTA paper https://arxiv.org/abs/2009.07736; "MOTA scores can be negative and are unbounded.").

Thank you very much for your answer. I think my custom model for salmon is too bad, and I'm trying to improve it. It's hard because I can not use pretrained weights or re-ID model.

By the way, I have a question about how to validate an existing result. When I put track.py's result to validate, it works. When I put my validation set's gt.txt to validate, i think it will show 100, but it showed 0.
image

Never tried to input GT as predicted data as well. Will get back at you when I try this out 😄

image Would you like to help me see if my gt.txt is correct? I compared it with the format of the mot17 dataset. I didn't find the error. I also uploaded gt.txt above.

You may try to change your cls_id to 1-based (starting from 1), then retry.