siyuanliii/masa

the inference code of tracking like Figure 3. (b) in paper

Closed this issue · 3 comments

jru001 commented

Hi Siyuan,
Thanks for your great work and released codes!
I'm wondering how to track by combining the detection head (of MASA Adapter) and SAM directly, just like the Figure 3. (b) in paper. Will it be released later?

siyuanliii commented

hi thanks, SAM's prediction is not consistent across video frames hence leading to heavy flickering due to missing detections. Therefore, we have not provided such a demo yet. We will try to reduce the flickering effect first. However, it is straightforward and simple to test it for yourself first. You can replace the detection bounding boxes with the output of masa trained detection head and run.

jru001 commented

Thanks for your reply, looking forward to your new achievements.

ddshan commented

Hi Siyuan,

Congrats on this amazing work! It is very good to use for tracking!

I am interested in associating SAM masks between frames and still have questions about it.

Regarding the suggestion you gave above -- "replace the detection bounding boxes with the output of masa-trained detection head and run", could you give some quick guidance on how to use the masa-trained detection head?
Currently, the demos use grounding-DINO/Yolox/Co-DETR to find objects. What changes to make so as to call the masa-trained detector?