Ali2500/BURST-benchmark

Problems about the open world setting.

yahooo-m opened this issue · 9 comments

Great job!!!
I have a question about the demo on open world tracking and segmentation. I see that you only use the off-the-shelf model to generate results. Whether the model is trained in the train set?
I download the annotations about the dataset. And i show the categories in the train. For open world tracking, it should contain the normal classes, but it has the overall classes. if i do the open-world task, should I divide the data myself?

Looking forward to your kindly reply!

Hi,

I'm not sure if I understand the question fully, but for the open-world task you should train on only the common class set, but eval with all the classes.

Sorry for confused!
Does the train.json only contain the common classes?

Another question about the data.
I run the provided demo to visualize the result. I got bad results like the following images. How to use the annotation for open-world?
微信图片_202305101427402

Sorry for confused! Does the train.json only contain the common classes?

No, the training json annotations file contains objects belonging to all classes. You can use the info in class_splits.json to filter out only the common class objects.

Another question about the data. I run the provided demo to visualize the result. I got bad results like the following images. How to use the annotation for open-world? 微信图片_202305101427402

What's wrong with the given image?

Another question about the data. I run the provided demo to visualize the result. I got bad results like the following images. How to use the annotation for open-world? 微信图片_202305101427402

What's wrong with the given image?

Could you kindly explain why the annotation does not include the right two sandwiches?

The annotations in our dataset are not exhaustive. Please see the "Federated Annotations" paragraph in Sec. 2 of the paper for more details.

Thanks