xinghaochen/TinySAM

how to evaluate on LVIS？

ranpin opened this issue · 2 comments

ranpin commented

Hello, I find your work and it's great.

I'm a newbie and I'm having some difficulties in reproducing the LVIS validation effect, and I've looked up a lot of information but I can't figure it out. Therefore, I would like to get your help. Specifically:

  1. you don't have the instructions and results for reproducing LVIS in your readme, I wonder if it would be convenient to update it? Thanks.

  2. when I use your script and the shared lvis_instances_results_vitdet.json and lvis_v1_val.json for validation test, it always reports error that it didn't find certain images, which I found out actually belong to coco train dataset. But the LVIS val dataset I downloaded from the LVIS website is exactly the same as coco val, i.e. it does not contain the images from coco train. I would like to ask, how exactly is the LVIS validation set divided into components? Or is there a recommended download link? Thanks.

shuh15 commented

Hi~

  1. You can refer to (https://github.com/xinghaochen/TinySAM/blob/main/eval/tinysam_zero_shot_ins_eval.py) for evaluation on LVIS, we have provided all the functions needed for evaluation on LVIS, what you need to do is to specify the 'eval_type' and paths.
  2. You can refer to the official site of LVIS, https://www.lvisdataset.org/dataset.
ranpin commented

Hi~

  1. You can refer to (https://github.com/xinghaochen/TinySAM/blob/main/eval/tinysam_zero_shot_ins_eval.py) for evaluation on LVIS, we have provided all the functions needed for evaluation on LVIS, what you need to do is to specify the 'eval_type' and paths.
  2. You can refer to the official site of LVIS, https://www.lvisdataset.org/dataset.

Thanks, I solved it.