IDEA-Research/Grounded-SAM-2

error when testing a picture that has a low resolution?

Closed this issue · 7 comments

baiyuting commented

I use a 294x78 PNG to test grounded_sam2_hf_model_demo.py, but I get following errors, any solution?

  File "D:\project\Grounded-SAM-2\sam2\sam2_image_predictor.py", line 417, in _predict
    low_res_masks, iou_predictions, _, _ = self.model.sam_mask_decoder(
  File "D:\program\anaconda3\envs\llama_adapter\lib\site-packages\torch\nn\modules\module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "D:\program\anaconda3\envs\llama_adapter\lib\site-packages\torch\nn\modules\module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "D:\project\Grounded-SAM-2\sam2\modeling\sam\mask_decoder.py", line 136, in forward
    masks, iou_pred, mask_tokens_out, object_score_logits = self.predict_masks(
  File "D:\project\Grounded-SAM-2\sam2\modeling\sam\mask_decoder.py", line 203, in predict_masks
    assert image_embeddings.shape[0] == tokens.shape[0]
AssertionError
rentainhe commented

Would you like to provide the demo image you use for us to test this issue, which may be more convenient for solving this problem. It seems like SAM 2 did not support image with very low resolution.

baiyuting commented

I find the problem. The input_boxes shape is [0,4] after the processor.post_process_grounded_object_detection， so I add if(xx) to avoid this situation. Thanks.

rentainhe commented

I find the problem. The input_boxes shape is [0,4] after the processor.post_process_grounded_object_detection， so I add if(xx) to avoid this situation. Thanks.

It seems like grounding_dino detect nothing in this image, I think it's better to add some warnings here for debugging, we will try to update our code in the future release

rentainhe commented

Seems like the problem has been solved, I will close this issue first, feel free to reopen it if needed.

Masrur02 commented

Hi, @baiyuting
so I add if(xx) to avoid this situation

In which code and where you add this logic?

Kairui-SHI commented

Hi, @baiyuting so I add if(xx) to avoid this situation

In which code and where you add this logic?

I think it means to set a detection to verify the input_boxes.shape, like if it's [0, 4]. It shows detect nothing.

Masrur02 commented

Hi,
I am using the grounded_sam2_local_demo.py. For this code, I do not see the boxes. shape. The assertion error occurs when the tokens[0]==0. That means the grounding dino is probably not predicting anything. If it does not predict anything, how can we pass the error?