Trying to use SAM to segment engineering rock samples, but I need to solve two issues
citystrawman opened this issue · 2 comments
Hello, I am trying to use SAM to segment engineering rock samples as follows:
Here's a photo that contains a box of rock samples, and what I need to do is segment all the rocks and then choose the ones that is longer than a specific length(as we know the length of the box, we can easily calculate the length of each rock based on its pixel size).
By using SAM SamAutomaticMaskGenerator, I can get the segmented elements of this photo as follows:
However, there're two issues which I think should be solved first so that my final target can be reached:
The first one is the overlapping elements. As SAM may have overlapping segment elements, I may count repeated rocks. Is it possible to make SAM generate all non-overlapping elements?
The second one is how to separate the rocks from backgrounds. SAM segments everything including photo background, and is it possible so that I can only segment rocks?
I appreciate if anyone could help solve this issue. Thank you !
I think non-maximum overlapping suppression (NMS) can solve your overlapping problem an you can use a classifier to to distinguish which one is background or a rock. I am not sure of that works but you can also use a pretrained network to extract embeddings and calculate a similarity between all these objects. Since you have a lot of rock probably what you would like to segment will create a cluster.
I think non-maximum overlapping suppression (NMS) can solve your overlapping problem an you can use a classifier to to distinguish which one is background or a rock. I am not sure of that works but you can also use a pretrained network to extract embeddings and calculate a similarity between all these objects. Since you have a lot of rock probably what you would like to segment will create a cluster.
Thanks. As for non-maximum overlapping, I think segment-anything has already implemented it, but it still has possibility to generate overlapping masks. Just use the above images as an example, I have generated two images as follows:
the first one lists all masks while the second one lists top 15 large masks. as you can see, mask 9 contains mask 58, mask 17, as well as mask 39. I think that is due to the semantic segmentation similar as the example shown by official:
