Panseg to Instance Segmentation Script Produces Inaccurate Annotations when objects overlap

[NickHerrig]

I'm using the script to transform the panseg annotations to instance segmentation and am seeing undesirable annotations.

https://github.com/bytedance/coconut_cvpr2024/blob/main/tutorials/kmaxdeeplab_instance/create_instance_by_panseg.py

Here is example image 000000004134.jpg from the val2017 dataset, specifically the dining table class overlapping the person.

.

Here is another example image 000000025139.jpg from the val2017 dataset where the zebra segmentation appear off.

It looks like the script was recently updated, do you know what could be causing this?

[NickHerrig]

For example, here are the masks for the zebra instance, So i expected them to align more closely.
Please ignore my class labels, as I purposefully didn't provide these

[xdeng7]

I'm using the script to transform the panseg annotations to instance segmentation and am seeing undesirable annotations.

https://github.com/bytedance/coconut_cvpr2024/blob/main/tutorials/kmaxdeeplab_instance/create_instance_by_panseg.py

Here is example image 000000004134.jpg from the val2017 dataset, specifically the dining table class overlapping the person.

.

Here is another example image 000000025139.jpg from the val2017 dataset where the zebra segmentation appear off.

It looks like the script was recently updated, do you know what could be causing this?

hi, thanks for the question. I believe this could be the problem of how you open the instance mask. I used pycocotools api, it looks good on my side. You can check my recent commit on the visualization: https://github.com/bytedance/coconut_cvpr2024/blob/main/tutorials/kmaxdeeplab_instance/vis_converted_instance.ipynb

attach my visualization results below:

[xdeng7]

For example, here are the masks for the zebra instance, So i expected them to align more closely. Please ignore my class labels, as I purposefully didn't provide these

for the zebra, the instance mask should be good as in my visualization.

[NickHerrig]

Thank you, I also recreated this with CV2 and just drawing the polygon segments.
I appreciate you looking at this. Closing now.