How to Generate Masks from CLEVR?

[hockeybro12]

Hello,

In the paper, you say: "Because the original CLEVR dataset does not include object masks, we generate these 4,000 training images ourselves using the CLEVR dataset generation tool". However, I don't see this process in this code release.

Can you release the code for how to do this, or give some insight into the process you used to do this?

Thanks!

[michaal94]

I think it is done in CLEVR-Ref+ generation code, I personally did that by changing rendering script after a part that uses shadeless rendering for occlusion detection - in CLEVR generation code they render image shadeless with random colour assignments - you can use those colours to create a mask for segmentation.

[drimpossible]

Hi!
To get masks from CLEVR dataset, you can add this line to image_generation/render_images.py ~line 311 (after the function add_random_objects is called):

render_shadeless(blender_objects, path=output_image[:-4]+'_mask.png')

Hope it helps!

[hockeybro12]

Thanks!

[goonmeet]

@michaal94 Would you be able to provide additional details on how to create the masks?

[michaal94]

In CLEVR generation code they check whether the objects do not occlude themselves. They do that by rendering the whole scene shadeless (line 487) - render_shadeless(objects, path). The image is saved and rendered in a temporary file. As drimpossible said you can substitute the path to the temporary file with a path you want your masked to be saved at. In such a way, you will have an image of the scene and an image of the segmentation mask. What you may want to do is to keep track of colours used in render_shadeless such that you preserve the information which colour corresponds to which object (keep track of random colours in line 535 or create a list of colours to be used to have them the same in every image).

Hope this helps.

[goonmeet]

In CLEVR generation code they check whether the objects do not occlude themselves. They do that by rendering the whole scene shadeless (line 487) - render_shadeless(objects, path). The image is saved and rendered in a temporary file. As drimpossible said you can substitute the path to the temporary file with a path you want your masked to be saved at. In such a way, you will have an image of the scene and an image of the segmentation mask. What you may want to do is to keep track of colours used in render_shadeless such that you preserve the information which colour corresponds to which object (keep track of random colours in line 535 or create a list of colours to be used to have them the same in every image).

Hope this helps.

This is helpful! Thanks :D