dtorre38/mujoco_opencv

bounding box does not overlap objects in scene

Closed this issue · 3 comments

Hi, thanks for the great repo.

in opencv_python.py, just running it normally without modifying the code,

Screenshot 2024-07-18 at 7 13 37 PM

the code seems to be working correctly but the green bounding box does not seem to be rendered correctly. The green rectangle seems to be "under" the yellow cube and red plane. Is there anyway to render the box over on top of it?

Hi,

You are correct, I have noticed this issue, but I haven't gotten around to fixing it. It has to do with how the scene is rendered. It seems that the bounding box (in its current implementation) cannot be rendered over geometries. I couldn't find an easy fix, so I just let it be. If you find a fix please let me know.

Hi, after spending an embarrassingly amount of time trying to find a solution,

import OpenGL
from OpenGL.GL import *

then put,

glClear(GL_DEPTH_BUFFER_BIT)
mj.mjr_drawPixels(pixels None, body_viewport, self.context)

the glClear must go right before your mjr_drawPixels. This seems to work for me on my own project. The whole problem is some sort of z-depth problem per pixel.

Thanks! This solves all my issues! This fix also works with the rendering of the depth. I'm going to make the fixes across all the code and push the changes.