d4l3k/torchdrive

Question Rendering Depth

gbyy422990 opened this issue · 6 comments

gbyy422990 commented

Hi, I am really interested in depth Rendering that mentioned in your blog(https://fn.lc/post/voxel-sfm/). Could you please provide more code that i can try?

d4l3k commented

Let me know if you have any more questions :)

gbyy422990 commented

The voxel task + rendering code is at https://github.com/d4l3k/torchdrive/blob/main/torchdrive/tasks/voxel.py

The custom raymarcher that I'm using to render depth + semantic classes is at https://github.com/d4l3k/torchdrive/blob/main/torchdrive/raymarcher.py#L10

For meshes I've upstreamed the shaders to https://github.com/facebookresearch/pytorch3d/blob/e245560abb8f019a24880faf7557ed3b2eec6cc0/pytorch3d/renderer/mesh/shader.py#L374-L442

@d4l3k really thanks bro. I mean how do you render depth from occupancy grid? you mentioned here:
image

But I didn't find more information in your code, like function render_dist, _points, _dist_grid, etc.

gbyy422990 commented

@d4l3k what i want to do is that since my network produce occupancy grid, then i render depths from the occupancy grid, we can compare render depths with our depth GT.

d4l3k commented

@gbyy422990 take a look at self.renderer and how it's used https://github.com/d4l3k/torchdrive/blob/main/torchdrive/tasks/voxel.py#L133-L151

I'd also recommend reading up on pytorch3d raymarching https://pytorch3d.org/tutorials/fit_textured_volume

gbyy422990 commented

@d4l3k hhh, got it, I didn’t read the code carefully earlier.