mihaidusmanu/d2-net

How to manually give the correspondences to train the network

Aniket-Gujarathi opened this issue · 2 comments

Hi ,
I would like to know how can I manually give the correspondences to train the network, instead of giving pose, depth, rgb and intrinsics. It would be much convenient for me train that way in my project.
1. Could you point me towards the code where we could provide direct correspondences.
2. I have looked into the warp function in the loss.py. And it looks like it is returning the pixel locations for corresponding pixels as pos1 and pos2. But I am not sure of ids tensor, is it returning the descriptors' index for pos1 in row major order? Could we replace this warping function with our input data.

In theory you can replace the function warp with your correspondences.

The ids are used to filter out points / descriptors that do not have correspondences. As you stated, it is simply the row-major index of valid points of the grid.

d2-net/lib/loss.py

Lines 70 to 72 in 2a4d88f

fmap_pos1 = fmap_pos1[:, ids]
descriptors1 = descriptors1[:, ids]
scores1 = scores1[ids]

Hi,
Thank you for the clarification.