The code is inconsistent with the paper
Junda24 opened this issue · 7 comments
Thank you for your excellent work. However, I have a doubt. According to the statement of your paper, the depth assumption of SRC image is used to obtain the similarity score by projection transformation to ref image. However, in Code, the depth assumption is set in the Ref Image coordinate system to obtain 3D points, which are transformed to the SRC image coordinate system and sampled from the SRC image. Finally, the similarity score is obtained by dot product with the REF image. This is actually helpful to get the depth of the ref image, instead of SRC image. I don't understand why you make the depth assumption in the Ref image coordinate system and then project it.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
Hi @cjd24-coder, sorry for the confusion. I realize the variable naming(ref,src) in the code is flipped from the paper. We indeed make the depth assumption in the coordinate system of the image that we want to predict depth.
but the proj matric in homo_warping should be: proj = torch.matmul(ref_proj, torch.inverse(src_proj)) if you flipped from the paper. If I understand the formula for homo warping correctly, the formula for proj should correspond to the transformation.
Hello author, I want to know how you get this accuracy, in the case of projection transformation error. Secondly, after I fixed your projection bug, the ABS_rel error in another Dense Depth for Autonomous Driving (DDAD) dataset was reduced from 20% to 12%. However, monocular single-view networks can also achieve 13% accuracy on this data set. Do you have any tuning skills for MVS2D? Can you tell me? Thank you
I think mVS2D performance should be more than one percent more accurate than single-view networks
Hi @cjd24-coder , I don't think the proj matric calculation in homo_warping is wrong in homo_warping. It should be just proj = torch.matmul(src_proj, torch.inverse(ref_proj)), given that ref_proj is the world2cam projection matrix of ref frame(the frame we want to predict depth)
sorry, i misunderstand.