Superpoint + superglue w/ hloc give 90 degree camera orientation error
sfm-user-1 opened this issue · 3 comments
Hi, thanks for making this repo and the notebooks.
I tried out the superpoint+superglue+colmap notebook here
https://github.com/cvg/Hierarchical-Localization/blob/master/pipeline_SfM.ipynb
using the 35-image dataset here: http://vision.maths.lth.se/calledataset/de_guerre/de_guerre.zip
![spsg_vs_sift_colmap](https://private-user-images.githubusercontent.com/141111364/301087641-09e5fbdf-567d-4c49-89a5-6f63bf310a12.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTg1NDgxOTYsIm5iZiI6MTcxODU0Nzg5NiwicGF0aCI6Ii8xNDExMTEzNjQvMzAxMDg3NjQxLTA5ZTVmYmRmLTU2N2QtNGM0OS04OWE1LTZmNjNiZjMxMGExMi5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNjE2JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDYxNlQxNDI0NTZaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1lODYzOWNlYTJhY2E2YTlhNWQ1ODEzMDIzNTNjMTI2YThlYjdmNGUzMjNkMmIwZDU5MmIxYTBiZWU4YWFiMWRkJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.9Ro63JQxQZNMNe5KJLq_jLrsAabLfywQCNzU9h2X6lo)
It seems superpoint+superglue w/ HLOC estimates most camera orientations incorrectly by 90 degrees error about the +z camera axis (roll). I don't see any such issue with vanila COLMAP. I wonder if you could try reproducing or investigating this result?
Thanks again!
Hi, a gentle bump on this. 90 degree orientation error for most cameras seems like it is a bug in the hloc source code. Thanks!
This indeed looks like a problem, I'll investigate once I find the time. Thank you for reporting this!
Found the culprit - PR #373 solves this bug. Unfortunately SuperPoint+SuperGlue cannot match 90-degree rotated images, so for now hloc registers only 26/35 images (those in portrait mode). Fixing this is pretty high on our priority list. For now I suggest baking the EXIF rotation into the images by reading+writing them with OpenCV:
import cv2
for p in input_dir.iterdir():
cv2.imwrite(str(output_dir / p.name), cv2.imread(str(p)))
This prevents COLMAP from sharing intrinsics between rotated images but the negative impact should be negligible.