There are something wrong with the code.
hjh530 opened this issue · 12 comments
First——pip install pytorch-lightning==1.5.10can not match pip，so I use pip install pytorch-lightning==1.9.0
Second——the numpy should be1.x ，I install pip install numpy==1.23.5 .It work.
Third——in my computer I need install matplotliband omegaconf
Last——demo.py line 305 we need turn buf_ndarray = np.frombuffer(fig.canvas.tostring_rgb(), dtype="u1") to buf_ndarray = np.frombuffer(fig.canvas.buffer_rgba(), dtype="u1") This is because the tostring_rgb function was deprecated in Matplotlib 3.8 and will be removed in 3.10. We need to change the 307 lines of the code im = buf_ndarray.reshape(h, w, 3) to im= buf_ndarray.reshape(h, w, 4)
That works in my computer.
Thank you for your hard work！！！Maybe this can be useful for others.
New issue
We have to modify the hloc/triangulation.py function under the holc folder because the version is updated and your function is still not updated.
https://github.com/cvg/Hierarchical-Localization/pull/353/commits/34b7e40657463701aa39b0bfa83b60e170015cbb
- Regarding the version of pytorch-lightning, since the version I have been using is 1.5.10, it would be completely fine if using version 1.9.0 does not affect the code operation due to API changes.
- Regarding numpy, I remember that it will be installed together when installing one of the packages (maybe pytorch), so I didn't pay much attention to its version. As long as the code can run, there should be no major problem.
- Regarding matplotlib and omegaconf, thank you very much for your feedback. The code of gim indeed needs to install these two packages, and I have added them to the environment preparation in the README.
- Thank you for your improvement suggestions on the visualization code. I have also updated it to the source code.
- Regarding hloc/triangulation.py, we will update it later after testing. If you can submit a PR, that would also be a good choice.
Finally, thank you very much for your recognition and help. 🤗
- 关于 pytorch-lightning 的版本, 因为我一直使用的版本是 1.5.10, 如果使用 1.9.0 的版本不会因为 api 的改动而影响代码运行是完全没问题的.
- 关于 numpy, 我印象中它会在安装其中一个 package (也许是 pytorch) 的时候一起安装, 所以我没太注意到它的版本. 只要代码能运行, 应该问题就不大.
- 关于 matplotlib 和 omegaconf, 很感谢你的反馈, gim 的代码确实需要安装这两个 package, 我已经将它们加入 README 的 环境准备中了.
- 感谢你对可视化代码的改进建议, 我也已经将其更新到源代码中.
- 关于 hloc/triangulation.py 我们经过测试以后后续也会更新, 如果你可以提交一个 PR 也是个不错的选择
最后, 十分感谢你的认可和帮助 🤗
Thanks for your reply, I would also like to ask you about how to specify the model of the camera in SFM？
We use pinhole camera model ^_^
感谢您在百忙之中回复我，我今天尝试了一下您的代码进行稀疏重建，在colmap的gui中，您的匹配效果确实完美，之前在colmap中没有匹配上的图像在您的工程中做到了非常完美的效果，但是这里我有一些困惑和问题。第一代码运行时间确实比SIFT长，大概76张图像用了四十分钟，我的配置是RTX4090八张显卡；第二关于您下午说到相机模型用的是针孔模型，但是我尝试找了一下午并没有发现您的代码里是如何设置相加模型的，我想请您能给我指示一下具体需要从哪里入手修改。
最后衷心感谢您和您的团队的工作，我现在正在尝试应用到下游一些任务中去。
- 目前 gim 仓库中的三维重建默认使用的是
$GIM_{DKM}$ 的模型, 它确实是速度慢但是性能好, 所以时间会比 SIFT 要更久. 根据我们的测试结果, 目前 gim 公布的模型性能降序排序是,$GIM_{DKM} > GIM_{LoFTR} > GIM_{LightGlue}$ . 它们的速度快慢排序则是,$GIM_{LightGlue} > GIM_{LoFTR} > GIM_{DKM}$ ,$GIM_{LightGlue}$ 最快. 你可以以此作为参考进行模型的选择. 不过, 目前 gim 只公布了$GIM_{DKM}$ 的重建代码, 如果你有需要, 我可以后面有空的时候公布$GIM_{LightGlue}$ 和$GIM_{LoFTR}$ 的重建代码. 到时候你可以试一下速度更快的$GIM_{LightGlue}$ 和$GIM_{LoFTR}$ 能否满足你的重建需求. - 你提到用了 76 张图片进行重建, 目前重建代码默认是对图片两两匹配进行的重建. 如果你的相机运动不是闭环的 (比如你的相机在车子上, 车子一直在路上往前开没有回头), 那其实没必要所有的图片两两匹配. 你可以选择有重叠的图片进行匹配就行.
- 目前 gim 的图像匹配部分的代码只会用 1 张显卡, 所以想要充分利用 8 张显卡就需要对图像匹配这部分的代码进行修改. 不过, 后面的密集重建步骤好像会使用 8 张显卡.
- 设置相机模型为针孔相机应该是在如下的某个地方. 因为我这里利用了 hloc 的重建框架, 我记得它用的是默认相机模型, 而默认相机模型就是针孔相机.
- Currently, the 3D reconstruction in the gim repository by default uses the model of
$GIM_{DKM}$ . It is indeed slow but has better performance. Therefore, it will take longer than SIFT. According to our test results, the currently published performance ranking of gim models in descending order is$GIM_{DKM} > GIM_{LoFTR} > GIM_{LightGlue}$ . The speed ranking of them is$GIM_{LightGlue} > GIM_{LoFTR} > GIM_{DKM}$ , and$GIM_{LightGlue}$ is the fastest. You can use this as a reference for model selection. However, currently gim only release the reconstruction code of$GIM_{DKM}$ . If you need it, I can release the reconstruction codes of$GIM_{LightGlue}$ and$GIM_{LoFTR}$ when I have free time later. At that time, you can try whether the faster$GIM_{LightGlue}$ and$GIM_{LoFTR}$ can meet your reconstruction needs. - You mentioned using 76 images for reconstruction. Currently, the reconstruction code by default reconstructs by pairwise matching of images. If your camera movement is not closed-loop (for example, your camera is on a car and the car is constantly moving forward on the road without turning back), then it is actually not necessary to pairwise match all images. You can choose images with overlap for matching.
- Currently, the code for the image matching part of gim will only use 1 GPU. Therefore, if you want to fully utilize 8 GPUs, you need to modify the code for this part of image matching. However, it seems that the dense reconstruction step will use 8 GPUs.
- Setting the camera model to a pinhole camera should be somewhere as follows. Because I use the reconstruction framework of hloc here, I remember it uses the default camera model, and the default camera model is a pinhole camera.
Line 28 in e06dba6
Line 98 in e06dba6
Line 139 in e06dba6
感谢您的回复，不过默认相机模型不是SIMPLE_PINHOLE或者PINHOLE而是采用和colmap一样的SIMPLE_RADIAL，我在作者的回答中找到了一些修改相机模型的方法https://github.com/cvg/Hierarchical-Localization/issues/278，但是还没有进行尝试。我希望您能告诉我在哪里可以修改匹配的代码使得能够用上更多的GPU来加速匹配？
这涉及到对 hloc 框架的改动了, 目前 hloc 是只能用单卡一对一对图像的读进去进行匹配的. 很难用一两句话说清楚, 不过, 理解 hloc 的框架也不是很难, 你可以去试一下 👍
感谢您的回复，最近我仔细阅读了hloc的代码，需要在reconstruction代码中设定相机模型，否则就是默认模型，希望您能百忙之中发布速度更快的匹配方法，目前这种匹配方法虽然稳健且有效，但是应用于下游的一些任务耗时太长，另外我也需要仔细看看您提到的多卡进行匹配，希望能够加速匹配的效率
我有时间就会去尽快发布的, 最近真的忙, 见谅哈, 兄弟.
辛苦了，这块我自己先学习一下 @xuelunshen
@xuelunshen 您训练的数据、代码能发布吗，谢谢了
@xuelunshen 您训练的数据、代码能发布吗，谢谢了
最近真的很忙, 得等一下哈, 肯定会公布的