How to import a point cloud into the HLOC workflow to be used as a reference for localization?

[messileonardo]

Hi there, I would like to scale the sfm point cloud (generated by COLMAP) externally and import it into the HLOC workflow to use it as a reference for localization. To do that, I guess that the variable that you call "model", that is:

"model = reconstruction.main(sfm_dir, images_mapping, sfm_pairs, features, matches, image_list=references)",

must be re-generated from the re-imported point cloud.
So my question is: how can I import a point cloud into the HLOC workflow and use it as a reference for localization?

Looking at the documentation, I found that a function called "import_PLY" exist.

"import_PLY(...)
import_PLY(self: pycolmap.Reconstruction, arg0: str) -> None
Import from PLY format. Note that these import functions are only intended for visualization of data and usable for reconstruction."

Can it be used to import a point cloud into the HLOC workflow in order to use that point cloud as a reference for localization?

Many thanks in advance!

[messileonardo]

Hi there, I managed the importing phase with the following line of code:
pycolmap.Reconstruction.import_PLY(model, str(import_ply_path))
I get no errors in this step, but I am not sure it is correct.
Assuming it is correct, what I am trying to do now (with no success) is using the externally-scaled point cloud in the Hloc workflow. In particular, I would like to carry out localization with reference to the externally-scaled point cloud. I guess that the "model" variable must be recomputed in some way.
The error I get in the localization step is the following:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[20], line 24
     19 conf = {
     20     'estimation': {'ransac': {'max_error': 12}}, # reprojection error in pixel https://github.com/colmap/pycolmap/blob/ea42af8726de2cb49ff9b189000d7ef0408e8ea2/README.md?plain=1#L160
     21     'refinement': {'refine_focal_length': True, 'refine_extra_params': True},
     22 }
     23 localizer = QueryLocalizer(model, conf)
---> 24 ret, log = hloc_pose_from_cluster(localizer, query, camera, ref_ids, features, matches)
     25 print(f'found {ret["num_inliers"]}/{len(ret["inliers"])} inlier correspondences.')
     26 # visualization.visualize_loc_from_log(images, query, log, model, top_k_db=3)

File ~/services/jupyterlab/Hierarchical-Localization-v2/Hierarchical-Localization/hloc/localize_sfm.py:124, in hloc_pose_from_cluster(localizer, qname, query_camera, db_ids, features_path, matches_path, **kwargs)
    122 mkp_idxs = [i for i in idxs for _ in kp_idx_to_3D[i]]
    123 mp3d_ids = [j for i in idxs for j in kp_idx_to_3D[i]]
--> 124 ret = localizer.localize(kpq, mkp_idxs, mp3d_ids, query_camera, **kwargs)
    125 ret['camera'] = {
    126     'model': query_camera.model_name,
    127     'width': query_camera.width,
    128     'height': query_camera.height,
    129     'params': query_camera.params,
    130 }
    133 # mostly for logging and post-processing

File ~/services/jupyterlab/Hierarchical-Localization-v2/Hierarchical-Localization/hloc/localize_sfm.py:58, in QueryLocalizer.localize(self, points2D_all, points2D_idxs, points3D_id, query_camera)
     56 def localize(self, points2D_all, points2D_idxs, points3D_id, query_camera):
     57     points2D = points2D_all[points2D_idxs]
---> 58     points3D = [self.reconstruction.points3D[j].xyz for j in points3D_id]
     59     ret = pycolmap.absolute_pose_estimation(
     60         points2D, points3D, query_camera,
     61         estimation_options=self.config.get('estimation', {}),
     62         refinement_options=self.config.get('refinement', {}),
     63     )
     64     return ret

File ~/services/jupyterlab/Hierarchical-Localization-v2/Hierarchical-Localization/hloc/localize_sfm.py:58, in <listcomp>(.0)
     56 def localize(self, points2D_all, points2D_idxs, points3D_id, query_camera):
     57     points2D = points2D_all[points2D_idxs]
---> 58     points3D = [self.reconstruction.points3D[j].xyz for j in points3D_id]
     59     ret = pycolmap.absolute_pose_estimation(
     60         points2D, points3D, query_camera,
     61         estimation_options=self.config.get('estimation', {}),
     62         refinement_options=self.config.get('refinement', {}),
     63     )
     64     return ret

KeyError: ''

Any suggestions?
Many thanks!

[messileonardo]

Some more details:

• I scaled the ply point cloud in cloud compare
• I reimported the point cloud in the same ply format