Basic version code for "Attention-Enhanced Cross-modal Localization Between Spherical Images and Point Clouds“
The dataset is built based on KITTI360
Stitch dual-fisheye into spherical image by FFmpeg
ffmpeg -y -i $file -vf v360=dfisheye:e:yaw=-90:ih_fov=187.8:iv_fov=185 -c:v libx265 -b:v 40000k -bufsize 5000k -preset ultrafast -c:a copy out.mp4
ffmpeg -i out.mp4 -f image2 ./$(basename $file .png).png
For global Lidar map making and sub-maps division, refer to this
the ML environment is based on PyTorch 1.7.0
Before running mytrain.py, specific 2D dataset directory path in mytrain.py or when input the command.
--dataset_root_dir refers to the root directory of the 2D dataset
Specify the 3D dataset directory in ./mycode/msls.py
path_to_3d = " "
Train the network
python mytrain.py