How did you preprocess the MatrixCity data?
DerKleineLi opened this issue · 14 comments
Hey, thanks for the great work!
I would like to ask how you get the point cloud for initialization when training on MatrixCity data? Or could you provide the ply file from SfM for the dataset?
Thanks in advance :)
We first train 3D-GS withrandomly initialized point clouds. After that, we extract the position information from ply file as initialization (only xyz).
We will show the initial point cloud used, the train_matrixcity.sh file, and the training results for reference.
Initial point cloud : https://drive.google.com/file/d/1J5sGnKhtOdXpGY0SVt-2D_VmL5qdrIc5/view?usp=sharing
ckpt for MatrixCity : https://drive.google.com/file/d/1SBs7IjxHQghI5rJ7n9vi9umz4--FnSVY/view?usp=sharing
@tongji-rkr , thanks for the great work! I'm wondering how many images you used while training for the MatrixCity. Were you using the officially released images (maybe a subset of it)? Or did you render some new images for testing? I'm trying to replicate your result for MatrixCity, so I can later use my own datasets.
Thank you!
We use aerial images of small city from the Matrixcity dataset, comprising approximately 5000-6000 images. If you aim to achieve better reconstruction results, you can utilize the entire set of images for training.
We use aerial images of small city from the Matrixcity dataset, comprising approximately 5000-6000 images. If you aim to achieve better reconstruction results, you can utilize the entire set of images for training.
Thanks for your quick reply. Im wondering were all images downsampled to 1.6k resolution for training? Could you please kindly share how much cpu RAM and vRAM were used during the training?
Yes, during training, all images are downsampled to a 1.6k resolution, but the original resolution is 1920*1080. If you choose to use the original resolution, I believe it should work fine. The GPU memory usage is approximately 30G. Please note that when training a large-scale scene, you can set data_device=cpu to reduce GPU memory. If the GPU memory is still insufficient, you can downsample the input point cloud.
Also, you can try increasing visible_threshold to reduce GPU memory consumption, but the metric will drop slightly.
@tongji-rkr @jianglh-WHU , thank you both for your quick reply, and thanks again for the great work!
There are different subsets in MatrixCity Dataset. May I ask which subset is this ply file belong to? Thanks!
@TarzanZhao ,the ply belongs to the small city of MatrixCity Dataset.
@jianglh-WHU Thank you!
Are you referring to all images in the 4 sub-folders in this screenshot?
@TarzanZhao, this ply file is used for the aerial images of the small city dataset. If you want to use the street images, I recommend fusing the RGBD images into a point cloud. You can use this script: https://github.com/city-super/MatrixCity/blob/main/scripts/rgbd2pc.py.
@jianglh-WHU Thanks so much! Let me have a look.