custom dataset

[dyccyber]

when i try to rebuild my own dataset, firstly i use python convert.py -s data to get colmap data
and i use python imgs2poses.py data to get poses_bounds.npy
then i use bash scripts/run_llff.sh to train but the results are worse than 3dgs
the depth maps are good

any advice?

[Fictionarry]

Hi, if original 3DGS can work well, it shows that the point cloud initialization is good enough, and thus you can cancel the --rand_pcd in the script to use the same SfM point cloud initialization. In this situation, I consider the position learning rate can be smaller, e.g. you can try to adjust it to --position_lr_init 0.0016 --position_lr_final 0.000016 as the first try.

You can also try this, which brings better quality if the initialization is dense enough.

Additionally, scripts/run_llff.sh uses only 3 sampled training views by default, but not all the images. Ensure that this is according to your practical requirement, as I'm not sure if the SfM point cloud used for 3DGS initialization in your dataset is also built from the same 3 training views.

[dyccyber]

thanks for your reply ，but when i cancel the --rand_pcd，the training iteration exactly is 2000，the cuda crash：
Training progress: 7%|######2 | 2000/30000 [00:52<13:30, 34.55it/s, Loss=0.1288423]Traceback (most recent call last):
File "train_llff.py", line 400, in
training(lp.extract(args), op.extract(args), pp.extract(args), args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from, args.near)
File "train_llff.py", line 121, in training
render_pkg = render_for_opa(viewpoint_cam, gaussians, pipe, background)
File "/root/autodl-tmp/DNGaussian/gaussian_renderer/init.py", line 258, in render_for_opa
cov3D_precomp=cov3D_precomp,
File "/root/miniconda3/envs/dngaussian/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "/root/miniconda3/envs/dngaussian/lib/python3.7/site-packages/diff_gaussian_rasterization/init.py", line 222, in forward
raster_settings,
File "/root/miniconda3/envs/dngaussian/lib/python3.7/site-packages/diff_gaussian_rasterization/init.py", line 41, in rasterize_gaussians
raster_settings,
File "/root/miniconda3/envs/dngaussian/lib/python3.7/site-packages/diff_gaussian_rasterization/init.py", line 92, in forward
num_rendered, color, depth, alpha, radii, geomBuffer, binningBuffer, imgBuffer = _C.rasterize_gaussians(*args)
RuntimeError: CUDA error: invalid configuration argument
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
when i add the --rand_pcd and everything is normal，have you ever encountered this situation?

[Fictionarry]

when i add the --rand_pcd and everything is normal，have you ever encountered this situation?

It seems like a CUDA memory error. I have just tried to remove --rand_pcd on the LLFF fern scene twice time, where I separately 1) use the SfM point cloud from all views as initialization, and 2) use only 3 training views to run the SfM process. The code works with no problem raised in both of these two situations. You may also try to run it on "fern" to check whether this problem is caused by the environment or the dataset. And some test points can be added like --test_iterations 1000 1500 1900 to check if the reconstruction runs well in the iterations before 2000.