wanmeihuali/taichi_3d_gaussian_splatting

Custom datasets

henrypearce4D opened this issue · 3 comments

henrypearce4D commented

@wanmeihuali Hi this is really cool, very impressive!
I have tested with the datasets you provided and got everything working.
Next I would like to test with my own data of human scans and I'm having trouble following how to convert my own data into the right format. Could you go into more detail on this please or could we get into contact to discuss further and other ideas?

Best
Henry

wanmeihuali commented

Hi Henry, please check this file for how to prepare your own data: https://github.com/wanmeihuali/taichi_3d_gaussian_splatting/blob/main/docs/RawDataFormat.md
basically what you need is two JSON files, train.json, and val.json, also a parquet file for input pointcloud. You can email me by kuangyuansun@gmail.com if you want to discuss some detail or further ideas.

henrypearce4D commented

Thanks!

I managed to modify the tool for blender by providing an stl and transforms.

after some time I run into this issue

 13%|█████████▎                                                              | 3900/30001 [2:14:41<15:01:28,  2.07s/it]
Traceback (most recent call last):
  File "C:\Users\INFINITE\taichi_3d_gaussian_splatting\gaussian_point_train.py", line 20, in <module>
    trainer.train()
  File "C:\Users\INFINITE\taichi_3d_gaussian_splatting\taichi_3d_gaussian_splatting\GaussianPointTrainer.py", line 188, in train
    self._plot_grad_histogram(
  File "C:\Users\INFINITE\taichi_3d_gaussian_splatting\taichi_3d_gaussian_splatting\GaussianPointTrainer.py", line 300, in _plot_grad_histogram
    writer.add_histogram("grad/xyz_grad", xyz_grad, iteration)
  File "C:\Users\INFINITE\.conda\envs\taichi_3d_gaussian_splatting\lib\site-packages\torch\utils\tensorboard\writer.py", line 485, in add_histogram
    histogram(tag, values, bins, max_bins=max_bins), global_step, walltime
  File "C:\Users\INFINITE\.conda\envs\taichi_3d_gaussian_splatting\lib\site-packages\torch\utils\tensorboard\summary.py", line 355, in histogram
    hist = make_histogram(values.astype(float), bins, max_bins)
  File "C:\Users\INFINITE\.conda\envs\taichi_3d_gaussian_splatting\lib\site-packages\torch\utils\tensorboard\summary.py", line 362, in make_histogram
    raise ValueError("The input has no element.")
ValueError: The input has no element.

Any idea what this means?

wanmeihuali commented

@henrypearce4D . It seems like a bug that is triggered when no points are projected in the camera plane. I do suggest you check if your camera pose is correct, a good method is to check the tensorboard for the early images.
I'll try to fix the crash caused by no points in the camera plane later, but no point in the camera usually means the camera pose is wrong(or may be the blender plugin selects some strange poses that is not facing the object...)
Anyway, if the image rendered looks good, maybe it is some other bug, the _plot_grad_histogram function is only intent for debugging, so you can safely comment it out.