How to see complete rendered image in SIBR Viewer?
yt0409 opened this issue · 25 comments
Great job! It's so cool that it makes me want to kneel down on both knees.I have encountered a problem，I successfully run SIBR remote viewer, but only see some colored points. How can I see the complete rendered image, like 3DGS? I just download the graden dataset and run SIBR with
SIBR_remoteGaussian_app.exe -s D:\Desktop\data\octreegs\garden\garden\data -m D:\Desktop\data\octreegs\garden\garden\ckpt
Check if you are using the `Splats` mode.thank you very much,i find the reason is that i use SIBR_remoteGaussian_app.exe(no splat mode) instead of SIBR_gaussianViewer_app.exe. hope you get many many paper~!!
@tongji-rkr ， i have the same question, how to do ?
i use :bash single_train.sh with default args,
then my data path is:
and do scene="boli" in single_train.sh
my output dir is:
outputs/data/boli/baseline
my vierwe cmd is:
.\viewer\bin\SIBR_remoteGaussian_app.exe -s data/boli -m outputs/data/boli
Please use SIBR_gaussianViewer_app.exe!!!
@tongji-rkr , i see the exe need 5 args
i use :
.\viewer\bin\SIBR_gaussianViewer_app.exe -s data/boli -path data/boli -m outputs/data/boli/baseline --iteration 40000 -model-path outputs/data/boli/baseline
but error:
[SIBR] -- INFOS --: Initialization of GLFW
[SIBR] -- INFOS --: OpenGL Version: 4.6.0 NVIDIA 551.86[major: 4, minor: 6]
[SIBR] -- INFOS --: Dataset type:
Number of input Images to read: 107
Number of Cameras set up: 107
LOADSFM: Try to open data/boli/sparse/0/points3D.bin
Num 3D pts 106479
[SIBR] -- INFOS --: SfM Mesh 'data/boli/sparse/0/points3d.bin successfully loaded. (106479) vertices detected. Init GL ...
[SIBR] -- INFOS --: Init GL mesh complete
[SIBR] -- INFOS --: opacity_mlp : 0
[SIBR] -- INFOS --: cov_mlp : 0
[SIBR] -- INFOS --: color_mlp : 0
[SIBR] -- INFOS --: embedding_appearance : 0
[SIBR] -- INFOS --: opacity_mlp : 0
[SIBR] -- INFOS --: cov_mlp : 0
[SIBR] -- INFOS --: color_mlp : 0
This means the locations of the mlps are wrong.
@tongji-rkr
i have do the output fold style,
still wrong, what is your cmd command?
.\viewer\bin\SIBR_gaussianViewer_app.exe -m model_path -s data_path
You can download this(https://drive.google.com/file/d/1Jy-TKGjEG3yDggD4RF5oSfmOvcqsQRhW/view?usp=sharing) as a reference.
@tongji-rkr
i have tested the cmd you gived before, still wrong, now i will use the data in google you support to test it, thank you for your help !
Waitting result......
@tongji-rkr
it is sucessful, it looks good.
thank you for your help!
next, if i want to cast the result point_cloud file to mesh file , how to do ? or use which software or project ?
@tongji-rkr
#8 (comment)
do you see project SuGar, it looks like can get mesh from ply.
do yoou have a plan to add it ?
Hi~@luoww1992, Octree-GS is not specifically designed for surface reconstruction. If you're interested in that topic, we recommend referring to our another work: GSDF.
@tongji-rkr ,
i have got a mon-video by phone, then i find that:
args auto get:
levels=7
init_level=3
but the iterations is 40000 still,
so,
1, the init_level meanings,
2, the iterations should be (levels+1)*10000, is it better than 40000 ?
-
i see the code after training:
if args.warmup:
logger.info("\n Warmup finished! Reboot from last checkpoints")
new_ply_path = os.path.join(args.model_path, f'point_cloud/iteration_{args.iterations}', 'point_cloud.ply')
training(lp.extract(args), op.extract(args), pp.extract(args), dataset, args.test_iterations, args.save_iterations, args.checkpoint_iterations, args.start_checkpoint, args.debug_from, wandb=wandb, logger=logger, ply_path=new_ply_path)
what is the meaning of warmup ? refine?
i am triying to test different args by this video, can you give me some advices ?
Firstly, init_level means that at the beginning of training, we allow 0~init_level anchors to be trained. In the first 10,000 .iterations, we gradually open anchors in finer LOD levels, and in the next 30,000 rounds, we use all anchors for training to fit.
Therefore, I think (levels+1)*10000 is unreasonable, which should be due to your misunderstanding of our training strategy.
Finally, warmup is the setting in Scaffing-GS. In the case of no point cloud, we first randomly initialize for training, and then use the first training result as the initialization of the second training.
@tongji-rkr
current result:
basedefault: phone video about house on street, all args are default (levels=7,init_level=3, iterations= 40000)
test1: change args: change iterations= 80000, now training is 55000+, the result viewer is better much than basedefault.
waitting update....
Yes, in some larger scenes or more complex scenes, increasing iterations will improve the reconstruction effect, but in the public dataset, 40000 iterations is enough for fitting.
@tongji-rkr
in README.MD, maybe you should add a description about how to improve the effect of the scene by change some main args for most scenes, like 360 object , outdoor, street, house....
Thank you for your advice. In fact, in the .sh file, we have given the parameter Settings suitable for most scenarios. If the effect is not as expected in a certain scenario, I will solve the corresponding problem in a new issue.
@tongji-rkr
i know that in .sh file, what i mean is there are too many existing parameters, but too little description of the parameters, and what effect does the parameter change have on the reconstruction, e.g. the iterations parameter becomes larger will improve the reconstruction effect....
then add it in README, we can change it by dataset ourself
I understand what you mean. We will sort out the experimental results, weight some key parameters and explain the effect.
@tongji-rkr
ok, i know, my test is running, I will continue to update the previous answers, I hope the test result will be helpful to you。
thank you for your help !
thank you very much,i find the reason is that i use SIBR_remoteGaussian_app.exe(no splat mode) instead of SIBR_gaussianViewer_app.exe. hope you get many many paper~!!非常感谢，我发现原因是我使用SIBR_remoteGaussian_app. exe（无splat模式）而不是SIBR_gaussianViewer_app. exe。希望你能得到很多很多的纸～！！
Hi, do u have the problem that there is nothing after running the exe fie?