RuntimeError: Sizes of tensors must match except in dimension 1
Opened this issue · 9 comments
Hi @LightwheelAI ,
I tried to run your awesome project on my 3090 computer, but got some error while training
- I also have some problem in data processing, so I use the preprocessed data
2094681306939952000that you provided - I modified
frame_selectas indicated in the README
- I use points3D2.bin as init points
when I ran train.sh, I got error below:
File "/home/sil/miniconda3/envs/street-gaussians-ns/bin/sgn-train", line 8, in <module>
sys.exit(entrypoint())
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 262, in entrypoint
main(
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 247, in main
launch(
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 189, in launch
main_func(local_rank=0, world_size=world_size, config=config)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 100, in train_loop
trainer.train()
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/engine/trainer.py", line 298, in train
self.eval_iteration(step)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/utils/decorators.py", line 70, in wrapper
ret = func(self, *args, **kwargs)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/utils/profiler.py", line 112, in inner
out = func(*args, **kwargs)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/engine/trainer.py", line 536, in eval_iteration
_, eval_loss_dict, eval_metrics_dict = self.pipeline.get_eval_loss_dict(step=step)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/utils/profiler.py", line 112, in inner
out = func(*args, **kwargs)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/pipelines/base_pipeline.py", line 324, in get_eval_loss_dict
model_outputs = self.model(ray_bundle)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/models/base_model.py", line 143, in forward
return self.get_outputs(ray_bundle)
File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto_scene_graph.py", line 363, in get_outputs
out = super().get_outputs(camera)
File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto.py", line 858, in get_outputs
colors_crop = torch.cat((features_dc_crop, features_rest_crop), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 626493 but got size 809985 for tensor number 1 in the list.
Exception in thread Thread-13:
Traceback (most recent call last):
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/viewer/render_state_machine.py", line 224, in run
outputs = self._render_img(action.camera_state)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/viewer/render_state_machine.py", line 168, in _render_img
outputs = self.viewer.get_model().get_outputs_for_camera(camera, obb_box=obb)
File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto.py", line 1106, in get_outputs_for_camera
outs = self.get_outputs(camera.to(self.device))
File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto_scene_graph.py", line 363, in get_outputs
out = super().get_outputs(camera)
File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto.py", line 858, in get_outputs
colors_crop = torch.cat((features_dc_crop, features_rest_crop), dim=1)
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 626493 but got size 809985 for tensor number 1 in the list.
Should I change any configurations?
Best
I notice that object_hsZGh_tYp9MXN_75m601LA appear two times, which cause an AssertionError from assert model_name not in self.visible_model_names at line 387(street_gaussians_ns/sgn_splatfacto_scene_graph.py)
I use another preprocessed dataset 8398516118967750070, it seems work, and will update when I get model out, thanks~
Hi @LightwheelAI ,
I tried to run your awesome project on my 3090 computer, but got some error while training
- I also have some problem in data processing, so I use the preprocessed data
2094681306939952000that you provided- I modified
frame_selectas indicated in the README
- I use points3D2.bin as init points
when I ran train.sh, I got error below:
File "/home/sil/miniconda3/envs/street-gaussians-ns/bin/sgn-train", line 8, in <module> sys.exit(entrypoint()) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 262, in entrypoint main( File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 247, in main launch( File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 189, in launch main_func(local_rank=0, world_size=world_size, config=config) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/scripts/train.py", line 100, in train_loop trainer.train() File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/engine/trainer.py", line 298, in train self.eval_iteration(step) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/utils/decorators.py", line 70, in wrapper ret = func(self, *args, **kwargs) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/utils/profiler.py", line 112, in inner out = func(*args, **kwargs) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/engine/trainer.py", line 536, in eval_iteration _, eval_loss_dict, eval_metrics_dict = self.pipeline.get_eval_loss_dict(step=step) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/utils/profiler.py", line 112, in inner out = func(*args, **kwargs) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/pipelines/base_pipeline.py", line 324, in get_eval_loss_dict model_outputs = self.model(ray_bundle) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/models/base_model.py", line 143, in forward return self.get_outputs(ray_bundle) File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto_scene_graph.py", line 363, in get_outputs out = super().get_outputs(camera) File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto.py", line 858, in get_outputs colors_crop = torch.cat((features_dc_crop, features_rest_crop), dim=1) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 626493 but got size 809985 for tensor number 1 in the list. Exception in thread Thread-13: Traceback (most recent call last): File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/threading.py", line 932, in _bootstrap_inner self.run() File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/viewer/render_state_machine.py", line 224, in run outputs = self._render_img(action.camera_state) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/nerfstudio/viewer/render_state_machine.py", line 168, in _render_img outputs = self.viewer.get_model().get_outputs_for_camera(camera, obb_box=obb) File "/home/sil/miniconda3/envs/street-gaussians-ns/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto.py", line 1106, in get_outputs_for_camera outs = self.get_outputs(camera.to(self.device)) File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto_scene_graph.py", line 363, in get_outputs out = super().get_outputs(camera) File "/home/sil/simulation_ws/scripts/street-gaussians-ns/street_gaussians_ns/sgn_splatfacto.py", line 858, in get_outputs colors_crop = torch.cat((features_dc_crop, features_rest_crop), dim=1) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 626493 but got size 809985 for tensor number 1 in the list.Should I change any configurations? Best
Hi, thanks for your trying! I think this may be a problem caused by the initial point cloud, if there are SfM floaters in the initial point cloud, there is a possibility that they may block the dynamic object so that its gaussians are censored to zero causing the error to be reported.
best
Hi @LightwheelAI ,
Thanks for your reply, I just reinstall all dependencies then error disappear
I use 2094681306939952000 and results are as below:
background_rgb
https://github.com/LightwheelAI/street-gaussians-ns/assets/43367892/1498f776-4137-4299-8af6-8b73e8d6d39b
object_rgb
https://github.com/LightwheelAI/street-gaussians-ns/assets/43367892/897d5765-695b-4ec1-8f15-849320fabddf
rgb
https://github.com/LightwheelAI/street-gaussians-ns/assets/43367892/f04e04cd-f621-42c0-a138-10c923e0291d
It seems that there is a problem with moving object segmentation, resulting in the reconstructed moving object being blurry
I'd like to know if you have any suggestions for improvement?
Best
Hi, thanks for your trying!
There are three key factors to improve the quality of moving target reconstruction in our practice.
- Remove the floater of the moving target in the SfM point cloud
- Improve the accuracy of the bounding box
- Improve the accuracy of the camera pose
I use another preprocessed dataset
8398516118967750070, it seems work, and will update when I get model out, thanks~
hello, which init point cloud file you used for training sequence 8398516118967750070?
It seems that the problem is caused by viewer of nerfstudio. After turning off nertstudio in explorer when I trained the model, this error disappeared
It seems that the problem is caused by viewer of nerfstudio. After turning off nertstudio in explorer when I trained the model, this error disappeared
Hello, I have encountered the same problem, my computer is Ubuntu system, may I ask you how to operate, to solve this problem
It seems that the problem is caused by viewer of nerfstudio. After turning off nertstudio in explorer when I trained the model, this error disappeared
Hello, I have encountered the same problem, my computer is Ubuntu system, may I ask you how to operate, to solve this problem
Yeah, I met the error when I tried to open nerfstudio viewer and move around the scene while training. So I just run the train.sh without nerfstudio viewer or any operation, then it worked well.
Regarding other cases in the issue, it can also be caused by some dependencies, especially some packages about nerfstudio. For example, you can try gsplat 0.1.11 and nerfstudio 1.1.0 as #31 mentioned. Here is my environment, hope it helps
Ubuntu 20.04 with RTX 3090
requirements.txt