Training on Custom data
Opened this issue · 6 comments
Hello, how can I train on a custom video data using your framework?
Hi @MultiTrickFox ,
Of course you can! To do it using the code here you'll need to create your own dataset class.
As an example, https://github.com/sarafridov/K-Planes/blob/main/plenoxels/datasets/video_datasets.py is the one we used for DNeRF and DyNeRF datasets.
An alternative is to use the kplanes integration into nerfstudio. With that code, you can use the nerfstudio support for custom datasets, depending on the format of your data this could be easy or not. Also have a look at these other docs for nerfstudio + kplanes
Hi @MultiTrickFox , Of course you can! To do it using the code here you'll need to create your own dataset class. As an example, https://github.com/sarafridov/K-Planes/blob/main/plenoxels/datasets/video_datasets.py is the one we used for DNeRF and DyNeRF datasets.
An alternative is to use the kplanes integration into nerfstudio. With that code, you can use the nerfstudio support for custom datasets, depending on the format of your data this could be easy or not. Also have a look at these other docs for nerfstudio + kplanes
Hi @MultiTrickFox , Of course you can! To do it using the code here you'll need to create your own dataset class. As an example, https://github.com/sarafridov/K-Planes/blob/main/plenoxels/datasets/video_datasets.py is the one we used for DNeRF and DyNeRF datasets.
An alternative is to use the kplanes integration into nerfstudio. With that code, you can use the nerfstudio support for custom datasets, depending on the format of your data this could be easy or not. Also have a look at these other docs for nerfstudio + kplanes
Thank you for your contribution. I have also tried the addon（kplanes-nerfstudio）, but I can only train on existing datasets. I'm not sure how to process the my own data（dynamic real world video）. The data processing methodsand tools in NERF Studio seem to be only focused on static scenes, and I haven't seen any specific parameters for handling dynamic scenes. Can you provide me with more information?
Hi @liuxin2322!
Actually, in nerfplayer there are some dynamic datasets, like Nuscenes. Some models there can also deal with it, you just need to process time additionally. However, the kplanes integration to nerfstudio seems to be buggy ATM=(.
Hi @liuxin2322! Actually, in nerfplayer there are some dynamic datasets, like Nuscenes. Some models there can also deal with it, you just need to process time additionally. However, the kplanes integration to nerfstudio seems to be buggy ATM=(.
YEP! I have noticed the code like Nuscenes yesterday. I plan to process the video and generate mask , then use colmap to calculate camera pose . ANYWAY, Thanks for your reply
@Kane-LAU Have you successfully run the code on your own custom videos? I am really curious about how to achieve that. I wonder if you could share the code.
Thanks
can you explain how can i generate the bds.npy file if i'm aiming to create my own dataset