ashawkey/RAD-NeRF

Add training to colab

gloomiebloomie opened this issue · 7 comments

gloomiebloomie commented

Hi I'm currently receiving a lot of errors when trying to train my own model with my own video. Could you guys add training into the colab so there is a less complicated guide that it more likely to run?

gloomiebloomie commented

I've mostly made one now if anyone wants it reach out I just didn't finish running it due to runtime on google colab free version. I also have started running it on windows.

Eskec14 commented

Hi could you post or send the working colab version? Im having the same troubles as you, so it would be of great help.

gloomiebloomie commented

I made a repo with the colab, it is just the demo with added training command. You still have to create a directory in data with your name and upload a 512x512 video there. Make sure to add .mat file and other files needed in 3DMM directory. Make sure you are in RAD-NeRF dir before running training or processing script. If you are using free colab it will take multiple runtimes to get from process.py to first to 3rd training seconds. Just rerun when out of run time installing requirements again but mounting drive helps keep 3DMM files intact and the uploaded video working better.

  • 😄1
Eskec14 commented

Would you mind sharing where you got the files required to be in the 3DMM folder?

gloomiebloomie commented

it's in the readme prepare basel face model

  1. download 01_MorphableModel.mat from https://faces.dmi.unibas.ch/bfm/main.php?nav=1-2&id=downloads and put it under data_utils/face_tracking/3DMM/
  • 😄1
Eskec14 commented

Thank you, I missed that part.

ashutoshsinha25 commented

getting this error when running the covert_BFF.py file,

/content/RAD-NeRF/data_utils/face_tracking
Traceback (most recent call last):
File "/content/RAD-NeRF/data_utils/face_tracking/convert_BFM.py", line 4, in
original_BFM = loadmat("3DMM/01_MorphableModel.mat")
File "/usr/local/lib/python3.10/dist-packages/scipy/io/matlab/_mio.py", line 227, in loadmat
matfile_dict = MR.get_variables(variable_names)
File "/usr/local/lib/python3.10/dist-packages/scipy/io/matlab/_mio5.py", line 330, in get_variables
res = self.read_var_array(hdr, process)
File "/usr/local/lib/python3.10/dist-packages/scipy/io/matlab/_mio5.py", line 290, in read_var_array
return self._matrix_reader.array_from_header(header, process)
File "_mio5_utils.pyx", line 665, in scipy.io.matlab._mio5_utils.VarReader5.array_from_header
File "_mio5_utils.pyx", line 694, in scipy.io.matlab._mio5_utils.VarReader5.array_from_header
File "_mio5_utils.pyx", line 768, in scipy.io.matlab._mio5_utils.VarReader5.read_real_complex
File "_mio5_utils.pyx", line 445, in scipy.io.matlab._mio5_utils.VarReader5.read_numeric
File "_mio5_utils.pyx", line 350, in scipy.io.matlab._mio5_utils.VarReader5.read_element
File "_streams.pyx", line 171, in scipy.io.matlab._streams.ZlibInputStream.read_string
File "_streams.pyx", line 164, in scipy.io.matlab._streams.ZlibInputStream.read_into
OSError: could not read bytes

.mat file is present under 3DMM/ .Im trying to run the training process in colab (free version). Any suggests ? Im running this file with this code :

%cd data_utils/face_tracking/
!python convert_BFM.py

@gloomiebloomie