bshall/UniversalVocoding

How long does it takes to train from the scratch?

Closed this issue · 4 comments

Thank you for sharing your great work.
As i have changed many parameters(n_mel, fft, hop, window etc), I am training this model from scratch with VCTK dataset.
Could you tell me the environment you had and how long it took?
I have geforce rtx 2080 ti, and it seems to take whole month :(

Hi @sbkim052,

It'll depend on what the settings you use but it definitely shouldn't take a whole month. First, I'd advise not to use the full sample rate for VCTK. It'll make training a lot faster if you resample to 22050Hz or something lower. I have an rtx 2070 super and can train the model in 6ish hours. Maybe you could post your parameters just to see if things are reasonable?

Also, issue #5 has some discussion around training times so it might be worth checking out.

Thank you @bshall .
I have an additional question.
Why does it take so long to load the checkpoint when executing generate.py?

Sorry about the delay @sbkim052,

How long is does it take for you? It will take a bit of time to move the model to the GPU but it shouldn't be very long.

Hi @bshall
It took about 2minutes or so to load the model.
First I thought it took 2minutes for inferencing, but as you said, the time for moving the model to GPU takes long
and the inferencing time is not that long.