yiqun-wang/HFS

Training time and TQDM

Opened this issue · 1 comments

Hi,

Before all, thank you for your inspiring work !
I'm trying to process some custom data (that I preprocessed with COLMAP as indicated in the original NeuS repository) and compare the result with plain NeuS but I feel like the training process takes much more time than it should be (actually ~5*training time of plain NeuS).
I have strange outputs from TQDM in the logs. It seems that it cannot estimate the true duration of an iteration. I get 7 digits it/s which is obviously wrong and consequently the remaining time metric is wrong as well (0 s. remaining).
Obviously training an additional MLP for the second SDF should make it slower to train but I didn't expect such drawback...

I was wondering if it was 'normal' as in you're aware of the situation or should I consider this as unusual and investigate my data ?

Thank you in advance!

I just provided pre-trained models for better inference. For custom data, not sure if it is caused by the hardware environment, moreover it is hard to judge the impact of the size of the custom data on time efficiency.