About the training time

Question

About the training time

liuyang-ict opened this issue 3 years ago · 5 comments

liuyang-ict commented 3 years ago

How much V100 GPU-time when TubeDETR was trained on HC-STVG?

Answer 1 · 2022-04-13T11:25:02.000Z

20 epochs on HC-STVGv2 took about 7 hours on 16 V100 for the best configuration (stride 4 resolution 352).

Answer 2 · 2022-04-13T12:41:42.000Z

Thanks a lot~~

Answer 3 · 2022-06-19T15:13:39.000Z

If fewer GPUs are used for training, like 2~8 V100, will the accuracy decrease?
By the way, are your V100 cards all 32GB? Thanks in advance.

Answer 4 · 2022-06-19T15:19:41.000Z

By the way again, I think the preproc_hstvgv2.py in preproc should be preproc_hcstvgv2.py? :)

Answer 5 · 2022-06-19T19:21:18.000Z

I haven't tried training with a lower number of V100 but I don't think this would decrease the results. All the V100 used in experiments are 32GB, but you can use 16GB V100 with a lower resolution and/or higher temporal stride. The name of the preproc_hstvgv2.py file indeed had a typo, thanks for catching this!