Running time

Question

ChengChen2020 opened this issue 3 years ago · 5 comments

Hi,

Could you please share that how long it takes to run on Area 5 for 100 epochs in your settings?
We need around 7 days which is slow.

Thank you!

Answer 1 · 2021-12-06T12:40:03.000Z

As a reference, you can take the training log shared by a user on one of the closed issues.

With 4 GPUs you see the full training took around 16 hours, and the evaluation took around 40 minutes

Answer 2 · 2021-12-07T07:32:46.000Z

Could you share the hardware (GPU) version you used for training around 16 hours? Thanks!

Answer 3 · 2021-12-07T08:38:19.000Z

@aaron-h-code This is not my run, it was shared in this issue saying that the code was run on 4 RTX 3090 with the default config in this repo

Answer 4 · 2021-12-07T13:02:08.000Z

@aaron-h-code This is not my run, it was shared in this issue saying that the code was run on 4 RTX 3090 with the default config in this repo

@QuanticDisaster OK, 4 RTX 3090...

Answer 5 · 2022-02-22T22:50:53.000Z

RTX 8000 * 4 GPUS for 20 hours