Training speed is too slow

Question

Training speed is too slow

Closed this issue 10 months ago · 6 comments

Hello author, how long does it take you to train? Why do I train so slowly once?

Filling replay memory
100%|█████████████████████████████████▊| 24897/25000.0 [00:30<00:00, 820.30it/s]
Running manhattan32
1%|▎ | 27789/3000000 [04:37<81:43:32, 10.10it/s]

This is the GPU I am using:

GeForce RTX 2080 Ti computeCapability: 7.5

Answer 1 · 2022-01-18T00:03:55.000Z

Hello author, how long does it take you to train? Why do I train so slowly once?

Filling replay memory
100%|█████████████████████████████████▊| 24897/25000.0 [00:30<00:00, 820.30it/s]
Running manhattan32
1%|▎ | 27789/3000000 [04:37<81:43:32, 10.10it/s]

This is the GPU I am using:

GeForce RTX 2080 Ti computeCapability: 7.5

Have you solved the problem of slow training speed? I also encountered it. Despite using a GPU, it is still extremely slow ...

Answer 2 · 2022-07-18T09:20:27.000Z

作者你好，你训练需要多长时间？为什么我训练得这么慢？

填充重播内存 100%|█████████████████████████████████|24897/25000.0 [00：30<00：00， 820.30it/s] 运行曼哈顿32 1%|▎ |27789/3000000 [04：37<81：43：32， 10.10it/s]

这是我正在使用的GPU：

GeForce RTX 2080 Ti 计算能力：7.5

Hello, I would like to know how you started training? When I follow the author's 'How to use?' Start training and receive an error.

Answer 3 · 2022-07-20T03:58:02.000Z

我也遇到了同样的问题，并且在--gpu时调用了两块GPU，但是我用nvidia-smi查看的时候GPU使用率很低，严重怀疑是不是没有调用GPU？训练速度超级慢。有没有同学遇到相同的问题？或者顺利调用两块GPU跑成功的？

Answer 4 · 2022-10-19T03:23:52.000Z

Have you solved this problem, please? About solving the problem of too slow training.（请问您解决了这个问题吗？关于解决训练速度过慢的问题。）

Answer 5 · 2022-10-19T03:24:13.000Z

Have you solved this problem, please? About solving the problem of too slow training.（请问您解决了这个问题吗？关于解决训练速度过慢的问题。）

Have you solved this problem, please? About solving the problem of too slow training.（请问您解决了这个问题吗？关于解决训练速度过慢的问题。）

Answer 6 · 2022-10-19T03:24:39.000Z

Hello author, how long does it take you to train? Why do I train so slowly once?

Filling replay memory
100%|█████████████████████████████████▊| 24897/25000.0 [00:30<00:00, 820.30it/s]
Running manhattan32
1%|▎ | 27789/3000000 [04:37<81:43:32, 10.10it/s]

This is the GPU I am using:

GeForce RTX 2080 Ti computeCapability: 7.5

Have you solved this problem, please? About solving the problem of too slow training.（请问您解决了这个问题吗？关于解决训练速度过慢的问题。）