Slower Training Than Ultralytics

[davidhuangal]

I have noticed that training is significantly slower with MMYOLO as opposed to Ultralytics using the same parameters and environment. I.e., using the same set of GPUs with the same batch size, both using AMP, both using distributed training, etc.

By significantly, I mean in the range of 3x-4x. Has anyone else run into this issue or figured out how to fix it? I have even tried using the cached mosaic augmentation and even simply removing the mosaic augmentation as the FAQ mentioned this could be a bottleneck and saw no significant increase in training speed.