open-mmlab/mmengine

[Feature] Support dataset streaming?

Opened this issue · 0 comments

What is the feature?

Hi, I would like to know whether mmengine dataset and dataloader support dataset streaming.
I want to load a very large amount of data to continue pre-train.

Any other context?

No response