InMemoryCache dp will yield from source dp once even when data is fully cached
hirayaku opened this issue · 0 comments
hirayaku commented
🐛 Describe the bug
The InMemoryCacheHolderIterDataPipe will try yielding from source_dp once even when the data from the source is fully cached. This can cause unexpected IO for users. Adding a condition check before enumerating source_dp should fix it.
Versions
Torch==2.0.1, TorchData==0.6.1