Issue with shuffle=True, when using with pytorch dataloader

Question

anjith2006 opened this issue 4 years ago · 2 comments

Hi,

Thank you for the very useful repo.

ValueError: DataLoader with IterableDataset: expected unspecified shuffle option, but got shuffle=True

is there a way to use shuffling with the interface. Also is there a way to preceompute
the length of the dataset.

Thanks

Answer 1 · 2021-04-17T23:36:59.000Z

TFRecordDataset and MultiTFRecordDataset both shuffle the data internally. Provide a "shuffle_queue_size" to the dataset.

Answer 2 · 2021-04-18T00:11:09.000Z

I added an example in the Readme.md