webdataset

High Performance I/O for Large Scale Deep Learning

Pinned Repositories

tarp
Fast and simple stream processing of files in tar files, useful for deep learning, big data, and many other applications.
Language:Go119 5 1115
wdsconversions
conversions of standard pytorch datasets to webdataset format
Language:Jupyter Notebook1 0 01
webdataset
A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.
Language:Python2.2k 22 317170
WebDataset.jl
A high performance I/O library for deep learning in Julia, based on the PyTorch WebDataset library
Language:Julia10 3 01

webdataset/webdataset
A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.
Language:Python2.2k 22 317170
webdataset/tarp
Fast and simple stream processing of files in tar files, useful for deep learning, big data, and many other applications.
Language:Go119 5 1115
webdataset/WebDataset.jl
A high performance I/O library for deep learning in Julia, based on the PyTorch WebDataset library
Language:Julia10 3 01
webdataset/wdsconversions
conversions of standard pytorch datasets to webdataset format
Language:Jupyter Notebook1 0 01