pytorch/data

Issues

• MacOS state_dict tests in CI are failing during shutdown

  #1256 opened a month ago by andrewkho
  2

• torchdata not compatible with torch 2.3.0

  #1244 opened a month ago by mmdanziger
  1

• [StatefulDataLoader] macOS tests are too slow

  #1247 opened a month ago by andrewkho
  0

• Move to removesuffix string method after python 3.8 support is dropped

  #1240 opened 2 months ago by gokulavasan
  0

• DataLoader2 with multiprocess raise exception: Can not request next item while we are still waiting response for previous request

  #1228 opened 3 months ago by npuichigo
  1

• DataLoader2 Memory Behavior is very strange on Epoch Resets

  #1185 opened a year ago by andrew-bydlon
  9

• Future of torchdata and dataloading

  #1196 opened a year ago by laurencer
  28

• Make archive datapipes faster

  #1191 opened 9 months ago by lei-rs
  1

• Dataloader is slow with iterdatapipes and shuffle that has large in-memory fields (because traverse_dps is slow)

  #1226 opened 4 months ago by olegsinavski
  3

• PyTorch 2.2: import torchdata fails on ubuntu-20.04 github runners

  #1225 opened 4 months ago by agunapal
  3

• prefetcher shutdown hang when running with multiprocess + distributed reading service

  #1176 opened a year ago by zhengwy888
  1

• `v2.1.2+cu118` and `v2.1.1+cu118` run into torchdata `ImportError: libssl.so.3: cannot open shared object file: No such file or directory`, that `v2.1.0+cu118` doesn't have an issue with

  #1220 opened 5 months ago by justinxzhao
  1

• Iterating a data pipe, created with random split, ends in error as the code tries to iterate past the data pipe lenght

  #1218 opened 5 months ago by thecaptain2000
  0

• S3FileLoaderIterDataPipe buffer_size

  #1215 opened 7 months ago by commonism
  0

• Loading `.tfrecords` files that require a deserialization method

  #1201 opened 8 months ago by fteufel
  0

• Passing dict in datapipe/dataset will have memory leak problem

  #1183 opened a year ago by qmpzzpmq
  3

• Crash when resetting datapipe with bufffer of filehandles

  #1161 opened a year ago by falckt
  3

• `header()` causes DataLoader with MPRS to hang

  #1148 opened a year ago by wchang-apixio
  3

• Is torchdata still being actively developed?

  #1192 opened a year ago by lendle
  6

• Calling __iter__ twice on DataLoader2 causes hang with MPRS

  #1198 opened a year ago by JohnHBrock
  2

• Does Collator need to exist?

  #1181 opened a year ago by lendle
  1

• tcp connection refused when multinode training

  #1142 opened a year ago by rxqy
  3

• torchdata has a very low accuracy

  #1194 opened a year ago by jc-su
  0

• An iterator that can stream over stdin

  #1193 opened a year ago by erip
  0

• Dataloader2 with FullSyncIterDataPipe throws error during initilization

  #1190 opened a year ago by chenxingyu-cs
  3

• FileExistsError when using `on_disk_cache` and multiple workers

  #1186 opened a year ago by knoriy
  1

• Roadmap for mixed chain of multithread and multiprocessing pipelines?

  #1184 opened a year ago by npuichigo
  2

• Memory spikes with large DataPipes

  #1150 opened a year ago by sesquipedalianist
  11

• MultiplexerLongest example snippet isn't very useful

  #1182 opened a year ago by lendle
  0

• what is the right way to serialize DataLoader2 so that pipeline with shuffle can resume from the right place?

  #1177 opened a year ago by zhengwy888
  2

• Using ShardingFilterIterDataPipe with MPRS may cause unnecessary batch drops.

  #1180 opened a year ago by yuxinyuan
  0

• Allow more than 1 dispatching process in MultiProcessReadingService

  #1178 opened a year ago by ben-davidson-6
  1

• Mux with MPRS causes operations after sharding_round_robin_dispatcher to run on the same worker

  #1175 opened a year ago by JohnHBrock
  3

• Deadlock / Race-condition after latest update

  #1169 opened a year ago by sehoffmann
  9

• MPRS hangs if child processes get killed externally (e.g. from OOM reaper)

  #1173 opened a year ago by sehoffmann
  3

• Support for proper Distributed & Multiprocessing Sharding

  #1174 opened a year ago by sehoffmann
  0

• [RFC] Performance Profiling Tools

  #1149 opened a year ago by NivekT
  3

• Zipped csv file cannot be parsed using torchdata’s CSVParser datapipe

  #1172 opened a year ago by seunggs
  1

• Caching doesn't work with cycle

  #1167 opened a year ago by Modexus
  0

• MPRS with DL2 memory use increases over time vs. using older DL

  #1165 opened a year ago by rejuvyesh
  0

• `.fork(n)` returns original datapipe for `n==1` instead of singleton list

  #1164 opened a year ago by sehoffmann
  0

• Additional basic functions beyond .map to allow for more functional programming

  #1145 opened a year ago by hhoeflin
  1

• Add zlib support to Decompressor

  #1163 opened a year ago by JohnHBrock
  1

• InMemoryCache dp will yield from source dp once even when data is fully cached

  #1159 opened a year ago by hirayaku
  0

• Race condition when prefetching forks

  #1162 opened a year ago by falckt
  1

• default pin_memory_fn with string Sequences and batch_size 1

  #1157 opened a year ago by Modexus
  1

• in 'webdataset' the keys in the resulting dict have leading '.'

  #1144 opened a year ago by hhoeflin
  7

• Improve AsyncIODataPipe

  #1155 opened a year ago by ejguan
  0

• AttributeError: 'SequenceWrapperMapDataPipe' object has no attribute 'prefetch . when using MultiProcessingReadingService

  #1143 opened a year ago by One-sixth
  2

• Shuffle batches across workers

  #1140 opened a year ago by platers
  2