pytorch/data

`.fork(n)` returns original datapipe for `n==1` instead of singleton list

sehoffmann opened this issue ยท 0 comments

๐Ÿ› Describe the bug

pipe.fork(n) returns the original pipe when n == 1 and not a list with 1 element. This introduces two issues:

  1. It's unexpected to the user
  2. It requires the user to add extra code to handle this edgecase if the number of forks is dynamically determined during runtime

For instance, consider the following two snippets that will fail if n==1:

pipes = IterableWrapper(self.years).fork(n_vars)
for pipe in pipes:
     process_pipe(pipe)

=> iterates over elements of the original pipe and not over the actual pipes.

pipes = IterableWrapper(self.years).fork(n_vars)
for idx in range(n_vars):
     process_pipe(pipes[idx])

=> Will raise an NotImplementedError from Dataset.__getitem__().

To handle this case, the user needs to add extra code specifically for n==1.

Versions

nightly / latest