`.fork(n)` returns original datapipe for `n==1` instead of singleton list
sehoffmann opened this issue ยท 0 comments
sehoffmann commented
๐ Describe the bug
pipe.fork(n)
returns the original pipe when n == 1
and not a list with 1 element. This introduces two issues:
- It's unexpected to the user
- It requires the user to add extra code to handle this edgecase if the number of forks is dynamically determined during runtime
For instance, consider the following two snippets that will fail if n==1
:
pipes = IterableWrapper(self.years).fork(n_vars)
for pipe in pipes:
process_pipe(pipe)
=> iterates over elements of the original pipe and not over the actual pipes.
pipes = IterableWrapper(self.years).fork(n_vars)
for idx in range(n_vars):
process_pipe(pipes[idx])
=> Will raise an NotImplementedError
from Dataset.__getitem__()
.
To handle this case, the user needs to add extra code specifically for n==1
.
Versions
nightly / latest