Generalize dask.delayed calls to go through ChunkManager
TomNicholas opened this issue · 0 comments
[Deepak: Should we add
chunked_array_type
andfrom_array_kwargs
toopen_mfdataset
?
I actually don't think we need to - from_array_kwargs
is only going to get directly passed down to open_dataset
, and hence could be considered part of **kwargs
.
This should actually just work, except in the case of parallel=True
. For that we could add delayed
to the ChunkManager
ABC, so that if cubed does implement cubed.delayed
it could be added, else a NotImplementedError
would be raised. I think all of this wouldn't be necessary if we had lazy concatenation in xarray though (xref #4628). That suggestion would mean we should also replace other instances of dask.delayed
in other parts of the codebase though... I think I will split this into a separate issue in the interests of getting this one merged.
Originally posted by @TomNicholas in #7019 (comment)