pydata/xarray

Generalize dask.delayed calls to go through ChunkManager

TomNicholas opened this issue · 0 comments

[Deepak: Should we add chunked_array_type and from_array_kwargs to open_mfdataset?

I actually don't think we need to - from_array_kwargs is only going to get directly passed down to open_dataset, and hence could be considered part of **kwargs.

This should actually just work, except in the case of parallel=True. For that we could add delayed to the ChunkManager ABC, so that if cubed does implement cubed.delayed it could be added, else a NotImplementedError would be raised. I think all of this wouldn't be necessary if we had lazy concatenation in xarray though (xref #4628). That suggestion would mean we should also replace other instances of dask.delayed in other parts of the codebase though... I think I will split this into a separate issue in the interests of getting this one merged.

Originally posted by @TomNicholas in #7019 (comment)