pangeo-data/rechunker

Allow for single partial chunk

rsignell-usgs opened this issue · 4 comments

rsignell-usgs commented

When I create a large rechunked dataset, I use rechunker on each time chunk, then append. The last time chunk currently fails because rechunker thinks the chunk plan is invalid (because only a single partial chunk is being written).

Here's a full example: https://nbviewer.jupyter.org/gist/rsignell-usgs/c0b87ed1fa5fc694e665fb789e8381bb

rabernat commented

This is what @davidbrochart was working on in #28. That stalled because we did some internal refactoring which broke his approach. But maybe we can convince him to pick it up again? 😊

davidbrochart commented

I'd be happy to work on this, but didn't have time so far 😞

rabernat commented

I wish we could argue that this is within scope for the Quantstack Zarr EOSS work! 😆

davidbrochart commented

Me too! But not related to C++ or Zarr v3 unfortunately.
If QuantStack could be involved in another Zarr project, I'd love to work on these things.