csiro-easi/ccog

tile data layout

Opened this issue · 1 comments

artttt commented

The gdal COG driver writes the tile binary data in a row major order. GDAL also adds some extra optimisations that require this order.

CCOG doesn't strictly stick to this ordering although it i think its possible to achieve it by rechunking the data into blocksize high strips, or higher but full width strips. Not ideal.

Row major order isn't used because on first look it would need quite a bit of data caching or some other inefficiencies to achieve.

On the other hand the not quite row major order is probably not a major issue in many use cases.

So, this should either be fixed or documented.

Its also worth noting here that early descriptions found online of GDALs cog driver floated the idea of optionally using z order (aka Morton order). This seems to be more achievable for writing from dask chunks.

artttt commented

Confirmed that the following works.

"CCOG doesn't strictly stick to this ordering although it i think its possible to achieve it by rechunking the data into blocksize high strips, or higher but full width strips. Not ideal."

If the above conditions are met the data layout matches gdal output and I've made it retain the gdal cog driver ghost data optimisation.