Issue 1
vincentsarago opened this issue · 1 comments
Internal block size for COG seems like an endless discussion.
Tilebench won't solve everything but with luck and nice reproducible process we will have a better understanding.
Blue - internal blocks
Red - Mercator grid
Methods
Input project has quite an effect so we should make sure to test with several projection
Tools:
- https://github.com/developmentseed/morecantile
- https://github.com/geospatial-jeff/async-cog-reader (compute internal blocks shapes)
- GDAL / Rasterio
Expected results
- Timings (S3 latency and data transfer)
- Number of GET Called
- Transfered byte
Dataset
- Sentinel 2 - Int16 1band
- Naip - 3 bands Byte
Ref:
https://github.com/opendatacube/benchmark-rio-s3/blob/master/report.md
https://github.com/vincentsarago/rio-tiler-bench
https://github.com/vincentsarago/gdal-http2-benchmark
https://github.com/vincentsarago/awspds-benchmark
tests non-square block size
As mentioned in https://github.com/opendatacube/benchmark-rio-s3/blob/master/report.md, the smaller the Y blocksize is the more GET requests GDAL will need to do because we can't merge GET requests in the Y axis (by design).
If we use block size being 512x1024 (X,Y) this could reduce the tile size by 1/2 and keep a nice number of GET requests needed.
Note: Right now GDAL cannot create non squared blocks for Overview but so this will be valid only for raw data. I'm going to start a discussion in gdal mailing list to see what people think about non-square blocksize.