Forward rasterio env options in multibase Reader
Opened this issue · 2 comments
vincentsarago commented
in MultiBaseReader
(STAC) we initiate a new rasterio.Env
to be able to set the Header size from each file. Sadly when we do this we loose any env options passed in a top level env
with rasterio.Env(
session=AWSSession(
aws_access_key_id="MyDevseedId",
aws_secret_access_key="MyDevseedKey",
)
):
with rasterio.open("s3://ds-satellite/cogs/NaturalEarth/world_grey.tif") as src:
print(src.profile)
with rasterio.Env():
with rasterio.open("s3://ds-satellite/cogs/NaturalEarth/world_grey_1024_512.tif") as src:
print(src.profile)
{'driver': 'GTiff', 'dtype': 'uint8', 'nodata': None, 'width': 21580, 'height': 10780, 'count': 3, 'crs': CRS.from_epsg(4326), 'transform': Affine(0.01666666666667, 0.0, -179.8333333333333,
0.0, -0.01666666666667, 89.83333333333331), 'blockxsize': 128, 'blockysize': 128, 'tiled': True, 'compress': 'jpeg', 'interleave': 'pixel', 'photometric': 'ycbcr'}
rasterio/_base.pyx in rasterio._base.DatasetBase.__init__()
RasterioIOError: Access Denied
First reported in NASA-IMPACT/veda-backend#192
vincentsarago commented
😭 This is also true for mosaic
from rio_tiler.tasks import create_tasks, filter_tasks
import rasterio
from rasterio.session import AWSSession
with rasterio.Env(
session=AWSSession(
aws_access_key_id="MyDevseedId",
aws_secret_access_key="MyDevseedKey",
),
):
paths = [
"s3://ds-satellite/cogs/NaturalEarth/world_grey.tif",
"s3://ds-satellite/cogs/NaturalEarth/world_grey_512_256.tif",
]
def reader(path):
with rasterio.open(path) as src:
return src.profile
tasks = filter_tasks(create_tasks(reader, paths, threads=2))
>> RasterioIOError: Access Denied
I always assumed that rasterio was picking the envs from top level but no. It will create a new Env per thread, I guess this is why stackstac does https://github.com/gjoseph92/stackstac/blob/main/stackstac/rio_reader.py
ranchodeluxe commented
@vincentsarago : given the lack of motion on rasterio/rasterio#2866 is there a fix you're working on here now? Or is this something I can take swing at when I get a chance?