Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable dynamic definition of Reader for MultiBaseReader #711

Merged
merged 7 commits into from
Sep 9, 2024

Conversation

vincentsarago
Copy link
Member

@vincentsarago vincentsarago commented May 22, 2024

The main PR goal is to add a method to enable dynamic definition of the Asset's Reader based on its type (e.g COG or NetCDF).

The addition is non-breaking, the default reader will always be self.reader

from typing import Type, Set

from rio_tiler.types import AssetInfo
import attr
from rio_tiler.io import STACReader as OfficialSTACReader
from rio_tiler.io import Reader, XarrayReader, BaseReader
valid_types = {
    "image/tiff; application=geotiff",
    "image/tiff; application=geotiff; profile=cloud-optimized",
    "image/tiff; profile=cloud-optimized; application=geotiff",
    "image/tiff; application=geotiff; profile=cloud-optimized",
    "image/vnd.stac.geotiff; cloud-optimized=true",
    "image/tiff",
    "image/x.geotiff",
    "image/jp2",
    "application/x-hdf5",
    "application/x-hdf",
    "application/vnd+zarr",
    "application/x-netcdf",
}


@attr.s
class STACReader(OfficialSTACReader):

    include_asset_types: Set[str] = attr.ib(default=valid_types)

    def _get_reader(self, asset_info: AssetInfo) -> Type[BaseReader]:
        """Get Asset Reader."""
        asset_type = asset_info.get("type", None)

        if asset_type and asset_type in [
            "application/x-hdf5",
            "application/x-hdf",
            "application/vnd.zarr",
            "application/x-netcdf",

        ]:
            return XarrayReader

        return Reader

with STACReader("https://planetarycomputer.microsoft.com/api/stac/v1/collections/noaa-cdr-sea-surface-temperature-optimum-interpolation/items/oisst-avhrr-v02r01.20240506") as src:
    print(src)
    print(src.assets)
    info = src._get_asset_info("netcdf")
    print(info)
    print(info["type"])
    print(src._get_reader(info))


>> STACReader(bounds=[-180, -90.0, 180, 90.0], crs=CRS.from_epsg(4326), assets=['err', 'ice', 'sst', 'anom', 'netcdf'], input='https://planetarycomputer.microsoft.com/api/stac/v1/collections/noaa-cdr-sea-surface-temperature-optimum-interpolation/items/oisst-avhrr-v02r01.20240506', item=<Item id=oisst-avhrr-v02r01.20240506>, tms=<TileMatrixSet title='Google Maps Compatible for the World' id='WebMercatorQuad' crs='[http://www.opengis.net/def/crs/EPSG/0/3857>](http://www.opengis.net/def/crs/EPSG/0/3857%3E), minzoom=0, maxzoom=24, geographic_crs=CRS.from_epsg(4326), include_assets=None, exclude_assets=None, exclude_asset_types=None, reader=<class 'rio_tiler.io.rasterio.Reader'>, reader_options={}, fetch_options={}, ctx=<class 'rasterio.env.Env'>, include_asset_types={'image/tiff; application=geotiff', 'image/jp2', 'application/x-hdf', 'image/tiff; application=geotiff; profile=cloud-optimized', 'image/x.geotiff', 'application/x-hdf5', 'image/vnd.stac.geotiff; cloud-optimized=true', 'application/x-netcdf', 'application/vnd+zarr', 'image/tiff; profile=cloud-optimized; application=geotiff', 'image/tiff'})
>> ['err', 'ice', 'sst', 'anom', 'netcdf']
>> {'url': 'https://noaacdr.blob.core.windows.net/sea-surface-temp-optimum-interpolation/data/v2.1/avhrr/202405/oisst-avhrr-v02r01.20240506.nc', 'metadata': {'created': '2024-05-21T09:12:00Z', 'updated': '2024-05-21T09:12:00Z'}, 'type': 'application/x-netcdf'}
>> application/x-netcdf
>> <class 'rio_tiler.io.xarray.XarrayReader'>

cc @abarciauskas-bgse

@vincentsarago vincentsarago marked this pull request as ready for review September 9, 2024 10:46
@vincentsarago
Copy link
Member Author

Note: This is just a first step! I think we will need to way to define ReaderOption dynamically

def _reader(asset: str, *args: Any, **kwargs: Any) -> ImageData:
idx = asset_indexes.get(asset) or indexes # type: ignore
asset_info = self._get_asset_info(asset)
url = asset_info["url"]
with self.ctx(**asset_info.get("env", {})):
with self.reader(url, tms=self.tms, **self.reader_options) as src: # type: ignore
data = src.tile(*args, indexes=idx, **kwargs)

either from asset_info struct or with an external asset_options: Optional[Dict[str, Any]] attribute as we do with asset_indexes

asset_indexes: Optional[Dict[str, Indexes]] = None, # Indexes for each asset

@vincentsarago vincentsarago merged commit b2b9f83 into main Sep 9, 2024
8 checks passed
@vincentsarago vincentsarago deleted the feature/dynamic-reader-for-MultiBaseReader branch September 9, 2024 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant