Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xarray and Sliderule IO Improvements #31

Merged
merged 5 commits into from
Dec 6, 2024
Merged

Xarray and Sliderule IO Improvements #31

merged 5 commits into from
Dec 6, 2024

Conversation

scottyhq
Copy link
Member

@scottyhq scottyhq commented Dec 6, 2024

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@scottyhq
Copy link
Member Author

scottyhq commented Dec 6, 2024

@Jack-Hayes I'm going to merge this since it has useful functions for AGU prep. Note this takes largely takes care of what you started in #30 but still would love your help with tests, docs, and any features you think are still missing (like the polygon validation).

@scottyhq scottyhq merged commit d4a1bf8 into main Dec 6, 2024
11 checks passed
@scottyhq scottyhq deleted the xarrayio branch December 6, 2024 16:37
@Jack-Hayes
Copy link
Member

Gotcha! I'm hoping to implement that test and an example notebook for the docs once I make the loading faster with dask if that sounds good to you

@dshean
Copy link
Member

dshean commented Dec 7, 2024

Great to explore dask, but it should be optional.

Hoping we can offer options to directly load overviews (instead of full-res raster) from COGs, or load coarser raster products.

@scottyhq
Copy link
Member Author

Great to explore dask, but it should be optional.

Yes, could certainly revisit choice to use dask by default here (but can override by passing chunks=None)

ds = odc.stac.load(items, bands=bands, geopolygon=aoi, chunks={}, **kwargs)

I'm not sure if it's better or not to include the kitchen sink of dependencies in this project or to keep a lot of the I/O and visualization dependencies as optional... Including everything is the current approach but may get complicated (for example if we add PDAL for custom 3DEP workflows, which pins older arrow & hdf versions which i'd like to avoid b/c of search relying on recent geoparquet/arrow)...

]
# all = [
# "matplotlib-base>=3.9.2,<4",
# "sliderule>=4.7.1,<5"
# ]

Hoping we can offer options to directly load overviews (instead of full-res raster) from COGs, or load coarser raster products.

With odc-stac, reading from appropriate overviews should happen automatically by setting the resolution= keyword, regardless of dask usage (for example https://coincident.readthedocs.io/en/latest/examples/contextual_data.html#load-raster-datasets-with-xarray)

More discussion of reading overviews including confirming they are used via logging is here gjoseph92/stackstac#196

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create Altimeter Point Plotting Function
3 participants