-
Notifications
You must be signed in to change notification settings - Fork 224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
**BREAKING** pygmt.grdcut: Refactor to store output in virtualfiles for grids #3115
Open
seisman
wants to merge
98
commits into
main
Choose a base branch
from
gmtimage
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
98 commits
Select commit
Hold shift + click to select a range
bcf43f0
Wrap GMT's standard data type GMT_IMAGE for images
seisman a052a1a
Initial implementation of to_dataarray method for _GMT_IMAGE class
weiji14 59d523c
pygmt.grdcut: Support both grid and image output
seisman 56a6d65
Merge branch 'main' into datatypes/gmtimage
seisman 3315324
Merge branch 'main' into gmtimage
seisman cea3374
Fix
seisman 80d9837
Refactor
seisman 22fba56
fix
seisman f71e79c
Merge branch 'main' into datatypes/gmtimage
weiji14 4cce4a2
Small typo fixes and add output type-hint for to_dataarray
weiji14 e02b650
Fix mypy error using np.array([0, 1, 2]) instead of np.arange
weiji14 f3d4b1f
Parse name and data_attrs from grid/image header
weiji14 4390136
Transpose array to (band, y, x) order and add doctest for to_dataarray
weiji14 5f25669
Set registration and gtype from header
weiji14 a3c6c14
Print basic shape and padding info in _GMT_IMAGE doctest
weiji14 5888e10
Only set Conventions = CF-1.7 attribute for NetCDF grid type
weiji14 798e658
Merge branch 'main' into datatypes/gmtimage
weiji14 3dbf2f2
Remove rioxarray import
weiji14 3a24ebd
Apply suggestions from code review
seisman 4eee7e6
Merge branch 'main' into gmtimage
seisman 5e390d4
Address reviewer's comments
seisman 003383d
Fix GMT_OUT
seisman 606ac7e
Merge branch 'main' into gmtimage
seisman c6cdcc8
Merge branch 'main' into gmtimage
seisman 377941a
Revert changes for _GMT_IMAGE
seisman 20617f5
Use rioxarray.open_rasterio for loading images
seisman a998718
Check if rioxarray is installed
seisman 86cab44
Improve grdcut
seisman 6031bab
Fix typos in grdcut
seisman eb0af2d
Add tests for grdcut images
seisman 7f6ca7d
Fix one failing test
seisman 21b194a
Fix open_rasterio
seisman e7eaf5c
Fix open_rasterio
seisman e3c8569
Make sure the image is loaded
seisman 1c8312c
Update pygmt/clib/session.py
seisman 3913430
Use rioxarray.open_rasterio in a context manager
seisman 812a225
Merge branch 'main' into gmtimage
seisman 90bd29e
Merge remote-tracking branch 'origin/gmtimage' into gmtimage
seisman ab77187
Fix mypy errors
seisman 6f3e474
Move grdcut image tests to a separate test file
seisman 5b07dd9
Fix copy & paste errors
seisman 31272ab
Run codspeed benchmark for test_grdcut_image_dataarray
seisman 6b860bf
Merge branch 'main' into datatypes/gmtimage
seisman 5a09329
Merge branch 'main' into gmtimage
seisman 279595b
Add the raster_kind function to determine the raster kind
seisman 7def4b5
Simplify the grdcut function
seisman be175d8
Merge branch 'main' into gmtimage
seisman 0bf9368
Merge branch 'main' into datatypes/gmtimage
seisman 7d437be
Use enum for grid ids
seisman 268e34e
Fix the band. Starting from 1
seisman 86765e1
Refactor the tests for images
seisman 86f3ffa
In np.reshape, a is a position-only parameter
seisman cc28247
Improve tests
seisman 1e2c973
Fix one failing doctest due to xarray changes
seisman 734dc28
The np.reshape's newshape parameter is deprecated
seisman 919dc00
Define grid IDs using IntEnum instead of Enum
seisman b1eacf1
Pass the new shape as a positional parameter
seisman aa4fdc9
Fix failing tests
seisman c87a3ec
One more fix
seisman a20d8a2
One more fix
seisman 926427b
Simplify a doctest
seisman c73328e
Improve the tests
seisman 2825eae
Merge branch 'datatypes/gmtimage' into gmtimage
seisman bf9275c
Remove the workaround for images
seisman fb97daa
Convert ctypes array to numpy array using np.ctypeslib.as_array
seisman 15b8d53
Fix the incorrect value due to floating number conversion in sphinter…
seisman 8433e78
Merge branch 'ctypesarray' into datatypes/gmtimage
seisman 3e3a6f3
Update the to_dataarray method to match the codes in GMT_GRID
seisman 12ef40a
image data should has uint8 dtype
seisman f64fbb8
Further improve the tests
seisman e9cb0a5
Merge branch 'datatypes/gmtimage' into gmtimage
seisman 4f2ae48
Merge branch 'main' into datatypes/gmtimage
seisman d49afed
Add a note that currently only 3-band images are supported
seisman a97d0b3
Apply suggestions from code review
seisman f70bec0
Merge branch 'main' into datatypes/gmtimage
seisman 2fd13fb
Remove the old GMTGridID enums from pygmt/datatypes/header.py
seisman 9972ba1
A minor fix
seisman ac6b7c3
Merge branch 'datatypes/gmtimage' into gmtimage
seisman 7c32d41
Merge branch 'main' into gmtimage
seisman 9ec00be
Let _raster_kind return grid by default
seisman f3a2f8e
Simplify the grdcut image tests
seisman 3c12e2b
Add one more test for file in & file out
seisman f852b0d
Fix typos
seisman 5f7683c
Merge branch 'main' into gmtimage
seisman bb1a0b0
Use the new load_blue_marble function
seisman 584b5af
Drop the spatial_ref coord
seisman b19ba00
Merge branch 'main' into gmtimage
seisman cceb929
Merge branch 'main' into gmtimage
seisman fac51f1
Update _raster_kind
seisman 517ee52
Revert "Update _raster_kind"
seisman d5ec308
Merge branch 'main' into gmtimage
seisman 7ca58fb
Remove the _raster_kind function
seisman 66580d4
Add the 'kind' parameter
seisman 194ee90
Minor update
seisman 473428c
Avoid keep the file open
seisman 84b2ed4
Remove one unnecessary pytest.skipif marker
seisman 7968096
Add it back because we still need rioxarray
seisman 90d5210
Merge branch 'main' into gmtimage
seisman File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,88 @@ | ||
""" | ||
Test pygmt.grdcut on images. | ||
""" | ||
|
||
from pathlib import Path | ||
|
||
import numpy as np | ||
import pytest | ||
import xarray as xr | ||
from pygmt import grdcut | ||
from pygmt.datasets import load_blue_marble | ||
from pygmt.helpers import GMTTempFile | ||
|
||
try: | ||
import rioxarray | ||
|
||
_HAS_RIOXARRAY = True | ||
except ImportError: | ||
_HAS_RIOXARRAY = False | ||
|
||
|
||
@pytest.fixture(scope="module", name="region") | ||
def fixture_region(): | ||
""" | ||
Set the data region. | ||
""" | ||
return [-53, -49, -20, -17] | ||
|
||
|
||
@pytest.fixture(scope="module", name="expected_image") | ||
def fixture_expected_image(): | ||
""" | ||
Load the expected grdcut image result. | ||
""" | ||
return xr.DataArray( | ||
data=np.array( | ||
[ | ||
[[90, 93, 95, 90], [91, 90, 91, 91], [91, 90, 89, 90]], | ||
[[87, 88, 88, 89], [88, 87, 86, 85], [90, 90, 89, 88]], | ||
[[48, 49, 49, 45], [49, 48, 47, 45], [48, 47, 48, 46]], | ||
], | ||
dtype=np.uint8, | ||
), | ||
coords={ | ||
"band": [1, 2, 3], | ||
seisman marked this conversation as resolved.
Show resolved
Hide resolved
|
||
"x": [-52.5, -51.5, -50.5, -49.5], | ||
"y": [-17.5, -18.5, -19.5], | ||
}, | ||
dims=["band", "y", "x"], | ||
attrs={ | ||
"scale_factor": 1.0, | ||
"add_offset": 0.0, | ||
}, | ||
) | ||
|
||
|
||
@pytest.mark.benchmark | ||
def test_grdcut_image_file(region, expected_image): | ||
""" | ||
Test grdcut on an input image file. | ||
""" | ||
result = grdcut("@earth_day_01d", region=region, kind="image") | ||
xr.testing.assert_allclose(a=result, b=expected_image) | ||
|
||
|
||
@pytest.mark.benchmark | ||
@pytest.mark.skipif(not _HAS_RIOXARRAY, reason="rioxarray is not installed") | ||
seisman marked this conversation as resolved.
Show resolved
Hide resolved
|
||
def test_grdcut_image_dataarray(region, expected_image): | ||
""" | ||
Test grdcut on an input xarray.DataArray object. | ||
""" | ||
raster = load_blue_marble() | ||
result = grdcut(raster, region=region, kind="image") | ||
xr.testing.assert_allclose(a=result, b=expected_image) | ||
|
||
|
||
def test_grdcut_image_file_in_file_out(region, expected_image): | ||
""" | ||
Test grdcut on an input image file and outputs to another image file. | ||
""" | ||
with GMTTempFile(suffix=".tif") as tmp: | ||
result = grdcut("@earth_day_01d", region=region, outgrid=tmp.name) | ||
assert result is None | ||
assert Path(tmp.name).stat().st_size > 0 | ||
if _HAS_RIOXARRAY: | ||
with rioxarray.open_rasterio(tmp.name) as raster: | ||
image = raster.load().drop_vars("spatial_ref") | ||
xr.testing.assert_allclose(a=image, b=expected_image) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Lines 117-118 should be caught by the
test_grdcut_fails
test.