Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests for Dataset.crop() #310

Merged
merged 47 commits into from
Jan 7, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
0bacfdf
Add fixture for new VISP asdf (remove this later too)
SolarDrew Oct 4, 2023
07326bb
Add very basic Stokes crop test
SolarDrew Oct 4, 2023
76248a7
Add another fixture for the new asdf
SolarDrew Oct 5, 2023
10b8645
Add my own test env for tox because it was being a pain
SolarDrew Oct 5, 2023
a1a2301
Fix the new fixture
SolarDrew Oct 5, 2023
cb45d60
More cropping tests
SolarDrew Oct 5, 2023
3467514
More variations on cropping tests
SolarDrew Oct 10, 2023
f8f1a68
Lots of changes to make the tests work
SolarDrew Oct 11, 2023
85fe54b
Add an extra test
SolarDrew Oct 11, 2023
e6150fa
Trim out some extra testing that probably isn't useful any more
SolarDrew Oct 11, 2023
6c41cfb
Correction and some more assertions
SolarDrew Oct 11, 2023
8c5a7ed
Trim more stuff, add more assertions
SolarDrew Oct 11, 2023
86448ad
Minor corrections
SolarDrew Oct 12, 2023
a7be3a5
Add test for cropping visp by time
SolarDrew Oct 13, 2023
8c1a731
Clear out some unneeded comments
SolarDrew Oct 13, 2023
1f8c859
Make latest test use user-defined coords and add assertions
SolarDrew Oct 13, 2023
c85c11f
Add test for cropping visp by lon/lat
SolarDrew Oct 13, 2023
2393e9a
More testing. Always more testing
SolarDrew Oct 13, 2023
03c88e6
Tweak a few numbers
SolarDrew Oct 16, 2023
fdf96be
Yet another assertion
SolarDrew Oct 16, 2023
fb7364c
Yes! It's more assertions!
SolarDrew Oct 17, 2023
3001020
Add some plots for now, might remove later
SolarDrew Oct 17, 2023
4cd2b09
Rebuild gwcs headers in files used for crop fixtures
SolarDrew Oct 24, 2023
52fe302
Make the assertions pass
SolarDrew Oct 24, 2023
f8bc814
Don't need this
SolarDrew Oct 24, 2023
27dccd6
Don't need plotting right now
SolarDrew Oct 24, 2023
c16f216
Bunch of coords need updating for fixed gwcs headers
SolarDrew Oct 27, 2023
43194c8
Add dkist simulator and inventory to deps
SolarDrew Oct 27, 2023
af32415
Tidying and correct numbers
SolarDrew Oct 30, 2023
7c755c0
Want this back after all
SolarDrew Oct 30, 2023
d983710
More assertions and tweak some numbers to make smaller arrays
SolarDrew Nov 3, 2023
44eaa40
Add new (compressed) dataset asdf for testing crop
SolarDrew Nov 7, 2023
6bf3f14
Update conftest for new compressed VISP test file
SolarDrew Nov 7, 2023
9419de5
Add compressed cryo file for crop testing
SolarDrew Nov 7, 2023
17b8da8
Update conftest for new cryo file
SolarDrew Nov 7, 2023
27e6ec6
Tidy up tests a little
SolarDrew Nov 8, 2023
9ed86f0
Don't need custom test env any more
SolarDrew Nov 8, 2023
ebfdbe5
Check Stokes headers of Stokes-cropped datasets
SolarDrew Nov 8, 2023
b211d59
Correct rebase fail
SolarDrew Jan 6, 2025
50e4f9e
Merge branch 'main' of github.com:DKISTDC/dkist into bugfix-192
SolarDrew Jan 6, 2025
5cb429b
Update tox.ini
SolarDrew Jan 6, 2025
1b49a64
Merge branch 'main' of github.com:DKISTDC/dkist into bugfix-192
SolarDrew Jan 6, 2025
9921256
Formatting
SolarDrew Jan 6, 2025
6a4bd03
Brainfart - thank you @Cadair
SolarDrew Jan 6, 2025
28fefa4
Skip dataset crop tests if gwcs isn't up to date
SolarDrew Jan 6, 2025
e94a5d0
Changelog
SolarDrew Jan 6, 2025
b052683
Update dkist/dataset/tests/test_crop_dataset.py
Cadair Jan 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/310.trivial.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add tests for cropping datasets by world coords
5 changes: 5 additions & 0 deletions dkist/dataset/tests/test_crop_dataset.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,15 @@
import pytest

import astropy.units as u
from astropy.coordinates import SkyCoord, SpectralCoord, StokesCoord
from astropy.time import Time

gwcs = pytest.importorskip("gwcs", "0.22.2")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking more in the test which needed this, but if they all do this is fine.

Cadair marked this conversation as resolved.
Show resolved Hide resolved


def test_crop_visp_by_only_stokes(croppable_visp_dataset):

Check warning on line 10 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L10

Added line #L10 was not covered by tests

cropped = croppable_visp_dataset.crop([

Check warning on line 12 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L12

Added line #L12 was not covered by tests
None,
None,
None,
Expand All @@ -17,15 +22,15 @@
StokesCoord("I"),
])

assert cropped.wcs.pixel_n_dim == croppable_visp_dataset.wcs.pixel_n_dim - 1
assert cropped.data.shape == croppable_visp_dataset.data.shape[1:]
assert (cropped.headers["DINDEX4"] == 1).all()

Check warning on line 27 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L25-L27

Added lines #L25 - L27 were not covered by tests


def test_crop_visp_by_time(croppable_visp_dataset):
coords = (croppable_visp_dataset.wcs.pixel_to_world(0, 0, 200, 0),

Check warning on line 31 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L30-L31

Added lines #L30 - L31 were not covered by tests
croppable_visp_dataset.wcs.pixel_to_world(2554, 975, 400, 3))
cropped = croppable_visp_dataset.crop([

Check warning on line 33 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L33

Added line #L33 was not covered by tests
SpectralCoord(630.242*u.nm),
SkyCoord(-415.65*u.arcsec, 163.64*u.arcsec,
frame=coords[0][0].frame),
Expand All @@ -40,31 +45,31 @@
None,
])

assert cropped.wcs.pixel_n_dim == croppable_visp_dataset.wcs.pixel_n_dim
assert cropped.data.shape[0] == croppable_visp_dataset.data.shape[0]
assert cropped.data.shape[1] == 201
assert cropped.data.shape[2:] == croppable_visp_dataset.data.shape[2:]

Check warning on line 51 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L48-L51

Added lines #L48 - L51 were not covered by tests

orig_coords = croppable_visp_dataset.axis_world_coords()
cropped_coords = cropped.axis_world_coords()
assert (cropped_coords[0][0] == orig_coords[0][200]).all()
assert (cropped_coords[0][-1] == orig_coords[0][400]).all()
assert (cropped_coords[1] == orig_coords[1]).all()
assert (cropped_coords[2] == orig_coords[2][200:401]).all()
assert (cropped_coords[3] == orig_coords[3]).all()

Check warning on line 59 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L53-L59

Added lines #L53 - L59 were not covered by tests


def test_crop_visp_by_lonlat(croppable_visp_dataset):
coords = (croppable_visp_dataset.wcs.pixel_to_world(500, 0, 200, 0),

Check warning on line 63 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L62-L63

Added lines #L62 - L63 were not covered by tests
croppable_visp_dataset.wcs.pixel_to_world(1000, 977, 600, 4))

coord0 = SkyCoord(-415.72*u.arcsec, 178.38*u.arcsec,

Check warning on line 66 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L66

Added line #L66 was not covered by tests
frame=coords[0][0].frame)

coord1 = SkyCoord(-394.63*u.arcsec, 193.23*u.arcsec,

Check warning on line 69 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L69

Added line #L69 was not covered by tests
frame=coords[1][0].frame)

cropped = croppable_visp_dataset.crop([

Check warning on line 72 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L72

Added line #L72 was not covered by tests
SpectralCoord(630.242*u.nm),
coord0,
Time("2022-10-24T19:08:09"),
Expand All @@ -77,23 +82,23 @@
None,
])

assert cropped.wcs.pixel_n_dim == croppable_visp_dataset.wcs.pixel_n_dim
assert cropped.data.shape[0] == croppable_visp_dataset.data.shape[0]
assert cropped.data.shape[1] == 401
assert cropped.data.shape[2] == croppable_visp_dataset.data.shape[2]
assert cropped.data.shape[3] == 501

Check warning on line 89 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L85-L89

Added lines #L85 - L89 were not covered by tests

orig_coords = croppable_visp_dataset.axis_world_coords()
cropped_coords = cropped.axis_world_coords()
assert (cropped_coords[0][0] == orig_coords[0][200][500:1001]).all()
assert (cropped_coords[0][-1] == orig_coords[0][600][500:1001]).all()
assert (cropped_coords[1] == orig_coords[1]).all()
assert (cropped_coords[2] == orig_coords[2][200:601]).all()
assert (cropped_coords[3] == orig_coords[3]).all()

Check warning on line 97 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L91-L97

Added lines #L91 - L97 were not covered by tests


def test_crop_cryo_by_only_stokes(croppable_cryo_dataset):
cropped = croppable_cryo_dataset.crop([

Check warning on line 101 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L100-L101

Added lines #L100 - L101 were not covered by tests
None,
None,
StokesCoord("I"),
Expand All @@ -104,20 +109,20 @@
StokesCoord("I"),
])

assert cropped.wcs.pixel_n_dim == croppable_cryo_dataset.wcs.pixel_n_dim - 1
assert cropped.data.shape == croppable_cryo_dataset.data.shape[1:]
assert (cropped.headers["DINDEX5"] == 1).all()

Check warning on line 114 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L112-L114

Added lines #L112 - L114 were not covered by tests


def test_crop_cryo_by_time(croppable_cryo_dataset):
coords = (croppable_cryo_dataset.wcs.pixel_to_world(0, 0, 0, 0, 0),

Check warning on line 118 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L117-L118

Added lines #L117 - L118 were not covered by tests
croppable_cryo_dataset.wcs.pixel_to_world(1751, 1888, 1, 1, 3))
coord0 = SkyCoord(-1011.06*u.arcsec, 314.09*u.arcsec,

Check warning on line 120 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L120

Added line #L120 was not covered by tests
frame=coords[0][0].frame)
coord1 = SkyCoord(-1275.20*u.arcsec, 174.27*u.arcsec,

Check warning on line 122 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L122

Added line #L122 was not covered by tests
frame=coords[1][0].frame)

cropped = croppable_cryo_dataset.crop([

Check warning on line 125 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L125

Added line #L125 was not covered by tests
coord0,
# Time has to be later than the start time because the crop is the smallest range that includes specified values
Time("2023-01-01T13:00:04"),
Expand All @@ -129,34 +134,34 @@
None,
])

assert cropped.wcs.pixel_n_dim == croppable_cryo_dataset.wcs.pixel_n_dim
assert cropped.data.shape[0] == croppable_cryo_dataset.data.shape[0]
assert cropped.data.shape[1] == croppable_cryo_dataset.data.shape[1] - 1
assert cropped.data.shape[2] == croppable_cryo_dataset.data.shape[2] - 1
assert cropped.data.shape[3:] == croppable_cryo_dataset.data.shape[3:]

Check warning on line 141 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L137-L141

Added lines #L137 - L141 were not covered by tests

orig_coords = croppable_cryo_dataset.axis_world_coords()
cropped_coords = cropped.axis_world_coords()

Check warning on line 144 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L143-L144

Added lines #L143 - L144 were not covered by tests

# Whole coordinate array is too large to compare, so check just the edges
assert (cropped_coords[0][0, 0, 0, :] == orig_coords[0][0, 0, 0, :]).all()
assert (cropped_coords[0][0, 0, -1, :] == orig_coords[0][0, 0, -1, :]).all()
assert (cropped_coords[0][0, 0, :, 0] == orig_coords[0][0, 0, :, 0]).all()
assert (cropped_coords[0][0, 0, :, -1] == orig_coords[0][0, 0, :, -1]).all()
assert (cropped_coords[1] == orig_coords[1][:2, :2]).all()
assert (cropped_coords[2] == orig_coords[2]).all()

Check warning on line 152 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L147-L152

Added lines #L147 - L152 were not covered by tests


def test_crop_cryo_by_only_lonlat(croppable_cryo_dataset):
coords = (croppable_cryo_dataset.wcs.pixel_to_world(0, 0, 0, 0, 0),

Check warning on line 156 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L155-L156

Added lines #L155 - L156 were not covered by tests
croppable_cryo_dataset.wcs.pixel_to_world(200, 200, 2, 2, 3))
coord0 = SkyCoord(-1011.1*u.arcsec, 314.1*u.arcsec,

Check warning on line 158 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L158

Added line #L158 was not covered by tests
frame=coords[0][0].frame)
coord1 = SkyCoord(-1039.5*u.arcsec, 297.6*u.arcsec,

Check warning on line 160 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L160

Added line #L160 was not covered by tests
frame=coords[1][0].frame)

# Crop using user-defined coords
cropped = croppable_cryo_dataset.crop([

Check warning on line 164 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L164

Added line #L164 was not covered by tests
coord0,
Time("2023-01-01T13:00:04"),
Cadair marked this conversation as resolved.
Show resolved Hide resolved
None,
Expand All @@ -167,14 +172,14 @@
None,
])

assert cropped.wcs.pixel_n_dim == croppable_cryo_dataset.wcs.pixel_n_dim
assert cropped.data.shape[:3] == croppable_cryo_dataset.data.shape[:3]
assert cropped.data.shape[3] == 201
assert cropped.data.shape[4] == 201

Check warning on line 178 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L175-L178

Added lines #L175 - L178 were not covered by tests

orig_coords = croppable_cryo_dataset.axis_world_coords()
cropped_coords = cropped.axis_world_coords()

Check warning on line 181 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L180-L181

Added lines #L180 - L181 were not covered by tests

assert (cropped_coords[0][0, 0] == orig_coords[0][0, 0, :201, :201]).all()
assert (cropped_coords[1] == orig_coords[1]).all()
assert (cropped_coords[2] == orig_coords[2]).all()

Check warning on line 185 in dkist/dataset/tests/test_crop_dataset.py

View check run for this annotation

Codecov / codecov/patch

dkist/dataset/tests/test_crop_dataset.py#L183-L185

Added lines #L183 - L185 were not covered by tests
Loading