Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix assert_called_once usage in resample tests #2648

Merged
merged 4 commits into from
Nov 26, 2023

Conversation

@djhoese
Copy link
Member

djhoese commented Nov 24, 2023

Thanks @ArrayBolt3! I just made a commit to your branch that calls the assert_called_once method. You can see here that it is actually a method not a property so it needs to be called:

https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called_once

Let's see if the tests pass!

@djhoese djhoese added bug component:tests cleanup Code cleanup but otherwise no change in functionality labels Nov 24, 2023
@djhoese djhoese self-assigned this Nov 24, 2023
@ArrayBolt3
Copy link
Contributor Author

@djhoese lol, you probably couldn't tell I'm not great with python :P

hrm... test results seem bad though...

@djhoese
Copy link
Member

djhoese commented Nov 24, 2023

Ah I think these tests failures are unrelated and being fixed by @pnuu in another PR.

@djhoese
Copy link
Member

djhoese commented Nov 26, 2023

Ok so the slice_data check is actually showing 3 calls instead of 1. The code that calls that method is:

satpy/satpy/scene.py

Lines 921 to 938 in f2f1b33

if reduce_data:
key = source_area
try:
(slice_x, slice_y), source_area = reductions[key]
except KeyError:
if resample_kwargs.get("resampler") == "gradient_search":
factor = resample_kwargs.get("shape_divisible_by", 2)
else:
factor = None
try:
slice_x, slice_y = source_area.get_area_slices(
destination_area, shape_divisible_by=factor)
except TypeError:
slice_x, slice_y = source_area.get_area_slices(
destination_area)
source_area = source_area[slice_y, slice_x]
reductions[key] = (slice_x, slice_y), source_area
dataset = self._slice_data(source_area, (slice_x, slice_y), dataset)

which clearly does a check if it has called the method with the particular area before. The calls from the error message:

2023-11-24T19:30:48.2360799Z E           Calls: [call(Area ID: test
2023-11-24T19:30:48.2361325Z E           Description: test
2023-11-24T19:30:48.2361784Z E           Projection ID: test
2023-11-24T19:30:48.2363167Z E           Projection: {'datum': 'WGS84', 'k_0': '1', 'lat_0': '25', 'lat_1': '25', 'lon_0': '-95', 'no_defs': 'None', 'proj': 'lcc', 'type': 'crs', 'units': 'm', 'x_0': '0', 'y_0': '0'}
2023-11-24T19:30:48.2364359Z E           Number of columns: 3
2023-11-24T19:30:48.2364841Z E           Number of rows: 3
2023-11-24T19:30:48.2366325Z E           Area extent: (-1000.0, -300.0, 200.0, 1500.0), (slice(0, 3, None), slice(0, 3, None)), <xarray.DataArray 'zeros_like-46f66d07f783c1a970c25af2c11c94bd' (y: 20, x: 20,
2023-11-24T19:30:48.2367716Z E                                                                            bands: 3)>
2023-11-24T19:30:48.2368785Z E           dask.array<zeros_like, shape=(20, 20, 3), dtype=float64, chunksize=(20, 20, 3), chunktype=numpy.ndarray>
2023-11-24T19:30:48.2369734Z E           Coordinates:
2023-11-24T19:30:48.2370500Z E             * bands    (bands) <U1 'R' 'G' 'B'
2023-11-24T19:30:48.2371122Z E           Dimensions without coordinates: y, x
2023-11-24T19:30:48.2371684Z E           Attributes:
2023-11-24T19:30:48.2372277Z E               _satpy_id:               DataID(name='comp19')
2023-11-24T19:30:48.2372921Z E               name:                    comp19
2023-11-24T19:30:48.2373825Z E               prerequisites:           [DataQuery(name='ds5', modifiers=('res_change',)...
2023-11-24T19:30:48.2374642Z E               optional_prerequisites:  []
2023-11-24T19:30:48.2375231Z E               optional_datasets:       []
2023-11-24T19:30:48.2375992Z E               area:                    Area ID: test\nDescription: test\nProjection ID:...),
2023-11-24T19:30:48.2376722Z E            call(Area ID: test
2023-11-24T19:30:48.2377198Z E           Description: test
2023-11-24T19:30:48.2377664Z E           Projection ID: test
2023-11-24T19:30:48.2379206Z E           Projection: {'datum': 'WGS84', 'k_0': '1', 'lat_0': '25', 'lat_1': '25', 'lon_0': '-95', 'no_defs': 'None', 'proj': 'lcc', 'type': 'crs', 'units': 'm', 'x_0': '0', 'y_0': '0'}
2023-11-24T19:30:48.2380397Z E           Number of columns: 6
2023-11-24T19:30:48.2380876Z E           Number of rows: 6
2023-11-24T19:30:48.2382352Z E           Area extent: (-1000.0, -300.0, 200.0, 1500.0), (slice(0, 6, None), slice(0, 6, None)), <xarray.DataArray 'zeros_like-8a3a336bb9ab815d7f114618174501b1' (y: 10, x: 10)>
2023-11-24T19:30:48.2384069Z E           dask.array<zeros_like, shape=(10, 10), dtype=float64, chunksize=(10, 10), chunktype=numpy.ndarray>
2023-11-24T19:30:48.2385050Z E           Dimensions without coordinates: y, x
2023-11-24T19:30:48.2385411Z E           Attributes:
2023-11-24T19:30:48.2385774Z E               _satpy_id:               DataID(name='comp19_big')
2023-11-24T19:30:48.2386174Z E               name:                    comp19_big
2023-11-24T19:30:48.2386721Z E               prerequisites:           [DataQuery(name='ds5', modifiers=('res_change',)...
2023-11-24T19:30:48.2387203Z E               optional_prerequisites:  []
2023-11-24T19:30:48.2387544Z E               optional_datasets:       []
2023-11-24T19:30:48.2387992Z E               area:                    Area ID: test\nDescription: test\nProjection ID:...),
2023-11-24T19:30:48.2388423Z E            call(Area ID: test
2023-11-24T19:30:48.2388707Z E           Description: test
2023-11-24T19:30:48.2388981Z E           Projection ID: test
2023-11-24T19:30:48.2390027Z E           Projection: {'datum': 'WGS84', 'k_0': '1', 'lat_0': '25', 'lat_1': '25', 'lon_0': '-95', 'no_defs': 'None', 'proj': 'lcc', 'type': 'crs', 'units': 'm', 'x_0': '0', 'y_0': '0'}
2023-11-24T19:30:48.2390725Z E           Number of columns: 3
2023-11-24T19:30:48.2391013Z E           Number of rows: 3
2023-11-24T19:30:48.2391861Z E           Area extent: (-1000.0, -300.0, 200.0, 1500.0), (slice(0, 3, None), slice(0, 3, None)), <xarray.DataArray 'zeros_like-46f66d07f783c1a970c25af2c11c94bd' (y: 20, x: 20,
2023-11-24T19:30:48.2392688Z E                                                                            bands: 3)>
2023-11-24T19:30:48.2393330Z E           dask.array<zeros_like, shape=(20, 20, 3), dtype=float64, chunksize=(20, 20, 3), chunktype=numpy.ndarray>
2023-11-24T19:30:48.2393892Z E           Coordinates:
2023-11-24T19:30:48.2394213Z E             * bands    (bands) <U1 'R' 'G' 'B'
2023-11-24T19:30:48.2394587Z E           Dimensions without coordinates: y, x
2023-11-24T19:30:48.2394918Z E           Attributes:
2023-11-24T19:30:48.2395285Z E               _satpy_id:               DataID(name='comp19_copy')
2023-11-24T19:30:48.2395691Z E               name:                    comp19_copy
2023-11-24T19:30:48.2396245Z E               prerequisites:           [DataQuery(name='ds5', modifiers=('res_change',)...
2023-11-24T19:30:48.2396721Z E               optional_prerequisites:  []
2023-11-24T19:30:48.2397069Z E               optional_datasets:       []
2023-11-24T19:30:48.2397535Z E               area:                    Area ID: test\nDescription: test\nProjection ID:...)]

I expect the first 2 as they are an area definition of size 3x3 and then a "high resolution" version of that area of size 6x6. As far as I can tell the 3rd one should not result in a call as it should be equivalent to the first 3x3 area.

@djhoese
Copy link
Member

djhoese commented Nov 26, 2023

Nevermind, no, the 3 calls to _slice_data are expected. There should only be 2 calls to the get_area_slices so that check should fail next after we fix the slice_data one. Let me see...

Copy link

codecov bot commented Nov 26, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (d250a9d) 95.21% compared to head (43ab244) 95.21%.
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2648   +/-   ##
=======================================
  Coverage   95.21%   95.21%           
=======================================
  Files         356      356           
  Lines       51588    51590    +2     
=======================================
+ Hits        49117    49119    +2     
  Misses       2471     2471           
Flag Coverage Δ
behaviourtests 4.22% <0.00%> (-0.01%) ⬇️
unittests 95.83% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@djhoese djhoese changed the title Use assert_called_once rather than called_once in tests Fix assert_called_once usage in resample tests Nov 26, 2023
@djhoese djhoese merged commit 684f364 into pytroll:main Nov 26, 2023
18 of 19 checks passed
@djhoese
Copy link
Member

djhoese commented Nov 26, 2023

Thanks for identifying this issue and getting things started. Sorry I took over and didn't really work with you on it, but I started getting confused by what the tests were expecting so I had to keep iterating on it so it made sense to me before I was comfortable merging. Thanks again.

@ArrayBolt3
Copy link
Contributor Author

Thanks for your work! And I don't mind you "taking over" at all - I'm just trying to help maintain the package in Ubuntu and noticed this causing a build faliure. You're far more equipped to be able to fix this sort of thing than I am. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug cleanup Code cleanup but otherwise no change in functionality component:tests
Projects
None yet
2 participants