Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add resized area example to resampling documentation #2816

Merged
merged 4 commits into from
Jul 17, 2024

Conversation

ludwigVonKoopa
Copy link
Contributor

Adding a resize method in the Scene class, to resize dataset in the resolution wanted.
You specify a size (x, y) in pixels, and resample is done.

This make it people able to easily resize a scene, for example creating a small image to see it faster, when you don't need a full resolution image.

Please let me know if something is missing, or if I should add something else

@djhoese
Copy link
Member

djhoese commented Jun 14, 2024

How does this differ from the Scene.aggregate method usage: https://satpy.readthedocs.io/en/stable/api/satpy.scene.html#satpy.scene.Scene.aggregate which uses xarray's coarsen.

CC @mraspaud who implemented .aggregate if I remember correctly.

@ludwigVonKoopa
Copy link
Contributor Author

ludwigVonKoopa commented Jun 14, 2024

Correct me if i'm wrong, but aggregate method can only reduce the size by a int coefficient (i.e. scene.aggregate(func="mean", x=2, y=2) which don't allow for example choosing an float coefficient or an exact size (lets say you need a 256x256 image, so you have to delete some pixels to make it fit the wanted size)

Copy link

codecov bot commented Jun 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.95%. Comparing base (440925d) to head (f73d408).
Report is 378 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2816      +/-   ##
==========================================
- Coverage   95.96%   95.95%   -0.02%     
==========================================
  Files         366      366              
  Lines       53593    53619      +26     
==========================================
+ Hits        51433    51450      +17     
- Misses       2160     2169       +9     
Flag Coverage Δ
behaviourtests 4.04% <ø> (-0.01%) ⬇️
unittests 96.05% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coveralls
Copy link

coveralls commented Jun 14, 2024

Pull Request Test Coverage Report for Build 9498919442

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 29 of 29 (100.0%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.008%) to 96.048%

Totals Coverage Status
Change from base Build 9497248893: 0.008%
Covered Lines: 51593
Relevant Lines: 53716

💛 - Coveralls

@djhoese
Copy link
Member

djhoese commented Jun 17, 2024

Yes, it looks like you are correct. Good point.

I'm torn on how I feel about this, but I'm leaning towards not wanting this added for a couple reasons:

  1. This is only a small convenience in my opinion. The logic comes down to 2 Scene methods (finest_area and resample) and a helper on the AreaDefinition (.copy). In general this wouldn't be a huge problem except for my next point...
  2. This adds more methods to the Scene which I'd like to avoid if at all possible. As Satpy goes forward and more and more features get added to xarray, we (myself at least) would like the Scene to be come a thin wrapper around xarray's DataTree if not completely replaced by it in future major releases of Satpy. More methods on the Scene means more backwards incompatible changes if we strive for this xarray object based approach in the future and more need for equivalent workflows for users to transition to.
  3. We generally regret our usage of **kwargs where it is used in Satpy. It makes things hard to debug and change. This also signals to me that the .resample call is unnecessary in this method as we're simply passing all arguments to the resample call and not using them otherwise.

The overall operation could be simplified to:

new_scn = scn.resample(scn.finest_area().copy(height=x, width=y), ...)

Of course I am not the only maintainer of Satpy so I'd like to hear what some of the others think (@pytroll/satpy-core).

@ludwigVonKoopa
Copy link
Contributor Author

I understand. I was not aware of the DataTree relation with satpy.

The overall operation could be simplified to:

new_scn = scn.resample(scn.finest_area().copy(height=x, width=y), ...)

Good point.
Maybe instead of adding another method, we could just add this one-liner into the documentation in one of the examples, or the resample page.
It would not add another method, and we would still have this 'tips' somewhere (so we can link this example for sharing purposes, i'm thinking on issue #368).

Or maybe that's too specific, I'm quite new to Pytroll so I lack experience on this project!

@mraspaud
Copy link
Member

I agree with @djhoese that adding a new method isn't so nice, in particular because that class is already quite bloated. In the long run, I believe we will have something based off datatree and thus it would be best to have such operations (aggregation, decimation, etc)
in an accessor.
In the meanwhile, documentation about the one-liner is good.

@djhoese we could also start having accessors to the scene object already now, and just make sure they are forward-compatible with datatree, what do you think?

@djhoese
Copy link
Member

djhoese commented Jun 18, 2024

Yes, I'm ok with have an accessor like interface on the Scene or creating an xarray accessor. However, there is the same bloat issue if this method were added there.

@mraspaud
Copy link
Member

Yes, I'm ok with have an accessor like interface on the Scene or creating an xarray accessor. However, there is the same bloat issue if this method were added there.

The accessor I mentioned would be one of many, so that we can separate concerns a bit more. Eg one for spatial transformations (resizing, aggregation, resampling, etc), one for creating composites, etc. Hopefully it wouldn't be so bloated then...

@djhoese
Copy link
Member

djhoese commented Jun 18, 2024

We should maybe move this discussion to an issue. At some point accessors aren't the right approach and become bloat themselves. A more traditional separate set of functions or classes might be more obvious.

@mraspaud
Copy link
Member

We should maybe move this discussion to an issue.

yes

Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@mraspaud
Copy link
Member

I'm ok with merging this, but what is the consensus here? Should we just add to the documentation instead?

@ludwigVonKoopa
Copy link
Contributor Author

I'm ok with not adding another method to the Scene class and just adding a tips in the documentation, but where does it fit the best ?
I'm thinking either :

Maybe you have a better idea ?

@djhoese
Copy link
Member

djhoese commented Jun 24, 2024

What about a new section after this one:

https://satpy.readthedocs.io/en/latest/resample.html#create-custom-area-definition

Something like "Create reduced data area" or...ugh that's a bad section title. Something better than this that isn't too long/wordy.

@ludwigVonKoopa
Copy link
Contributor Author

Does this kind of example suits you ?

@coveralls
Copy link

coveralls commented Jul 1, 2024

Pull Request Test Coverage Report for Build 9740461782

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage remained the same at 96.046%

Totals Coverage Status
Change from base Build 9702977764: 0.0%
Covered Lines: 51641
Relevant Lines: 53767

💛 - Coveralls

Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments, but otherwise this seems pretty good.

satpy/resample.py Outdated Show resolved Hide resolved
Sometimes you may want to create a small image with fixed size in pixels.
For example, to create an image of (x, y) pixels :

>>> small_scn = scn.resample(scn.finest_area().copy(height=y, width=x), ...)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be worth mentioning that this won't always work with native resampling if it isn't a factor of the original and/or show resampler="nearest" in the code just to suggest that the resampler should be specified.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes i'm adding this in the example to be clearer

@ludwigVonKoopa
Copy link
Contributor Author

does this changes suits you ?

(some tests don't pass, but I don't think they are relative to my modifications as... i'm only modifying some comments)

FAILED satpy/tests/test_modifiers.py::TestPSPRayleighReflectance::test_rayleigh_corrector[B01-wavelength0-1000-rayleigh_only-70-95-1-41.540239-exp_unique0] - tarfile.ReadError: file could not be opened successfully

@ludwigVonKoopa
Copy link
Contributor Author

correction : after rebase with main, no more unsuccessful checks

@coveralls
Copy link

coveralls commented Jul 8, 2024

Pull Request Test Coverage Report for Build 9944227638

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 8 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-0.02%) to 96.054%

Files with Coverage Reduction New Missed Lines %
satpy/tests/utils.py 2 93.16%
satpy/tests/reader_tests/gms/test_gms5_vissr_l1b.py 3 98.67%
satpy/tests/reader_tests/gms/test_gms5_vissr_navigation.py 3 97.18%
Totals Coverage Status
Change from base Build 9842330086: -0.02%
Covered Lines: 51679
Relevant Lines: 53802

💛 - Coveralls

satpy/resample.py Outdated Show resolved Hide resolved
satpy/resample.py Outdated Show resolved Hide resolved
@djhoese djhoese changed the title add resize method for Scene class Add resized area example to resampling documentation Jul 15, 2024
@djhoese djhoese self-assigned this Jul 15, 2024
Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My warning got a little bigger than I expected, but I think it is more clear. Maybe I should have just linked to the information about the native resampler... 🤔

@djhoese
Copy link
Member

djhoese commented Jul 15, 2024

Maybe I should have just linked to the information about the native resampler...

Of course the native resampler documentation doesn't mention this 🤦‍♂️

https://satpy.readthedocs.io/en/stable/api/satpy.resample.html#satpy.resample.NativeResampler

@djhoese djhoese merged commit 4c4ce24 into pytroll:main Jul 17, 2024
18 of 19 checks passed
Copy link
Contributor

@adybbroe adybbroe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Add ability to set output image size for Scene.show
6 participants