Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

model fitting tweaks and cleanup #1947

Merged
merged 6 commits into from
Dec 29, 2022

Conversation

kecnry
Copy link
Member

@kecnry kecnry commented Dec 21, 2022

Description

This pull request cleans up some duplicating caching and subset handling in model fitting while working on the solution to #1910 (whose actual fix is blocked by significant subset refactoring). It also improves on the intializer of Linear1D to estimate both the slope and intercept.

Change log entry

  • Is a change log needed? If yes, is it added to CHANGES.rst? If you want to avoid merge conflicts,
    list the proposed change log here for review and add to CHANGES.rst before merge. If no, maintainer
    should add a no-changelog-entry-needed label.

Checklist for package maintainer(s)

This checklist is meant to remind the package maintainer(s) who will review this pull request of some common things to look for. This list is not exhaustive.

  • Are two approvals required? Branch protection rule does not check for the second approval. If a second approval is not necessary, please apply the trivial label.
  • Do the proposed changes actually accomplish desired goals? Also manually run the affected example notebooks, if necessary.
  • Do the proposed changes follow the STScI Style Guides?
  • Are tests added/updated as required? If so, do they follow the STScI Style Guides?
  • Are docs added/updated as required? If so, do they follow the STScI Style Guides?
  • Did the CI pass? If not, are the failures related?
  • Is a milestone set? Set this to bugfix milestone if this is a bug fix and needs to be released ASAP; otherwise, set this to the next major release milestone.
  • After merge, any internal documentations need updating (e.g., JIRA, Innerspace)?

@kecnry kecnry force-pushed the model-fitting-cleanup branch from e4b8074 to 291c32d Compare December 21, 2022 20:16
@kecnry kecnry added this to the 3.2 milestone Dec 21, 2022
@kecnry kecnry added the plugin Label for plugins common to multiple configurations label Dec 21, 2022
Comment on lines +17 to +18
- Linear1D model component now estimates slope and intercept. [#1947]

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all the other changes are just internal and don't affect users - do we still want those listed in the change log somewhere?

@kecnry kecnry force-pushed the model-fitting-cleanup branch from 291c32d to b12718f Compare December 21, 2022 20:33
* dataset and subset components handle caching on their own, we don't need other caching at the plugin level
* we now apply the subset with a mask and so don't need to cache and pass the window along to specutils
@kecnry kecnry force-pushed the model-fitting-cleanup branch from b12718f to d72736a Compare December 21, 2022 20:55
@kecnry kecnry marked this pull request as ready for review December 21, 2022 21:02
@codecov
Copy link

codecov bot commented Dec 21, 2022

Codecov Report

Base: 91.79% // Head: 91.79% // Increases project coverage by +0.00% 🎉

Coverage data is based on head (3f7e9c2) compared to base (20b224e).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1947   +/-   ##
=======================================
  Coverage   91.79%   91.79%           
=======================================
  Files         140      140           
  Lines       14950    14943    -7     
=======================================
- Hits        13723    13717    -6     
+ Misses       1227     1226    -1     
Impacted Files Coverage Δ
...figs/default/plugins/model_fitting/initializers.py 97.64% <100.00%> (ø)
...igs/default/plugins/model_fitting/model_fitting.py 83.71% <100.00%> (-0.16%) ⬇️
...default/plugins/model_fitting/tests/test_plugin.py 100.00% <100.00%> (ø)
jdaviz/configs/imviz/plugins/viewers.py 88.83% <0.00%> (+0.05%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Copy link
Contributor

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need a change log for things users don't care about.

I also don't know how I should feel that no tests are affected by this.

@pllim
Copy link
Contributor

pllim commented Dec 23, 2022

I hardly use this plugin, so please tell me how I should review this in terms of usage workflow, or other devs can do it... 😬

@kecnry
Copy link
Member Author

kecnry commented Dec 28, 2022

I don't think we need a change log for things users don't care about.

Ok, agreed. I did add another bugfix entry since this fixes parameter initialization to handle the selected subset correctly.

I also don't know how I should feel that no tests are affected by this.

Model fitting has decent coverage already and this really mostly removes redundant code that has no affect on those tests.

I hardly use this plugin, so please tell me how I should review this in terms of usage workflow, or other devs can do it

There are only two things that should change when using the model fitting plugin when compared to main:

  • creating a model component should now be influenced by the selected subset (create a linear component with the entire spectrum and then a spectral subset... its now a much more reasonable starting point).
  • creating a linear model component should now estimate both slope and intercept, whereas before the intercept was set based on the median value which often was not a great assumption.

@@ -683,15 +659,15 @@ def _fit_model_to_spectrum(self, add_data):
models_to_fit = self._reinitialize_with_fixed()

# Apply masks from selected spectral subsets
self._apply_subset_masks(self._spectrum1d, self.spectral_subset)
masked_spectrum = self._apply_subset_masks(self.dataset.selected_obj, self.spectral_subset)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you intend to return a subset masked copy of self.dataset.selected_obj as masked_spectrum? If so, I think this might not do exactly what you're expecting, since the _apply_subset_masks method is still updating or creating a mask attribute on self.dataset.selected_obj.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.dataset.selected_obj is a cached property, so I think this is safe (but since its cached, I guess maybe that could cause an issue). Do you think I need to manually deepcopy somewhere to prevent multiple masks from being applied on top of each other? Can you reproduce a case where that happens?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, you're right, this does cause a problem. I'll push an update that makes a deepcopy within _apply_subset_masks.

otherwise subsets will just be applied on top of each other instead of the original non-masked spectrum
@kecnry kecnry force-pushed the model-fitting-cleanup branch from 141b62a to d356521 Compare December 28, 2022 17:03
Copy link
Contributor

@pllim pllim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't grok the internals, but linear estimate seems to work as advertised. And the tests pass. So approval from me. Thanks!

Copy link
Collaborator

@rosteen rosteen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. The estimates being based on the selected subset makes things so much easier, I was able to fit a gaussian successfully without updating any parameters from the initial estimates, which is a great change.

@kecnry kecnry merged commit 66adb92 into spacetelescope:main Dec 29, 2022
@kecnry kecnry deleted the model-fitting-cleanup branch December 29, 2022 17:28
kecnry added a commit to kecnry/jdaviz that referenced this pull request Jan 3, 2023
kecnry added a commit to kecnry/jdaviz that referenced this pull request Jan 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cubeviz plugin Label for plugins common to multiple configurations Ready for final review specviz
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants