Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add another cleaning method in cleaning.py #1672

Merged

Conversation

satoshifukami0115
Copy link
Contributor

I tried to implement a new image cleaning method used in MAGIC.
Distributions of a few Hillas parameters are checked in the link below.
https://indico.cta-observatory.org/event/3368/contributions/28400/attachments/19171/26360/CTA_ASWG_20210331_Fukami.pdf

I'm not sure if this method is better than other methods so far, but I think we can have an additional method as one option.
If you have any suggestions or comments, please let me know.

ctapipe/image/cleaning.py Outdated Show resolved Hide resolved
ctapipe/image/cleaning.py Outdated Show resolved Hide resolved
ctapipe/image/cleaning.py Outdated Show resolved Hide resolved
ctapipe/image/cleaning.py Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Mar 31, 2021

Codecov Report

Merging #1672 (ad5a2ae) into master (0b17d4c) will increase coverage by 0.07%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1672      +/-   ##
==========================================
+ Coverage   91.97%   92.04%   +0.07%     
==========================================
  Files         189      189              
  Lines       14910    14877      -33     
==========================================
- Hits        13713    13694      -19     
+ Misses       1197     1183      -14     
Impacted Files Coverage Δ
ctapipe/image/cleaning.py 96.84% <100.00%> (+1.45%) ⬆️
ctapipe/image/tests/test_cleaning.py 100.00% <100.00%> (ø)
ctapipe/reco/hillas_intersection.py 93.67% <0.00%> (-2.08%) ⬇️
ctapipe/reco/hillas_reconstructor.py 98.38% <0.00%> (-0.09%) ⬇️
ctapipe/reco/impact.py 44.98% <0.00%> (ø)
ctapipe/reco/__init__.py 100.00% <0.00%> (ø)
ctapipe/reco/shower_processor.py 100.00% <0.00%> (ø)
ctapipe/reco/tests/test_shower_processor.py 100.00% <0.00%> (ø)
ctapipe/reco/tests/test_hillas_intersection.py 100.00% <0.00%> (+2.79%) ⬆️
ctapipe/reco/reco_algorithms.py 96.96% <0.00%> (+5.30%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0b17d4c...ad5a2ae. Read the comment docs.

ctapipe/image/cleaning.py Outdated Show resolved Hide resolved
@maxnoe
Copy link
Member

maxnoe commented Apr 16, 2021

@satoshifukami0115 The brightest_island function is now merged and you can update here to use it.

@satoshifukami0115
Copy link
Contributor Author

satoshifukami0115 commented Apr 16, 2021

@maxnoe Thank you! I changed the part to use this function.

@maxnoe
Copy link
Member

maxnoe commented Apr 7, 2022

This looks good, only a test is missing @satoshifukami0115

There is also be a corresponding ImageCleaner class, so this cleaning becomes usable for the command line tools like ctapipe-process

@maxnoe maxnoe added this to the v0.14.0 milestone Apr 7, 2022
@satoshifukami0115
Copy link
Contributor Author

Thank you @maxnoe. The test means to include a test function in this file, right?
https://github.com/cta-observatory/ctapipe/blob/master/ctapipe/image/tests/test_cleaning.py

I will also work on the Cleaner class within this week and let you know once done.

@maxnoe
Copy link
Member

maxnoe commented Apr 7, 2022

@satoshifukami0115 Yes, exactly. Add a new test function there that tests if this cleaning works as intended, preferably by including some hand-crafted dummy images where you know which pixels should survive or not.

Copy link
Contributor

@kosack kosack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. Could you please also run black to re-format the code? Actually the correct method should be to install the git commit hook by first running this as shown in the Getting Started for Developers guide for ctapipe:

pre-commit install

After that, it should be automatically formatted when you commit. (you may have some commit failures if the code isn't already formatted, so just add/commit twice in that case)

Apply MAGIC-like image cleaning with timing information. See `ImageCleaner.__call__()`
"""

return mars_cleaning_1st_pass(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really want to run this function here, and not your time_constrained_clean() function?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this I should have changed to time_constrained_clean(). I will do it now...

@maxnoe maxnoe modified the milestones: v0.14.0, v0.15.0 Apr 13, 2022
`image[~mask] = 0`

"""
pixels_to_remove = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is an unused variable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is. I will remove this line...

mask_boundary = ( pixels_above_boundary & pixels_with_picture_neighbors ) & np.invert(mask_core)

# keep boundary pixels whose arrival times are within a certain time limit of the neighboring core pixels
pixels_to_remove = []
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here: this is an unused variable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, same here.

@satoshifukami0115
Copy link
Contributor Author

looks good. Could you please also run black to re-format the code? Actually the correct method should be to install the git commit hook by first running this as shown in the Getting Started for Developers guide for ctapipe:

pre-commit install

After that, it should be automatically formatted when you commit. (you may have some commit failures if the code isn't already formatted, so just add/commit twice in that case)

@kosack Thank you for your review. I did not run pre-commit. I will do it and push the branch again.

@maxnoe
Copy link
Member

maxnoe commented Apr 13, 2022

The pyflakes check is also complaining about some unused variables:

Run pyflakes ctapipe
ctapipe/image/cleaning.py:277:5 local variable 'pixels_to_remove' is assigned to but never used
ctapipe/image/cleaning.py:449:5 local variable 'pixels_to_remove' is assigned to but never used
ctapipe/image/tests/test_cleaning.py:1:1 'tkinter.Image' imported but unused
Error: Process completed with exit code 1.

@satoshifukami0115
Copy link
Contributor Author

The pyflakes check is also complaining about some unused variables:

Run pyflakes ctapipe
ctapipe/image/cleaning.py:277:5 local variable 'pixels_to_remove' is assigned to but never used
ctapipe/image/cleaning.py:449:5 local variable 'pixels_to_remove' is assigned to but never used
ctapipe/image/tests/test_cleaning.py:1:1 'tkinter.Image' imported but unused
Error: Process completed with exit code 1.

Yes, it seems that the tkinter module is not used in ctapipe/image/tests/test_cleaning.py.
I will remove this line.

@satoshifukami0115
Copy link
Contributor Author

I have performed the modifications you suggested and also confirmed that make doc worked. Please let me know if you have additional requests or comments. Thank you again for your comments so far.

@maxnoe
Copy link
Member

maxnoe commented Jul 5, 2022

@satoshifukami0115 Sorry for the late reply, this looks good know. Feel free to ping me the next time something is ready for review and I am not reacting for so long.

@maxnoe maxnoe merged commit 58b3c25 into cta-observatory:master Jul 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants