Add method to read bright star catalog and find bright stars in sky region #1105

moritzhuetten · 2019-07-12T23:46:59Z

added a new 'astro' class in ctapipe.utils with a get_bright_stars() method, which returns the Yale bright star catalog (read from ctapipe-extra) into an astropy table, or with arguments
get_bright_stars(pointing = astropy.SkyCoord(), radius = astropy.Angle(), magnitude_cut = float)
selects stars brighter than the chosen magnitude in a region of given radius around a sky position

codecov · 2019-07-13T00:04:27Z

Codecov Report

Merging #1105 into master will increase coverage by 0.05%.
The diff coverage is 96.87%.

@@            Coverage Diff             @@
##           master    #1105      +/-   ##
==========================================
+ Coverage   84.48%   84.53%   +0.05%     
==========================================
  Files         182      184       +2     
  Lines       11220    11274      +54     
==========================================
+ Hits         9479     9531      +52     
- Misses       1741     1743       +2

Impacted Files	Coverage Δ
ctapipe/utils/__init__.py	`100% <100%> (ø)`	⬆️
ctapipe/utils/tests/test_astro.py	`100% <100%> (ø)`
ctapipe/utils/astro.py	`95.65% <95.65%> (ø)`
ctapipe/tools/info.py	`82.17% <0%> (-1%)`	⬇️
ctapipe/image/tests/test_cleaning.py	`100% <0%> (ø)`	⬆️
ctapipe/image/cleaning.py	`96.29% <0%> (+0.06%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62df71b...073be5b. Read the comment docs.

ctapipe/utils/astro.py

ctapipe/utils/__init__.py

ctapipe/utils/astro.py

maxnoe · 2019-07-15T17:19:34Z

ctapipe/utils/__init__.py

@@ -6,6 +6,7 @@
 from .unstructured_interpolator import UnstructuredInterpolator
 from .datasets import (find_all_matching_datasets, get_table_dataset, get_dataset_path,
                       find_in_path)
+from .astro import get_bright_stars


Add this method to __all__

This is still missing!

This is still missing!

Now added

ctapipe/utils/astro.py

maxnoe

ok, so what's now missing is a test.

E.g. just loading the full catalog, loading a catalog with radius and pointing (e.g. pointing is crab and check that ceta tauri is there) and max magnitude (e.g. 3.5 and check that only zeta tauri is there)

kosack

Thanks for all the changes! Looks good now. The only think missing is that it does not appear in the API documentation anywhere, so you need to add something to docs/ctapipe_api/utils/index.rst (perhaps just a section on the new astro module and an automodapi directive for it). The easiest thing is to add a docstring at the top of the astro.py with some overall explaination of the module, and just add:

.. automodapi:: ctapipe.utils.astro
    :no-inheritance-diagram:

under the "Reference/API" section of the index.rst.

Then run "make doc" to see if it looks ok.

kosack

One more request: besides the documentation, you also need to add a unit test in utils/tests. So far no test covers this function.

maxnoe · 2019-07-16T11:11:00Z

Sorry, one more thing. I have a distaste for submodules called utils . They tend to acquire random stuff over time.

Is there a better place for this? E.g. the coordinates submodule?

moritzhuetten · 2019-07-16T11:19:01Z

Sorry, one more thing. I have a distaste for submodules called utils . They tend to acquire random stuff over time.

Is there a better place for this? E.g. the coordinates submodule?

Fine with me, if you prefer (or any other place). In any case, my idea for an 'astro' submodule was to gather there all astronomy-related stuff (which astropy or other modules) can not provide. For example, I discussed with @noahhdf last week whether potentially this could also a place to add some dark-patch-finder or selector (according to day, moon distance,...) or similar helpers. But that's another story, of course.

maxnoe · 2019-07-16T12:32:42Z

dark-patch-finder or selector (according to day, moon distance,...) or similar helpers. But that's another story, of course.

I build something like this some time ago for FACT using the hipparcos star catalogue, but haven't used it in at least two years since e don't do ratescans where the shifter selects the spot anymore.

https://github.com/fact-project/darkspot

It uses ephem and self coded transformation, wasn't used to astropy at the time.

kosack · 2019-07-16T13:29:57Z

We can probably make another issue or PR to deprecate ctapipe.utils and move the things inside to new places. I agree "utils" is really a catch-all for unrelated things.

kosack · 2019-07-16T13:30:29Z

I updated ctapipe-extra and released a new package v0.2.18 that should work with your tests. Let's see if it works.

ctapipe/utils/tests/test_astro.py

moritzhuetten · 2019-07-16T14:43:21Z

I updated ctapipe-extra and released a new package v0.2.18 that should work with your tests. Let's see if it works.

Hi Karl, thanks a lot!
however, when Travis runs it fails with

>       table = get_bright_stars(pointing, radius=2. * u.deg, magnitude_cut=3.5)
/home/travis/build/cta-observatory/ctapipe/ctapipe/utils/tests/test_astro.py:17: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/travis/build/cta-observatory/ctapipe/ctapipe/utils/astro.py:40: in get_bright_stars
    catalog = get_dataset_path("yale_bright_star_catalog5.fits.gz")
/home/travis/build/cta-observatory/ctapipe/ctapipe/utils/datasets.py:143: in get_dataset_path
    return ctapipe_resources.get(filename)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
resource_name = 'yale_bright_star_catalog5.fits.gz'
    def get(resource_name):
        """ get the filename for a resource """
        if not pkg_resources.resource_exists(__name__, resource_name):
            raise FileNotFoundError("Couldn't find resource: '{}'"
>                                   .format(resource_name))
E           FileNotFoundError: Couldn't find resource: 'yale_bright_star_catalog5.fits.gz'

Any idea why it doesn't find the file?

Thanks and cheers
Moritz

maxnoe · 2019-07-16T14:44:07Z

You need to increase the version number of ctapipe-extra in the yml environment files.

ctapipe/utils/tests/test_astro.py

kosack · 2019-07-16T14:54:40Z

yes, just update environment.yml, and py3.6_env.yml, py3.7_env.yml and change the line with ctapipe-extra to:

 - ctapipe-extra=0.2.18

That should fix it.

ctapipe/utils/astro.py

clicked approve by mistake - please see comments

kosack

use get_table_dataset() (see comments above)

kosack · 2019-07-16T15:18:57Z

Once you update the yml files it should be fine.

moritzhuetten · 2019-07-16T15:19:21Z

Once you update the yml files it should be fine.

And I am just updating the API doc...

moritzhuetten · 2019-07-16T15:38:25Z

And I am just updating the API doc...

Should be done now!

maxnoe · 2019-07-16T16:01:33Z

You forgot the two other yaml files (py3.6 and py3.7), unfortunately we need all three of them

moritzhuetten · 2019-07-16T16:26:56Z

You forgot the two other yaml files (py3.6 and py3.7), unfortunately we need all three of them

Added!

codecov-io · 2019-07-16T19:07:43Z

Codecov Report

Merging #1105 into master will increase coverage by 1.19%.
The diff coverage is 96.66%.

@@            Coverage Diff             @@
##           master    #1105      +/-   ##
==========================================
+ Coverage   84.48%   85.67%   +1.19%     
==========================================
  Files         182      185       +3     
  Lines       11220    11451     +231     
==========================================
+ Hits         9479     9811     +332     
+ Misses       1741     1640     -101

Impacted Files	Coverage Δ
ctapipe/utils/__init__.py	`100% <100%> (ø)`	⬆️
ctapipe/utils/tests/test_astro.py	`100% <100%> (ø)`
ctapipe/utils/astro.py	`95% <95%> (ø)`
ctapipe/reco/tests/test_hillas_intersection.py	`96.72% <0%> (-3.28%)`	⬇️
ctapipe/utils/unstructured_interpolator.py	`62.85% <0%> (-1.85%)`	⬇️
ctapipe/tools/info.py	`82.17% <0%> (-1%)`	⬇️
ctapipe/image/tests/test_cleaning.py	`100% <0%> (ø)`	⬆️
ctapipe/reco/tests/test_reconstruction_methods.py	`90.9% <0%> (ø)`
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 62df71b...886588c. Read the comment docs.

ctapipe/utils/__init__.py

* upstream/master: Reduce numba signatures to improve import time (cta-observatory#1108) Remove unused utils (cta-observatory#1112) Add method to read bright star catalog and find bright stars in sky region (cta-observatory#1105) Add Fields to calibration containers (cta-observatory#1111) Reconstruction fix (also for Divergent Pointing) (cta-observatory#946) updated AUTHORS with latest mailmap

moritzhuetten added 2 commits July 13, 2019 00:23

get bright star table

402e1ae

renamed bright star catalog file to read from ctapipe-extra

346c3f4

improved code quality

2ba85a5

kosack requested changes Jul 15, 2019

View reviewed changes

ctapipe/utils/astro.py Outdated Show resolved Hide resolved

maxnoe requested changes Jul 15, 2019

View reviewed changes

requested changed added

96c2b77

maxnoe requested changes Jul 15, 2019

View reviewed changes

ctapipe/utils/astro.py Outdated Show resolved Hide resolved

ctapipe/utils/astro.py Outdated Show resolved Hide resolved

style improvements

f4139b6

maxnoe reviewed Jul 15, 2019

View reviewed changes

ctapipe/utils/astro.py Outdated Show resolved Hide resolved

moritzhuetten added 2 commits July 15, 2019 17:53

Explicit error message for missing pointing input

7d57d25

coding style rules

245d6f5

maxnoe approved these changes Jul 15, 2019

View reviewed changes

maxnoe previously approved these changes Jul 15, 2019

View reviewed changes

maxnoe reviewed Jul 15, 2019

View reviewed changes

kosack requested changes Jul 16, 2019

View reviewed changes

moritzhuetten added 2 commits July 16, 2019 12:26

bugfix + better formatting

b214388

test added

371c631

moritzhuetten dismissed maxnoe’s stale review via 371c631 July 16, 2019 11:26

kosack reviewed Jul 16, 2019

View reviewed changes

ctapipe/utils/tests/test_astro.py Show resolved Hide resolved

moritzhuetten added 2 commits July 16, 2019 14:54

smaller radius in test

9df553e

import ctapipe_resources

e4edb5f

maxnoe reviewed Jul 16, 2019

View reviewed changes

ctapipe/utils/tests/test_astro.py Outdated Show resolved Hide resolved

kosack previously approved these changes Jul 16, 2019

View reviewed changes

ctapipe/utils/astro.py Outdated Show resolved Hide resolved

kosack requested changes Jul 16, 2019

View reviewed changes

moritzhuetten added 3 commits July 16, 2019 16:06

use get_table_dataset, clean up test file

b8e6f94

coding style

d9165fc

typo correction in comment

e489e6b

kosack previously approved these changes Jul 16, 2019

View reviewed changes

update environment.yml for ctapipe-extra

4a2824e

moritzhuetten dismissed kosack’s stale review via 4a2824e July 16, 2019 15:18

documentation added

8873d90

py 3.6 & 3.7 environment files updated

073be5b

Comply with codacy coding standards

83ae6a2

kosack previously approved these changes Jul 17, 2019

View reviewed changes

add get_bright_stars method to __all__

3f59b62

moritzhuetten dismissed kosack’s stale review via 3f59b62 July 17, 2019 10:13

maxnoe reviewed Jul 17, 2019

View reviewed changes

ctapipe/utils/__init__.py Outdated Show resolved Hide resolved

comma added

886588c

kosack approved these changes Jul 17, 2019

View reviewed changes

maxnoe approved these changes Jul 17, 2019

View reviewed changes

kosack merged commit 5a1a6d4 into cta-observatory:master Jul 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add method to read bright star catalog and find bright stars in sky region #1105

Add method to read bright star catalog and find bright stars in sky region #1105

moritzhuetten commented Jul 12, 2019

codecov bot commented Jul 13, 2019 •

edited

Loading

maxnoe Jul 15, 2019

maxnoe Jul 17, 2019

moritzhuetten Jul 17, 2019

maxnoe left a comment •

edited by dneise

Loading

kosack left a comment •

edited

Loading

kosack left a comment

maxnoe commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

maxnoe commented Jul 16, 2019 •

edited

Loading

kosack commented Jul 16, 2019

kosack commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

maxnoe commented Jul 16, 2019

kosack commented Jul 16, 2019

kosack left a comment

kosack commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

maxnoe commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

codecov-io commented Jul 16, 2019 •

edited by codecov bot

Loading

Add method to read bright star catalog and find bright stars in sky region #1105

Add method to read bright star catalog and find bright stars in sky region #1105

Conversation

moritzhuetten commented Jul 12, 2019

codecov bot commented Jul 13, 2019 • edited Loading

Codecov Report

maxnoe Jul 15, 2019

Choose a reason for hiding this comment

maxnoe Jul 17, 2019

Choose a reason for hiding this comment

moritzhuetten Jul 17, 2019

Choose a reason for hiding this comment

maxnoe left a comment • edited by dneise Loading

Choose a reason for hiding this comment

kosack left a comment • edited Loading

Choose a reason for hiding this comment

kosack left a comment

Choose a reason for hiding this comment

maxnoe commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

maxnoe commented Jul 16, 2019 • edited Loading

kosack commented Jul 16, 2019

kosack commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

maxnoe commented Jul 16, 2019

kosack commented Jul 16, 2019

kosack left a comment

Choose a reason for hiding this comment

kosack commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

maxnoe commented Jul 16, 2019

moritzhuetten commented Jul 16, 2019

codecov-io commented Jul 16, 2019 • edited by codecov bot Loading

Codecov Report

codecov bot commented Jul 13, 2019 •

edited

Loading

maxnoe left a comment •

edited by dneise

Loading

kosack left a comment •

edited

Loading

maxnoe commented Jul 16, 2019 •

edited

Loading

codecov-io commented Jul 16, 2019 •

edited by codecov bot

Loading