Refactor file discovery #1839
Refactor file discovery #1839
Conversation
pypeit/spectrographs/gemini_gnirs.py
Outdated
| @@ -587,6 +587,7 @@ class GNIRSIFUSpectrograph(GeminiGNIRSSpectrograph): | |||
| # * Need to store a wavelength solution for different grating options (Note, the Holy Grail algorithm works pretty well, most of the time) | |||
| name = 'gemini_gnirs_ifu' | |||
| pypeline = 'SlicerIFU' | |||
| allowed_extensions = ['.fits', '.fits.bz2'] | |||
There was a problem hiding this comment.
What happens is the user has a set of fits files in their Raw/ directory, and also the exact same files but compressed?
There was a problem hiding this comment.
This is consistent with the previous behavior: the code would find both. At least now, users can limit the search to one or the other using the --extension command-line argument. I added some text in the setup doc about this.
|
|
||
| if isinstance(raw_path, list): | ||
| return numpy.concatenate([files_from_extension(p, extension=extension) for p in raw_path]).tolist() | ||
| files = numpy.concatenate([files_from_extension(p, extension=extension) for p in raw_path]) | ||
| return numpy.unique(files).tolist() |
There was a problem hiding this comment.
I think we possibly need to check if there are strings present in this list that are identical, except for the extension? I would be doubtful that anyone would carry around identical compressed and uncompressed files in the same directory, but then again, it's possible.
There was a problem hiding this comment.
I'd rather not over-complicate it. I think it's enough to warn people that the code will potentially find both compressed and uncompressed files if they're in the same directory, which has always been true.
There was a problem hiding this comment.
OK - sounds good to me! 👍
|
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## aat_uhrf #1839 +/- ##
============================================
+ Coverage 38.23% 38.25% +0.02%
============================================
Files 209 209
Lines 48544 48562 +18
============================================
+ Hits 18560 18578 +18
Misses 29984 29984 ☔ View full report in Codecov by Sentry. |
|
I've started a dev-suite run for this branch. |
|
The tests are passing (except the HIRES failures), fixed in #1833. |
# Conflicts: # doc/releases/1.16.1dev.rst
Prompted by @rcooke-ast's changes to the file search functionality, I've done a couple of things:
.fitsor.fits.gz, but is overwritten for a few spectrographs.There's also an associated dev-suite PR that adds a unit test for this. Both PRs are directed into the relevant AAT/UHRF PRs.