Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor file discovery #1839

Merged
merged 16 commits into from
Aug 26, 2024
8 changes: 8 additions & 0 deletions doc/api/pypeit.scripts.extract_datacube.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
pypeit.scripts.extract\_datacube module
=======================================

.. automodule:: pypeit.scripts.extract_datacube
:members:
:private-members:
:undoc-members:
:show-inheritance:
1 change: 1 addition & 0 deletions doc/api/pypeit.scripts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Submodules
pypeit.scripts.compare_sky
pypeit.scripts.compile_wvarxiv
pypeit.scripts.edge_inspector
pypeit.scripts.extract_datacube
pypeit.scripts.flux_calib
pypeit.scripts.flux_setup
pypeit.scripts.identify
Expand Down
8 changes: 8 additions & 0 deletions doc/api/pypeit.spectrographs.aat_uhrf.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
pypeit.spectrographs.aat\_uhrf module
=====================================

.. automodule:: pypeit.spectrographs.aat_uhrf
:members:
:private-members:
:undoc-members:
:show-inheritance:
1 change: 1 addition & 0 deletions doc/api/pypeit.spectrographs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Submodules
.. toctree::
:maxdepth: 4

pypeit.spectrographs.aat_uhrf
pypeit.spectrographs.bok_bc
pypeit.spectrographs.gemini_flamingos
pypeit.spectrographs.gemini_gmos
Expand Down
8 changes: 4 additions & 4 deletions doc/coadd3d.rst
Original file line number Diff line number Diff line change
Expand Up @@ -155,7 +155,7 @@ There are several recommended steps of the coadd3d process that can be run separ

pypeit_sensfunc spec1d_StandardStarName.fits -o sens_StandardStarName.fits

For further details, see :doc:`_sensitivity_function`.
For further details, see :ref:`sensitivity_function`.

#. Step 4 - Generate a datacube of the science exposures. This is done by running the following command:

Expand All @@ -164,7 +164,7 @@ There are several recommended steps of the coadd3d process that can be run separ
pypeit_coadd_datacube ScienceName.coadd3d -o

Note that you will need to specify the sensitivity function file using the ``sensfile`` option in the
:doc:`coadd3d_file` file. For further details, see :ref:`coadd3d_fluxing`.
:ref:`coadd3d_file` file. For further details, see :ref:`coadd3d_fluxing`.

Combination options
===================
Expand Down Expand Up @@ -239,7 +239,7 @@ If you would like to flux calibrate your datacube, you need to
produce your standard star datacube first. Then extract the spectrum
of the standard star using the ``pypeit_extract_datacube`` script. This
will produce a ``spec1d`` file that you will need to use to generate a
sensitivity function in the usual way (see :doc:`_sensitivity_function`).
sensitivity function in the usual way (see :ref:`sensitivity_function`).
Then, when generating the datacube of the science frame you must include
the name of the sensitivity function in your ``coadd3d`` file as follows:

Expand Down Expand Up @@ -289,7 +289,7 @@ then you can specify the ``skysub_frame`` in the ``spec2d`` block of the
above. If you have dedicated sky frames, then it is generally
recommended to reduce these frames as if they are regular science
frames, but add the following keyword arguments at the top of your
:doc:`coadd3d_file`:
:ref:`coadd3d_file`:

.. code-block:: ini

Expand Down
3 changes: 2 additions & 1 deletion doc/cookbook.rst
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,8 @@ what we recommend:
- We will refer to that folder as ``RAWDIR``

The raw images can be gzip-compressed, although this means opening files will be
slower.
slower. See :ref:`setup-file-searching` for specific comments about the files
in your raw directory.

A word on calibration data
--------------------------
Expand Down
2 changes: 1 addition & 1 deletion doc/help/pypeit_cache_github_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
Script to download/cache PypeIt github data

positional arguments:
spectrograph A valid spectrograph identifier: bok_bc,
spectrograph A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
Expand Down
9 changes: 6 additions & 3 deletions doc/help/pypeit_chk_for_calibs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
options:
-h, --help show this help message and exit
-s SPECTROGRAPH, --spectrograph SPECTROGRAPH
A valid spectrograph identifier: bok_bc,
A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
Expand All @@ -35,8 +35,11 @@
vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
wht_isis_blue, wht_isis_red (default: None)
-e EXTENSION, --extension EXTENSION
File extension; compression indicators (e.g. .gz) not
required. (default: .fits)
File extension to use. Must include the period (e.g.,
".fits") and it must be one of the allowed extensions
for this spectrograph. If None, root directory will be
searched for all files with any of the allowed
extensions. (default: None)
--save_setups If not toggled, remove setup_files/ folder and its
files. (default: False)

29 changes: 29 additions & 0 deletions doc/help/pypeit_extract_datacube.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
.. code-block:: console

$ pypeit_extract_datacube -h
usage: pypeit_extract_datacube [-h] [-e EXT_FILE] [-s SAVE] [-o]
[-b BOXCAR_RADIUS] [-v VERBOSITY]
file

Read in a datacube, extract a spectrum of a point source,and save it as a spec1d
file.

positional arguments:
file spec3d.fits DataCube file

options:
-h, --help show this help message and exit
-e EXT_FILE, --ext_file EXT_FILE
Configuration file with extraction parameters (default:
None)
-s SAVE, --save SAVE Output spec1d filename (default: None)
-o, --overwrite Overwrite any existing files/directories (default:
False)
-b BOXCAR_RADIUS, --boxcar_radius BOXCAR_RADIUS
Radius of the circular boxcar (in arcseconds) to use for
the extraction. (default: None)
-v VERBOSITY, --verbosity VERBOSITY
Verbosity level between 0 [none] and 2 [all]. Default:
1. Level 2 writes a log with filename
extract_datacube_YYYYMMDD-HHMM.log (default: 1)

9 changes: 6 additions & 3 deletions doc/help/pypeit_obslog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
using PypeItMetaData.

positional arguments:
spec A valid spectrograph identifier: bok_bc,
spec A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
Expand Down Expand Up @@ -75,8 +75,11 @@
-s SORT, --sort SORT Metadata keyword (pypeit-specific) to use to sort the
output table. (default: mjd)
-e EXTENSION, --extension EXTENSION
File extension; compression indicators (e.g. .gz) not
required. (default: .fits)
File extension to use. Must include the period (e.g.,
".fits") and it must be one of the allowed extensions
for this spectrograph. If None, root directory will be
searched for all files with any of the allowed
extensions. (default: None)
-d OUTPUT_PATH, --output_path OUTPUT_PATH
Path to top-level output directory. (default: current
working directory)
Expand Down
2 changes: 1 addition & 1 deletion doc/help/pypeit_ql.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
Script to produce quick-look PypeIt reductions

positional arguments:
spectrograph A valid spectrograph identifier: bok_bc,
spectrograph A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
Expand Down
12 changes: 7 additions & 5 deletions doc/help/pypeit_sensfunc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,13 @@
-s SENS_FILE, --sens_file SENS_FILE
Configuration file with sensitivity function parameters
-f FLATFILE, --flatfile FLATFILE
Use the flat file for computing the sensitivity
function. Note that it is not possible to set
--flatfile and simultaneously use a .sens file with the
--sens_file option. If you are using a .sens file, set
the flatfile there via e.g.:
Use a flat calibration file to compute the blaze
function when generating the sensitivity function. This
is helpful to account for small scale undulations in the
sensitivity function. Note that it is not possible to
set --flatfile and simultaneously use a .sens file with
the --sens_file option. If you are using a .sens file,
set the flatfile there via e.g.:

[sensfunc]
flatfile = Calibrations/Flat_A_0_DET01.fits
Expand Down
9 changes: 6 additions & 3 deletions doc/help/pypeit_setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
options:
-h, --help show this help message and exit
-s SPECTROGRAPH, --spectrograph SPECTROGRAPH
A valid spectrograph identifier: bok_bc,
A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
Expand Down Expand Up @@ -39,8 +39,11 @@
--extension option to set the types of files to search
for. (default: current working directory)
-e EXTENSION, --extension EXTENSION
File extension; compression indicators (e.g. .gz) not
required. (default: .fits)
File extension to use. Must include the period (e.g.,
".fits") and it must be one of the allowed extensions
for this spectrograph. If None, root directory will be
searched for all files with any of the allowed
extensions. (default: None)
-d OUTPUT_PATH, --output_path OUTPUT_PATH
Path to top-level output directory. (default: current
working directory)
Expand Down
20 changes: 10 additions & 10 deletions doc/help/pypeit_trace_edges.rst
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,16 @@
default mosaic. (default: None)
-s SPECTROGRAPH, --spectrograph SPECTROGRAPH
A valid spectrograph identifier, which is only used if
providing files directly: bok_bc, gemini_flamingos1,
gemini_flamingos2, gemini_gmos_north_e2v,
gemini_gmos_north_ham, gemini_gmos_north_ham_ns,
gemini_gmos_south_ham, gemini_gnirs_echelle,
gemini_gnirs_ifu, gtc_maat, gtc_osiris, gtc_osiris_plus,
jwst_nircam, jwst_nirspec, keck_deimos, keck_esi,
keck_hires, keck_kcrm, keck_kcwi, keck_lris_blue,
keck_lris_blue_orig, keck_lris_red, keck_lris_red_mark4,
keck_lris_red_orig, keck_mosfire, keck_nires,
keck_nirspec_high, keck_nirspec_high_old,
providing files directly: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
keck_nires, keck_nirspec_high, keck_nirspec_high_old,
keck_nirspec_low, lbt_luci1, lbt_luci2, lbt_mods1b,
lbt_mods1r, lbt_mods2b, lbt_mods2r, ldt_deveny,
magellan_fire, magellan_fire_long, magellan_mage,
Expand Down
2 changes: 1 addition & 1 deletion doc/help/pypeit_view_fits.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
View FITS files with ginga

positional arguments:
spectrograph A valid spectrograph identifier: bok_bc,
spectrograph A valid spectrograph identifier: aat_uhrf, bok_bc,
gemini_flamingos1, gemini_flamingos2,
gemini_gmos_north_e2v, gemini_gmos_north_ham,
gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
Expand Down
10 changes: 5 additions & 5 deletions doc/help/run_pypeit.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
## PypeIt : The Python Spectroscopic Data Reduction Pipeline v1.16.1.dev109+g885cb1823
##
## Available spectrographs include:
## bok_bc, gemini_flamingos1, gemini_flamingos2, gemini_gmos_north_e2v,
## gemini_gmos_north_ham, gemini_gmos_north_ham_ns,
## gemini_gmos_south_ham, gemini_gnirs_echelle, gemini_gnirs_ifu,
## gtc_maat, gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
## keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
## aat_uhrf, bok_bc, gemini_flamingos1, gemini_flamingos2,
## gemini_gmos_north_e2v, gemini_gmos_north_ham,
## gemini_gmos_north_ham_ns, gemini_gmos_south_ham, gemini_gnirs_echelle,
## gemini_gnirs_ifu, gtc_maat, gtc_osiris, gtc_osiris_plus, jwst_nircam,
## jwst_nirspec, keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
## keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
## keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire, keck_nires,
## keck_nirspec_high, keck_nirspec_high_old, keck_nirspec_low, lbt_luci1,
Expand Down
1 change: 1 addition & 0 deletions doc/include/class_datamodel_pypeitimage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Attribute Type
``det_img`` `numpy.ndarray`_ `numpy.integer`_ If a detector mosaic, this image provides the detector that contributed to each pixel.
``detector`` :class:`~pypeit.images.detector_container.DetectorContainer`, :class:`~pypeit.images.mosaic.Mosaic` The detector (see :class:`~pypeit.images.detector_container.DetectorContainer`) or mosaic (see :class:`~pypeit.images.mosaic.Mosaic`) parameters
``exptime`` int, float Effective exposure time (s)
``filename`` str Filename for the image
``fullmask`` :class:`~pypeit.images.imagebitmask.ImageBitMaskArray` Image mask
``image`` `numpy.ndarray`_ `numpy.floating`_ Primary image data
``img_scale`` `numpy.ndarray`_ `numpy.floating`_ Image count scaling applied (e.g., 1/flat-field)
Expand Down
1 change: 1 addition & 0 deletions doc/include/inst_detector_table.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
============================ === ======== ======== ======== ======== ========================== ====================== ======== ======== ============ ========= ==========
Instrument Det specaxis specflip spatflip namp gain RN darkcurr min sat nonlinear platescale
============================ === ======== ======== ======== ======== ========================== ====================== ======== ======== ============ ========= ==========
``aat_uhrf`` 1 0 False False 1 1.0 0.0 0.0 -1.0e+10 65535.0 0.7600 0.0500
``bok_bc`` 1 1 False False 1 1.5 3.0 5.4 -1.0e+10 65535.0 1.0000 0.2000
``gemini_flamingos1`` 1 0 False False 1 3.8 6.0 1080.0 -1.0e+10 320000.0 0.8750 0.1500
``gemini_flamingos2`` 1 0 True False 1 4.44 5.0 1800.0 -1.0e+10 700000.0 1.0000 0.1787
Expand Down
Loading
Loading