Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Sort reader table by name + diverse fixes #2745

Merged
merged 8 commits into from
Feb 20, 2024
Merged

Sort reader table by name + diverse fixes #2745

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
c819c97
fix table order and update to datatables v2.
BENR0 Feb 16, 2024
46556d3
Add status description again and insert link to it in dev guide.
BENR0 Feb 16, 2024
b280236
fix missing info in multiple readers.
BENR0 Feb 16, 2024
a087971
refactor viirs platform name.
BENR0 Feb 16, 2024
40633bd
Update short_name in satpy/etc/readers/viirs_edr.yaml
djhoese Feb 16, 2024
f16e85a
refactor: set simonpr84's readers to nominal status.
BENR0 Feb 19, 2024
8101eac
refactor: change fsspec status for readers using hdf5_utils.
BENR0 Feb 19, 2024
029c0aa
Merge branch 'fix_reader_table_sort_order' of github.com:BENR0/satpy …
BENR0 Feb 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion doc/source/_static/main.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
$(document).ready( function () {
$('table.datatable').DataTable( {
"paging": false,
"dom": 'lfitp'
"layout": {
'topStart': 'info',
'topEnd': 'search',
'bottomStart': null
},
"order": [[0, 'asc']]
} );
} );
4 changes: 2 additions & 2 deletions doc/source/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,11 +194,11 @@ def __getattr__(cls, name):

html_css_files = [
"theme_overrides.css", # override wide tables in RTD theme
"https://cdn.datatables.net/1.10.23/css/jquery.dataTables.min.css",
"https://cdn.datatables.net/v/dt/dt-2.0.0/datatables.min.css",
]

html_js_files = [
"https://cdn.datatables.net/1.10.23/js/jquery.dataTables.min.js",
"https://cdn.datatables.net/v/dt/dt-2.0.0/datatables.min.js",
"main.js",
]

Expand Down
3 changes: 2 additions & 1 deletion doc/source/dev_guide/custom_reader.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,8 @@ The parameters to provide in this section are:
file format. This can be multiline if formatted properly in YAML (see
example below).
status
The status of the reader (one of: Nominal, Beta, Alpha)
The status of the reader (one of: Nominal, Beta, Alpha, Defunct; see :ref:`Status Description <Status Description>`
djhoese marked this conversation as resolved.
Show resolved Hide resolved
for more details).
supports_fsspec
If the reader supports reading data via fsspec (either true or false).
sensors
Expand Down
24 changes: 24 additions & 0 deletions doc/source/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,30 @@ Documentation

.. include:: reader_table.rst

.. _Status Description:
.. note::

Status description:

Defunct
Most likely the reader is not functional. If it is there is a good chance of
bugs and/or performance problems (e.g. not ported to dask/xarray yet). Future
development is unclear. Users are encouraged to contribute (see section
:doc:`dev_guide/CONTRIBUTING` and/or get help on Slack or by opening a Github issue).

Alpha
This denotes early development status. Reader is functional and implements some
or all of the nominal features. There might be bugs. Exactness of results is
not be guaranteed. Use at your own risk.

Beta
This denotes final developement status. Reader is functional and implements all
nominal features. Results should be dependable but there might be bugs. Users
are actively encouraged to test and report bugs.

Nominal
This denotes a finished status. Reader is functional and most likely no new
features will be introduced. It has been tested and there are no known bugs.

Indices and tables
==================
Expand Down
2 changes: 1 addition & 1 deletion satpy/etc/readers/agri_fy4a_l1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
reader:
name: agri_fy4a_l1
short_name: AGRI FY4A L1
long_name: FY-4A AGRI L1 data in HDF5 format
long_name: FY-4A AGRI Level 1 HDF5 format
description: FY-4A AGRI instrument HDF5 reader
status: Beta
supports_fsspec: false
Expand Down
4 changes: 4 additions & 0 deletions satpy/etc/readers/agri_fy4b_l1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@

reader:
name: agri_fy4b_l1
short_name: AGRI FY4B L1
long_name: FY-4B AGRI Level 1 data HDF5 format
description: FY-4B AGRI instrument HDF5 reader
status: Beta
supports_fsspec: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simonrp84 @BENR0 I could be wrong, but I think any reader that is based on the HDF5 utility file handler now supports fsspec because it uses open_file_or_filename:

satpy/satpy/readers/hdf5_utils.py

Lines 46 to 48 in 754e706

try:
f_obj = open_file_or_filename(self.filename)
file_handle = h5py.File(f_obj, "r")

No idea if it performs very well or if h5py works well with it, but I think the support is there.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I think this is true. I have set this to true for those readers using "hdf5_utils" but have not tested this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this something that should be tested via unittests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recently added tests for the helper function:

satpy/satpy/tests/test_readers.py

Lines 1207 to 1231 in e74729e

@pytest.mark.parametrize(
("file_thing", "create_read_func"),
[
(lazy_fixture("local_netcdf_filename"), _open_xarray_default),
(lazy_fixture("local_netcdf_filename"), _open_xarray_netcdf4),
(lazy_fixture("local_netcdf_filename"), _open_xarray_h5netcdf),
(lazy_fixture("local_netcdf_path"), _open_xarray_default),
(lazy_fixture("local_netcdf_path"), _open_xarray_netcdf4),
(lazy_fixture("local_netcdf_path"), _open_xarray_h5netcdf),
(lazy_fixture("local_netcdf_fsspec"), _open_xarray_default),
(lazy_fixture("local_netcdf_fsspec"), _open_xarray_h5netcdf),
(lazy_fixture("local_netcdf_fsfile"), _open_xarray_default),
(lazy_fixture("local_netcdf_fsfile"), _open_xarray_h5netcdf),
(lazy_fixture("local_hdf5_filename"), _open_h5py),
(lazy_fixture("local_hdf5_path"), _open_h5py),
(lazy_fixture("local_hdf5_fsspec"), _open_h5py),
],
)
def test_open_file_or_filename(file_thing, create_read_func):
"""Test various combinations of file-like things and opening them with various libraries."""
from satpy.readers import open_file_or_filename
read_func = create_read_func()
open_thing = open_file_or_filename(file_thing)
read_func(open_thing)

sensors: [agri]
default_channels:
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
4 changes: 4 additions & 0 deletions satpy/etc/readers/ghi_l1.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,11 @@

reader:
name: ghi_l1
short_name: GHI FY4A L1
long_name: FY-4A GHI Level 1 HDF5 format
description: FY-4A GHI instrument HDF5 reader
status: Beta
supports_fsspec: false
sensors: [ghi]
default_channels:
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
6 changes: 5 additions & 1 deletion satpy/etc/readers/meris_nc_sen3.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
reader:
description: NC Reader for MERIS data (Sentinel 3 like format)
name: meris_nc_sen3
short_name: MERIS Sentinel 3
long_name: Sentinel 3 MERIS NetCDF format
description: NC Reader for MERIS data (Sentinel 3 like format)
status: Beta
supports_fsspec: false
sensors: [meris]
default_channels: []
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
6 changes: 5 additions & 1 deletion satpy/etc/readers/mersi_ll_l1b.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
reader:
description: FY-3E Medium Resolution Spectral Imager - Low Light (MERSI-LL) L1B Reader
name: mersi_ll_l1b
short_name: MERSI Low Light FY3E L1B
long_name: FY-3E MERSI Low Light Level 1B
description: FY-3E Medium Resolution Spectral Imager - Low Light (MERSI-LL) L1B Reader
status: Beta
supports_fsspec: false
sensors: [mersi-ll]
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader

Expand Down
6 changes: 5 additions & 1 deletion satpy/etc/readers/sgli_l1b.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
reader:
name: sgli_l1b
short_name: SGLI GCOM-C L1B
long_name: GCOM-C SGLI Level 1B HDF5 format
description: Reader for SGLI data
status: Beta
supports_fsspec: false
reference: https://gportal.jaxa.jp/gpr/assets/mng_upload/GCOM-C/SGLI_Level1_Product_Format_Description_en.pdf
name: sgli_l1b
sensors: [sgli]
default_channels: []
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
Expand Down
2 changes: 1 addition & 1 deletion satpy/etc/readers/viirs_compact.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
reader:
name: viirs_compact
short_name: VIIRS Compact
long_name: SNPP VIIRS SDR data in HDF5 Compact format
long_name: JPSS VIIRS SDR data in HDF5 Compact format
description: Generic Eumetsat Compact VIIRS Reader
status: Nominal
supports_fsspec: false
Expand Down
6 changes: 5 additions & 1 deletion satpy/etc/readers/viirs_edr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
reader:
description: VIIRS NOAA Enterprise EDR product reader
name: viirs_edr
short_name: VIIRS EDR
long_name: JPSS VIIRS EDR NetCDF format
description: VIIRS NOAA Enterprise EDR product reader
status: Beta
supports_fsspec: false
reader: !!python/name:satpy.readers.yaml_reader.FileYAMLReader
sensors: [viirs]
group_keys: ['platform_shortname']
Expand Down
2 changes: 1 addition & 1 deletion satpy/etc/readers/viirs_l1b.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
reader:
name: viirs_l1b
short_name: VIIRS l1b
long_name: SNPP VIIRS Level 1b data in netCDF4 format
long_name: JPSS VIIRS Level 1b data in netCDF4 format
description: Generic NASA VIIRS L1B Reader
status: Nominal
supports_fsspec: false
Expand Down
2 changes: 1 addition & 1 deletion satpy/etc/readers/viirs_sdr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
reader:
name: viirs_sdr
short_name: VIIRS SDR
long_name: SNPP VIIRS data in HDF5 SDR format
long_name: JPSS VIIRS data in HDF5 SDR format
description: VIIRS SDR Reader
status: Nominal
supports_fsspec: false
Expand Down
Loading