Creating a Scene object with Terra MODIS data

[Krangaas]

Code Sample, a minimal, complete, and verifiable piece of code

import os
from satpy import Scene
from pyresample import utils
#from satpy.utils import debug_on
#debug_on()
BASEDIR = "C:\scrypts\K7"
FILES = os.path.join(BASEDIR, "MOD021KM.A2018175.0955.061.2018175194932.hdf")
print(FILES)
global_scene = Scene(filenames = [FILES])
print(global_scene.available_dataset_names())

Problem description

I want to look at weather satellite data over Svalbard and downloaded Terra MODIS data from LAADS, I am unable to create a Scene with the data though. It seems satpy is unable to find a reader for the data, but on satpy's readthedocs page (https://satpy.readthedocs.io/en/stable/) on the bottom table it says that satpy should have a reader for this type of data. I tried manually setting the reader with kwarg: reader ="hdfeos_l1b", but then I get: "ValueError: No supported files found".

I also had some difficulty creating a scene with NOAA-15 and NOAA-18 data, maybe I should open up a different issue for that problem?

Expected Output

I want to load up the data and get a printout of the available channels.

Actual Result, Traceback if applicable

No handlers could be found for logger "satpy.readers.yaml_reader"
Traceback (most recent call last):
  File "aq.py", line 9, in <module>
    global_scene = Scene(filenames = [FILES])
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\scene.py", 
line 143, in __init__
    reader_kwargs=reader_kwargs)
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\scene.py", 
line 188, in create_reader_instances
    ppp_config_dir=self.ppp_config_dir)
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\readers\__init__.py",
line 403, in load_readers
    reader_instance = load_reader(reader_configs, **reader_kwargs)
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\readers\__init__.py",
line 259, in load_reader
    **reader_kwargs
TypeError: 'NoneType' object is not callable

Versions of Python, package at hand and relevant dependencies

OS: Windows 7
SatPy Version: 0.8.4
PyResample Version: 1.9.3
Python version: 2.7

[djhoese]

A couple things:

1. You'll need to provide a 'reader' to the Scene creation (reader='hdfeos_l1b').
2. You'll need to provide the geolocation file and the band data file
3. We are just about to release satpy 0.9 which changes just about everything under the hood of satpy. It would be great if you could update to the newest beta. I think pip install -U --pre satpy should give you the newest beta 0.9.0b0. Note it requires the newest version of pyresample and I see you are using conda so conda install pyresample should work for that (or pip).

Hopefully we can get this all figured out for you. Good luck.

[Krangaas]

Thanks for the quick reply! I've updated to the newest beta version of satpy (0.9.0b0) and the newest version of pyresample (1.10.0), but I'm unsure of what a geolocation file and band data file is or where I can find them, on the LAADS site I was only given one file.

[djhoese]

Geolocation for MODIS L1B is usually kept in a MOD03 file.

[Krangaas]

Ok, I found the MOD03 file on LAADS with the same date and time as my file, now I need to get a hold of the band data file?

[djhoese]

Sorry, the MOD02 file is the band data, the MOD03 is the geolocation data. If you provide both of those to the scene (filenames=['MOD02...', 'MOD03...']) then it should hopefully work.

[Krangaas]

Thanks for the help, I'll have a look at it tomorrow.

[Krangaas]

I redownloaded the MOD02 and MOD03 files from LAADS and provided the files to the scene as you described, with debug_on() I get a message that satpy is unable to open the file set:

Code Sample, a minimal, complete, and verifiable piece of code

import os
from satpy import Scene
from pyresample import utils
from satpy.utils import debug_on
debug_on()

BASEDIR = "C:\scrypts\K8"
FILES1 = os.path.join(BASEDIR, "MOD021KM.A2018175.1940.061.2018176073241.hdf")
FILES2 = os.path.join(BASEDIR, "MOD03.A2018175.1940.061.2018176012619.hdf")
print("Band_Data_File = %s" %FILES1)
print("Geolocation_File = %s" %FILES2)

global_scene = Scene(reader='hdfeos_l1b', filenames = [FILES1, FILES2])
print(global_scene.available_dataset_names())

Actual Result, Traceback if applicable

(base) C:\scrypts>python SatPy_Test.py
Band_Data_File = C:\scrypts\K8\MOD021KM.A2018175.1940.061.2018176073241.hdf
Geolocation_File = C:\scrypts\K8\MOD03.A2018175.1940.061.2018176012619.hdf
[DEBUG: 2018-06-28 10:55:07 : satpy.scene] Setting 'PPP_CONFIG_DIR' to 'C:\Users\magnus\Anaconda2\lib\site-packages\satpy\etc'
[DEBUG: 2018-06-28 10:55:07 : satpy.readers] Reading ['C:\\Users\\magnus\\Anaconda2\\lib\\site-packages\\satpy\\etc\\readers\\hdfeos_l1b.yaml', 
'C:\\Users\\magnus\\Anaconda2\\lib\\site-packages\\satpy\\etc\\readers\\hdfeos_l1b.yaml', 
'C:\\Users\\magnus\\Anaconda2\\lib\\site-packages\\satpy\\etc\\readers\\hdfeos_l1b.yaml']
[INFO: 2018-06-28 10:55:07 : satpy.readers] Cannot use ['C:\\Users\\magnus\\Anaconda2\\lib\\site-packages\\satpy\\etc\\readers\\hdfeos_l1b.yaml', 
'C:\\Users\\magnus\\Anaconda2\\lib\\site-packages\\satpy\\etc\\readers\\hdfeos_l1b.yaml', 
'C:\\Users\\magnus\\Anaconda2\\lib\\site-packages\\satpy\\etc\\readers\\hdfeos_l1b.yaml']
[DEBUG: 2018-06-28 10:55:07 : satpy.readers] while constructing a Python object
cannot find module 'satpy.readers.hdfeos_l1b' (No module named pyhdf.error)
  in "<string>", line 438, column 18:
        file_reader: !!python/name:satpy.readers.hdfe ...
                     ^
[WARNING: 2018-06-28 10:55:07 : satpy.readers] Don't know how to open the following files: 
set(['C:\\scrypts\\K8\\MOD03.A2018175.1940.061.2018176012619.hdf', 'C:\\scrypts\\K8\\MOD021KM.A2018175.1940.061.2018176073241.hdf'])
Traceback (most recent call last):
  File "SatPy_Test.py", line 13, in <module>
    global_scene = Scene(reader='hdfeos_l1b', filenames = [FILES1, FILES2])
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\scene.py", line 150, in __init__
    reader_kwargs=reader_kwargs)
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\scene.py", line 195, in create_reader_instances
    ppp_config_dir=self.ppp_config_dir)
  File "C:\Users\magnus\Anaconda2\lib\site-packages\satpy\readers\__init__.py",
line 600, in load_readers
    raise ValueError("No supported files found")
ValueError: No supported files found

Versions of Python, package at hand and relevant dependencies

OS: Windows 7
SatPy Version: 0.9.0b0
PyResample Version: 1.10.0
Python version: 2.7

[mraspaud]

@Krangaas look here: [DEBUG: 2018-06-28 10:55:07 : satpy.readers] while constructing a Python object cannot find module 'satpy.readers.hdfeos_l1b' (No module named pyhdf.error)
-> the pyhdf package is missing...

[Krangaas]

Thanks again! Installed pyhdf package and managed to create the scene object, trying to load channel "1" with global_scene.load("1"), but I seem to be missing another file. I think the datafiles for MOD02HKM and MOD02QKM are also needed?

Actual Result, Traceback if applicable

[DEBUG: 2018-06-28 13:05:00 : satpy.readers.yaml_reader] No coordinates found for DatasetID(name='longitude', wavelength=None, resolution=250, polarization=None,
calibration=None, level=None, modifiers=())
[DEBUG: 2018-06-28 13:05:01 : satpy.readers.hdfeos_l1b] Interpolating from 1000 to 250
[DEBUG: 2018-06-28 13:09:58 : satpy.readers.yaml_reader] No coordinates found for DatasetID(name='latitude', wavelength=None, resolution=250, polarization=None,
calibration=None, level=None, modifiers=())
[WARNING: 2018-06-28 13:09:59 : satpy.readers.yaml_reader] Required file type 'hdf_eos_data_250m' not found or loaded for '1'
[WARNING: 2018-06-28 13:09:59 : satpy.scene] The following datasets were not created: DatasetID(name='1', wavelength=(0.62, 0.645, 0.67), resolution=250, polarization=None, calibration='reflectance', level=None, modifiers=())

[mraspaud]

you can also just stay in 1 km resolution with this:

global_scene = Scene(reader='hdfeos_l1b', filenames = [FILES1, FILES2], resolution=1000)

[djhoese]

@mraspaud I thought we fixed this so that it would realize the higher resolutions weren't available? Maybe it was just on the TODO list.

[Krangaas]

It didn't seem to allow setting the resolution when creating the scene object:

Code Sample

global_scene = Scene(reader='hdfeos_l1b', filenames = [FILES1, FILES2], resolution =1000)

Traceback

Traceback (most recent call last):
File "SatPy_Test.py", line 24, in
global_scene = Scene(reader='hdfeos_l1b', filenames = [FILES1, FILES2], resolution =1000)
TypeError: init() got an unexpected keyword argument 'resolution'

So I did this instead:

Code Sample

global_scene.load(["4","2","31"], resolution=1000)

[mraspaud]

Yes sorry my bad, it should indeed be in the .load

[mraspaud]

@djhoese that would be nice to have indeed

[Krangaas]

I managed to get some nice looking composites out of the data, thanks again for the assistance!
I also had some difficulty creating a scene object with NOAA-15 and NOAA-18 data, it seems to be a different problem altogether so I'll open up a new issue for that when I have the time.