Skip to content

Commit

Permalink
Adjust imports of sotoddb classes into new structure
Browse files Browse the repository at this point in the history
This imports ResultSet and the metadata *Db directly to core.metadata,
renaming the *DB classes as *Db.  The more stuff-on-disk code in
sotodlib.simple is imported as io.metadata.simple.  This has been
tested on simons1 with the SO sim and ACT Uranus demo code.
  • Loading branch information
mhasself committed Apr 27, 2020
1 parent 0ef6ec0 commit 0086aa1
Show file tree
Hide file tree
Showing 4 changed files with 17 additions and 16 deletions.
8 changes: 4 additions & 4 deletions sotodlib/core/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,11 +77,11 @@ def reload(self, load_list='all'):
"""
# Metadata support databases.
for key, cls in [('detdb', metadata.DetDB),
('obsdb', metadata.ObsDB),
('obsfiledb', metadata.ObsFileDB)]:
for key, cls in [('detdb', metadata.DetDb),
('obsdb', metadata.ObsDb),
('obsfiledb', metadata.ObsFileDb)]:
if (load_list == 'all' or key in load_list) and key in self:
# E.g. self.detdb = DetDB.from_file(self['detdb']
# E.g. self.detdb = DetDb.from_file(self['detdb']
db = cls.from_file(self[key])
setattr(self, key, db)
# The metadata loader.
Expand Down
6 changes: 5 additions & 1 deletion sotodlib/core/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,8 @@
"""Metadata containers.
"""

from sotoddb import resultset
# Import classes from sotoddb, with a rename DB -> Db. These will
# live in sotodlib soon.
from sotoddb import ResultSet, DetDB as DetDb, ObsDB as ObsDb, ObsFileDB as ObsFileDb
from sotoddb import ManifestDB as ManifestDb, ManifestScheme
from sotoddb import SuperLoader
12 changes: 6 additions & 6 deletions sotodlib/io/load.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
The basic routines here provide structures for automatically unpacking
series of G3 frames into coherent data structures.
Additional routines use an ObsFileDB (provided by the user) to
Additional routines use an ObsFileDb (provided by the user) to
optimize disk reads for loading particular data of interest to a user.
These routines rely on the completeness and accuracy of the ObsFileDB,
These routines rely on the completeness and accuracy of the ObsFileDb,
and are appropriate for large archives of files that have been
pre-scanned to log their contents.
Expand Down Expand Up @@ -355,14 +355,14 @@ def load_observation(db, obs_id, dets=None, prefix=None):
Arguments:
db (ObsFileDB): The database describing this observation file
db (ObsFileDb): The database describing this observation file
set.
obs_id (str): The identifier of the observation.
dets (list of str): The detector names of interest. If None,
loads all dets present in this observation. To load
only the ancillary data, pass an empty list.
prefix (str): The root address of the data files. If not
specified, the prefix is taken from the ObsFileDB.
specified, the prefix is taken from the ObsFileDb.
Returns an AxisManager with the data.
Expand Down Expand Up @@ -489,9 +489,9 @@ def hstack_into(dest, src_arrays):
#:
#: loader(db, obs_id, dets=None, prefix=None)
#:
#: Here db is an ObsFileDB, obs_id is a string, dets is a list of
#: Here db is an ObsFileDb, obs_id is a string, dets is a list of
#: string names of readout channels, and prefix is a string that overrides
#: the path prefix of ObsFileDB.
#: the path prefix of ObsFileDb.
#:
#: "This is an interim solution and the API will change", he said in
#: March 2020.
Expand Down
7 changes: 2 additions & 5 deletions sotodlib/io/metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,5 @@
"""Metadata I/O.
"""

from sotoddb import detdb
from sotoddb import loader
from sotoddb import obsdb
from sotoddb import obsfiledb
from sotoddb import proddb
from sotoddb import simple, loader
from sotoddb import SuperLoader

2 comments on commit 0086aa1

@keskitalo
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the latest sotoddb master installed. When I run python3 setup.py test I get:

ERROR: tests.test_toast_load (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.test_toast_load
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/reijo/Software/git/sotodlib/tests/test_toast_load.py", line 18, in <module>
    from sotodlib.sim_hardware import get_example
  File "/Users/reijo/Software/git/sotodlib/sotodlib/sim_hardware.py", line 16, in <module>
    from .core import Hardware
  File "/Users/reijo/Software/git/sotodlib/sotodlib/core/__init__.py", line 8, in <module>
    from .context import Context
  File "/Users/reijo/Software/git/sotodlib/sotodlib/core/context.py", line 5, in <module>
    from . import metadata
  File "/Users/reijo/Software/git/sotodlib/sotodlib/core/metadata.py", line 8, in <module>
    from sotoddb import ResultSet, DetDB as DetDb, ObsDB as ObsDb, ObsFileDB as ObsFileDb
  File "/Users/reijo/Software/macports/lib/python3.8/site-packages/sotoddb/__init__.py", line 2, in <module>
    from .loader import SuperLoader
  File "/Users/reijo/Software/macports/lib/python3.8/site-packages/sotoddb/loader.py", line 1, in <module>
    from sotodlib import core, metadata
ImportError: cannot import name 'metadata' from 'sotodlib' (/Users/reijo/Software/git/sotodlib/sotodlib/__init__.py)

Do you see this issue?

@mhasself
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my fault... see simonsobs/sotoddb#10

Review and merge that if it fixes your problem!

Please sign in to comment.