🚑 Package atlas_catalog.yaml into deepicedrain

So that calling `deepicedrain.catalog` will actually work when people `pip install deepicedrain` without cloning the git repository (otherwise a FileNotFoundError is raised). Added a plugin to pyproject.toml so that the ATLAS catalog can be loaded via intake through `intake.cat.atlas_cat` too! Done by moving atlas_catalog.yaml and tests/ into the deepicedrain folder. This relies on a bit of magic (good ones), using the Python 3.7+ importlib.resources module to locate the atlas_catalog.yaml file via a relative path where the package is installed (in site-packages). Basically following a modified version of https://github.com/intake/intake-examples/tree/04bbe1880f2a4d2c74c6ea9c54385c380c1b9a1e/data_package. Had to use {{ CATALOG_DIR }} to link to the test_catalog.yaml file too. Side effect of this is that we're bundling all our tests into the `deepicedrain` python package, which might be bad for file size but good for finding test examples I guess.
weiji14 · May 28, 2020 · 02904bb · 02904bb
1 parent 7ea9c8d
commit 02904bb
Show file tree

Hide file tree

Showing 9 changed files with 15 additions and 4 deletions.
diff --git a/.github/workflows/python-app.yml b/.github/workflows/python-app.yml
@@ -66,4 +66,4 @@ jobs:
 
       - name: Test with pytest
         shell: bash -l {0}
-        run: poetry run pytest --verbose tests/
+        run: poetry run pytest --verbose deepicedrain/
diff --git a/deepicedrain/__init__.py b/deepicedrain/__init__.py
@@ -1,10 +1,18 @@
+import importlib.resources
+import logging
+
 import intake
+
+import deepicedrain
 from deepicedrain.deltamath import calculate_delta
 from deepicedrain.spatiotemporal import Region
 
 __version__: str = "0.1.0"
 
 # Loads the ICESat-2 ATLAS intake data catalog
-catalog: intake.catalog.local.YAMLFileCatalog = intake.open_catalog(
-    uri="atlas_catalog.yaml"
+_catalog_path = importlib.resources.path(
+    package=deepicedrain, resource="atlas_catalog.yaml"
 )
+with _catalog_path as uri:
+    logging.info(f"Loading intake catalog from {uri}")
+    catalog: intake.catalog.local.YAMLFileCatalog = intake.open_catalog(uri=str(uri))
diff --git a/atlas_catalog.yaml → deepicedrain/atlas_catalog.yaml b/atlas_catalog.yaml → deepicedrain/atlas_catalog.yaml
@@ -105,7 +105,7 @@ sources:
           coastline: True
   test_data:
     args:
-      path: tests/test_catalog.yaml
+      path: '{{ CATALOG_DIR }}/tests/test_catalog.yaml'
     description: 'Sample ICESat-2 datasets for testing purposes'
     driver: intake.catalog.local.YAMLFileCatalog
     metadata: {}
diff --git a/tests/__init__.py → deepicedrain/tests/__init__.py b/tests/__init__.py → deepicedrain/tests/__init__.py
diff --git a/tests/test_calculate_delta.py → deepicedrain/tests/test_calculate_delta.py b/tests/test_calculate_delta.py → deepicedrain/tests/test_calculate_delta.py
diff --git a/tests/test_catalog.yaml → deepicedrain/tests/test_catalog.yaml b/tests/test_catalog.yaml → deepicedrain/tests/test_catalog.yaml
diff --git a/tests/test_deepicedrain.py → deepicedrain/tests/test_deepicedrain.py b/tests/test_deepicedrain.py → deepicedrain/tests/test_deepicedrain.py
diff --git a/tests/test_region.py → deepicedrain/tests/test_region.py b/tests/test_region.py → deepicedrain/tests/test_region.py
diff --git a/pyproject.toml b/pyproject.toml
@@ -35,6 +35,9 @@ black = "^19.10b0"
 jupytext = "^1.4.2"
 pytest = "^5.4.2"
 
+[tool.poetry.plugins."intake.catalogs"]
+"atlas_cat" = "deepicedrain:catalog"
+
 [build-system]
 requires = ["poetry>=0.12"]
 build-backend = "poetry.masonry.api"