Merge branch 'master' into dependabot/pip/aiohttp-3.9.2

.github/workflows/formatting.yml

@@ -8,10 +8,10 @@ jobs:
     runs-on: ubuntu-latest
     steps:
     - uses: actions/checkout@v2
-    - name: Set up Python 3.8
+    - name: Set up Python 3.9
       uses: actions/setup-python@v2
       with:
-        python-version: 3.8
+        python-version: 3.9
     - name: Python Blacken
       uses: psf/black@stable
       with:

.github/workflows/tests.yml

@@ -9,7 +9,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.7, 3.8]
+        python-version: ['3.9', '3.10', '3.11']
 
     steps:
     - uses: actions/checkout@v1

.gitignore

@@ -1,7 +1,7 @@
 .idea
 *.egg-info
 *venv
-conf
+.vscode
 conf.yml
 __pycache__
 *.pyc
@@ -11,3 +11,4 @@ docs/build/
 stac_generator-1.0.2.zip
 .lock
 GLOB-0.log
+cprofile

docs/source/api/stac_generator/stac_generator.rst

@@ -4,10 +4,10 @@ STAC Generator
 
 :fa:`github` `View on Github <https://github.com/cedadev/stac-generator>`_
 
-.. automodule:: stac_generator.core.processor
+.. automodule:: stac_generator.core.extraction_method
     :members:
 
 .. autoclass:: stac_generator.core.generator.BaseGenerator
 
-.. automodule:: stac_generator.core.collection_describer
+.. automodule:: stac_generator.core.baker
     :members:

docs/source/index.rst

@@ -10,12 +10,11 @@ change the source of the files, the output of the metadata and the processing ch
 which extracts the metadata. The framework leverages a modular, plugin architecture
 to allow users to modify the workflow to fit their needs.
 
-The process expects a stream of "assets" (an asset being a file, zarr object, etc.).
+The process expects a stream of "messages" for which the recipes can be run against.
 The source of this stream is configured with `input plugins <stac_generator/inputs>`_
 which could be as simple as listing directories on a file system or using message
 queues as part of a complex ingest system. The `generators <generators>`_ operate on this stream and
-pass to `output plugins <stac_generator/outputs>`_. The output is at the level
-of an "asset" so higher level aggregated objects may require an aggregation step.
+pass to `output plugins <stac_generator/outputs>`_.
 
 These outputs are also configurable so could dump to the terminal (for debugging), file,
 a data store (postgres, elasticsearch, etc.) or even a message queue for onward processing.
@@ -36,22 +35,17 @@ in a certain space and time.
 Generators
 ==========
 
-The different generators are designed to extract different levels of metadata to build the assets, items, and collections of the STAC Catalog.
+The different generators are designed to extract different levels of metadata to build the items, and collections of the STAC Catalog.
 
 .. list-table::
     :header-rows: 1
 
     * - Name
       - Description
-    * - :ref:`Asset Generator <stac_generator/generators:asset>`
-      - Generates STAC Assets via extraction methods specified in the :ref:`colelction descriptions <collection_descriptions/collection_descriptions:collection descriptions>`
-      focusing on file metadata (name, location, size, etc.)
-    * - :ref:`Item Generator <item_generator/generators:item>`
-      - Generates STAC Items via extraction methods specified in the :ref:`colelction descriptions <collection_descriptions/collection_descriptions:collection descriptions>`
-      focusing on aggregation from asset metadata.
-    * - :ref:`Collection Generator <stac_generator/generators:collection>`
-      - Generates STAC Collections via extraction methods specified in the :ref:`colelction descriptions <collection_descriptions/collection_descriptions:collection descriptions>`
-      focusing on aggregation from item metadata.
+    * - :ref:`Item Generator <item_generator/plugins/generators/item>`
+      - Generates STAC Items via extraction methods specified in the :ref:`colelction descriptions <recipe/recipes>`.
+    * - :ref:`Collection Generator <stac_generator/plugins/generators/collection>`
+      - Generates STAC Collections via extraction methods specified in the relivant :ref:`recipe <recipe/recipes>`.