Release/v1.2.0: SAR Support, Multi-Resolution Tiles, and Feature Utilities

.github/workflows/documentation.yaml → .github/workflows/documentation-publish.yml

@@ -1,12 +1,17 @@
-name: Docs
-on: [push, pull_request, workflow_dispatch]
+name: Generate and Publish Documentation for OSML Imagery Toolkit
+
+on:
+  workflow_call:
+
 permissions:
     contents: write
+
 jobs:
   docs:
+    if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v3
+      - uses: actions/checkout@v4
       - uses: actions/setup-python@v3
       - name: Install dependencies
         run: |
@@ -19,7 +24,6 @@ jobs:
           tox -e docs
       - name: Deploy
         uses: peaceiris/[email protected]
-        if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
         with:
           publish_branch: gh-pages
           github_token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/osml-imagery-toolkit-build.yml

@@ -0,0 +1,10 @@
+name: "OSML Imagery Toolkit Build Workflow"
+
+on:
+  pull_request:
+    branches: ["main", "dev"]
+
+jobs:
+  Build_Validate_Tox:
+    uses: ./.github/workflows/python-tox.yml
+    secrets: inherit

.github/workflows/osml-imagery-toolkit-publish.yml

@@ -0,0 +1,18 @@
+name: "OSML Imagery Toolkit Build and Publish Workflow"
+
+on:
+  push:
+    branches: ["main"]
+
+jobs:
+  Build_Validate_Tox:
+    uses: ./.github/workflows/python-tox.yml
+    secrets: inherit
+  Publish_Python:
+    needs: [Build_Validate_Tox]
+    uses: ./.github/workflows/python-publish.yml
+    secrets: inherit
+  Publish_Documentation:
+    needs: [Publish_Python]
+    uses: ./.github/workflows/documentation-publish.yml
+    secrets: inherit

.github/workflows/python-publish.yml

@@ -6,9 +6,10 @@
 # separate terms of service, privacy policy, and support
 # documentation.
 
-name: Upload Python Package
+name: Publish Python Package
 
 on:
+  workflow_call:
   release:
     types: [published]
 
@@ -17,11 +18,10 @@ permissions:
 
 jobs:
   deploy:
-
+    if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
     runs-on: ubuntu-latest
-
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python
       uses: actions/setup-python@v3
       with:
@@ -33,7 +33,7 @@ jobs:
     - name: Build package
       run: python -m build
     - name: Publish package
-      uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
+      uses: pypa/gh-action-pypi-publish@v1.8.10
       with:
         user: __token__
         password: ${{ secrets.PYPI_API_TOKEN }}

.github/workflows/python-package.yml → .github/workflows/python-tox.yml

@@ -1,27 +1,21 @@
 # This workflow will install Python dependencies, run tests and lint with a single version of Python
 # For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
 
-name: Tox Build/Validation
+name: Build/Validation with Tox
 
 on:
-  push:
-    branches: [ "main" ]
-  pull_request:
-    branches: [ "main" ]
+  workflow_call:
 
 permissions:
   contents: read
 
 jobs:
   build:
-
     runs-on: ubuntu-latest
-
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
       with:
-          lfs: 'true'
-          ssh-key: ${{ secrets.git_ssh_key }}
+        lfs: 'true'
     - name: Set up Python 3.10
       uses: actions/setup-python@v3
       with:

.pre-commit-config.yaml

@@ -24,7 +24,7 @@ repos:
     rev: 6.0.0
     hooks:
       - id: flake8
-        args: ["--ignore=E203,W503,W605", "--max-line-length=125"]
+        args: ["--ignore=E203,W503,W605", "--max-line-length=160", "--extend-exclude=src/aws/osml/formats"]
         types: [file, python]
 
   - repo: https://github.com/pre-commit/mirrors-autopep8

README.md

@@ -2,14 +2,24 @@
 
 The OversightML Imagery Toolkit is a Python package that contains image processing and photogrammetry routines commonly
 used during the analysis of imagery collected by satellites and unmanned aerial vehicles (UAVs). It builds upon GDAL
-by providing additional support for images compliant with the National Imagery Transmission Format (NITF) and Sensor
-Independent Complex Data (SICD) standards.
+by providing additional support for images compliant with the National Imagery Transmission Format (NITF), Sensor
+Independent Complex Data (SICD), and Sensor Independent Derived Data (SIDD) standards.
 
+This library contains four core packages under the `aws.osml` namespace:
+* **photogrammetry**: convert locations between the image (x, y) and geodetic (lon, lat, elev) coordinate systems
+* **gdal**: utilities to work with datasets loaded by GDAL
+* **image_processing**: common image manipulation routines
+* **features**: common geospatial feature manipulation routines
+
+## Documentation
+
+* **APIs**: You can find API documentation for the OSML Imagery Toolkit hosted on our [GitHub project page](https://aws-solutions-library-samples.github.io/osml-imagery-toolkit/).
+If you are working from the source code running `tox -e docs` will trigger the Sphinx documentation build.
+* **Example Notebooks**: Example notebooks for some operations are in the `examples` directory
 ## Installation
 
-The intent is to vend / distribute this software through a Python Package Index.
-If your environment has a distribution,
-you should be able to install it using pip:
+This software is available through a Python Package Index.
+If your environment has a distribution, you should be able to install it using pip:
 ```shell
 pip install osml-imagery-toolkit[gdal]
 ```
@@ -23,97 +33,6 @@ Note that GDAL is listed as an extra dependency for this package. This is done t
 don't want to use GDAL or those that have their own custom installation steps for that library. Future versions of
 this package will include image IO backbones that have fewer dependencies.
 
-
-## Documentation
-
-You can find documentation for this library in the `./doc` directory. Sphinx is used to construct a searchable HTML
-version of the API documents.
-
-```shell
-tox -e docs
-```
-
-## Example Usage
-
-This library contains four core packages under the `aws.osml` namespace.
-* photogrammetry: convert locations between the image (x, y) and geodetic (lon, lat, elev) coordinate systems
-* gdal: help load and manage datasets loaded by GDAL
-* image_processing: common image manipulation routines
-* formats: utilities for handling format specific information; normally not accessed directly
-
-```python
-from aws.osml.gdal import GDALImageFormats, GDALCompressionOptions, load_gdal_dataset
-from aws.osml.image_processing import GDALTileFactory
-from aws.osml.photogrammetry import ImageCoordinate, GeodeticWorldCoordinate, SensorModel
-```
-
-### Tiling with Updated Image Metadata
-
-Many applications break large remote sensing images into smaller chips or tiles for distributed processing or
-dissemination. GDAL's Translate function provides basic capabilities, but it does not correctly update geospatial
-metadata to reflect the new image extent. These utilities provide those functions so tile consumers can correctly
-interpret the pixel information they have been provided. For NITF imagery that includes the addition of a new ICHIPB
-TRE. With SICD the XML ImageData elements are adjusted to identify the sub-image bounds.
-
-```python
-# Load the image and create a sensor model
-ds, sensor_model = load_gdal_dataset("./imagery/sample.nitf")
-tile_factory = GDALTileFactory(ds,
-                               sensor_model,
-                               GDALImageFormats.NITF,
-                               GDALCompressionOptions.NONE
-                               )
-
-# Bounds are [left_x, top_y, width, height]
-nitf_encoded_tile_bytes = tile_factory.create_encoded_tile([0, 0, 1024, 1024])
-```
-
-### Tiling for Display
-
-Some images, for example 11-bit panchromatic images or SAR imagery with floating point complex data, can not be
-displayed directly without remapping the pixels into an 8-bit per pixel grayscale or RGB color model. The TileFactory
-supports creation of tiles suitable for human review by setting both the output_type and range_adjustment options.
-
-```python
-viz_tile_factory = GDALTileFactory(ds,
-                                   sensor_model,
-                                   GDALImageFormats.PNG,
-                                   GDALCompressionOptions.NONE,
-                                   output_type=gdalconst.GDT_Byte,
-                                   range_adjustment=RangeAdjustmentType.DRA)
-
-viz_tile = viz_tile_factory.create_encoded_tile([0, 0, 1024, 1024])
-```
-
-### More Precise Sensor Models
-
-OversightML provides implementations of the Replacement Sensor Model (RSM), Rational Polynomial
-Camera (RPC), and Sensor Independent Complex Data (SICD) sensor models to assist in geo positioning.
-When loading a dataset, the toolkit will construct the most accurate sensor model
-from the available image metadata. That sensor model can be used in conjunction with an optional
-elevation model to convert between image and geodetic coordinates.
-
-```python
-ds, sensor_model = load_gdal_dataset("./imagery/sample.nitf")
-elevation_model = DigitalElevationModel(
-    SRTMTileSet(version="1arc_v3"),
-    GDALDigitalElevationModelTileFactory("./local-SRTM-tiles"))
-
-# Note the order of ImageCoordinate is (x, y)
-geodetic_location_of_ul_corner = sensor_model.image_to_world(
-    ImageCoordinate([0, 0]),
-    elevation_model=elevation_model)
-
-lon_degrees = -77.404453
-lat_degrees = 38.954831
-meters_above_ellipsoid = 100.0
-
-image_location = sensor_model.world_to_image(
-    GeodeticWorldCoordinate([radians(lon_degrees),
-                             radians(lat_degrees),
-                             meters_above_ellipsoid]))
-```
-
 ## Contributing
 
 This project welcomes contributions and suggestions. If you would like to submit a pull request, see our

doc/conf.py

@@ -46,7 +46,7 @@ def setup(app):
 # -- Project information -----------------------------------------------------
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
-project = "OversightML Imagery Core"
+project = "OversightML Imagery Toolkit"
 copyright = "{}, Amazon.com".format(datetime.datetime.now().year)
 author = "Amazon Web Services"
 

doc/index.rst

@@ -1,18 +1,19 @@
 
-aws-osml-imagery-core
+osml-imagery-toolkit
 =====================
 
-The OversightML Imagery Core is a Python library that contains image processing and photogrammetry routines commonly
+The OversightML Imagery Toolkit is a Python package that contains image processing and photogrammetry routines commonly
 used during the analysis of imagery collected by satellites and unmanned aerial vehicles (UAVs). It builds upon GDAL
-by providing additional support for images compliant with the Sensor Independent Complex Data (SICD) and National
-Imagery Transmission Format (NITF) standards.
+by providing additional support for images compliant with the National Imagery Transmission Format (NITF), Sensor
+Independent Complex Data (SICD), and Sensor Independent Derived Data (SIDD) standards.
 
 .. toctree::
    :maxdepth: 1
 
    _apidoc/aws.osml.photogrammetry
    _apidoc/aws.osml.gdal
    _apidoc/aws.osml.image_processing
+   _apidoc/aws.osml.features
 
 
 Indices and tables