Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map algebra bindings #576

Merged
merged 1 commit into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
- spark config 'spark.databricks.labs.mosaic.raster.use.checkpoint' in addition to 'spark.databricks.labs.mosaic.raster.checkpoint'.
- python: `mos.enable_gdal(spark, with_checkpoint_path=path)`.
- scala: `MosaicGDAL.enableGDALWithCheckpoint(spark, path)`.
- Python bindings added for `rst_avg`, `rst_max`, `rst_median`, `rst_min`, and `rst_pixelcount`.

## v0.4.2 [DBR 13.3 LTS]
- Geopandas now fixed to "<0.14.4,>=0.14" due to conflict with minimum numpy version in geopandas 0.14.4.
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ The python bindings can be tested using [unittest](https://docs.python.org/3/lib
- Move to the `python/` directory and install the project and its dependencies:
`pip install . && pip install pyspark==<project_spark_version>`
(where 'project_spark_version' corresponds to the version of Spark
used for the target Databricks Runtime, e.g. `3.2.1`.
used for the target Databricks Runtime, e.g. `3.4.1` for DBR 13.3 LTS.
- Run the tests using `unittest`: `python -m unittest`

The project wheel file can be built with [build](https://pypa-build.readthedocs.io/en/stable/).
Expand Down
23 changes: 5 additions & 18 deletions docs/source/api/raster-functions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,7 @@ e.g. :code:`spark.read.format("gdal")`
Updates to the raster features for 0.4.1
----------------------------------------

* In 0.4.1, there are a new set of raster apis that have not yet had python bindings generated; however you can still
call the functions with pyspark function :code:`selectExpr`, e.g. :code:`selectExpr("rst_avg(...)")` which invokes the sql
registered expression. The calls are: :ref:`rst_avg`, :ref:`rst_max`, :ref:`rst_min`, :ref:`rst_median`, and :ref:`rst_pixelcount`.
* Also, scala does not have a :code:`df.display()` method while python does. In practice you would most often call
* Scala does not have a :code:`df.display()` method while python does. In practice you would most often call
:code:`display(df)` in scala for a prettier output, but for brevity, we write :code:`df.show` in scala.

.. note:: For mosaic versions > 0.4.0 you can use the revamped setup_gdal function or new setup_fuse_install.
Expand All @@ -52,8 +49,6 @@ rst_avg
.. function:: rst_avg(tile)

Returns an array containing mean values for each band.
The python bindings are available through sql,
e.g. :code:`selectExpr("rst_avg(tile)")`

:param tile: A column containing the raster tile.
:type tile: Column (RasterTileType)
Expand All @@ -64,7 +59,7 @@ rst_avg
.. tabs::
.. code-tab:: python

df.selectExpr("rst_avg(tile)"").limit(1).display()
df.selectExpr(mos.rst_avg("tile")).limit(1).display()
+---------------+
| rst_avg(tile) |
+---------------+
Expand Down Expand Up @@ -1346,8 +1341,6 @@ rst_max
.. function:: rst_max(tile)

Returns an array containing maximum values for each band.
The python bindings are available through sql,
e.g. :code:`selectExpr("rst_max(tile)")`

:param tile: A column containing the raster tile.
:type tile: Column (RasterTileType)
Expand All @@ -1358,7 +1351,7 @@ rst_max
.. tabs::
.. code-tab:: python

df.selectExpr("rst_max(tile)"").limit(1).display()
df.selectExpr(mos.rst_max("tile")).limit(1).display()
+---------------+
| rst_max(tile) |
+---------------+
Expand Down Expand Up @@ -1389,8 +1382,6 @@ rst_median
.. function:: rst_median(tile)

Returns an array containing median values for each band.
The python bindings are available through sql,
e.g. :code:`selectExpr("rst_median(tile)")`

:param tile: A column containing the raster tile.
:type tile: Column (RasterTileType)
Expand All @@ -1401,7 +1392,7 @@ rst_median
.. tabs::
.. code-tab:: python

df.selectExpr("rst_median(tile)"").limit(1).display()
df.selectExpr(mos.rst_median("tile")).limit(1).display()
+---------------+
| rst_median(tile) |
+---------------+
Expand Down Expand Up @@ -1609,8 +1600,6 @@ rst_min
.. function:: rst_min(tile)

Returns an array containing minimum values for each band.
The python bindings are available through sql,
e.g. :code:`selectExpr("rst_min(tile)")`

:param tile: A column containing the raster tile.
:type tile: Column (RasterTileType)
Expand All @@ -1621,7 +1610,7 @@ rst_min
.. tabs::
.. code-tab:: python

df.selectExpr("rst_min(tile)"").limit(1).display()
df.selectExpr(mos.rst_min("tile")).limit(1).display()
+---------------+
| rst_min(tile) |
+---------------+
Expand Down Expand Up @@ -1751,8 +1740,6 @@ rst_pixelcount
.. function:: rst_pixelcount(tile)

Returns an array containing valid pixel count values for each band.
The python bindings are available through sql,
e.g. :code:`selectExpr("rst_pixelcount(tile)")`

:param tile: A column containing the raster tile.
:type tile: Column (RasterTileType)
Expand Down
Loading
Loading