Skip to content

Commit

Permalink
feat: Flask to FastAPI (#701)
Browse files Browse the repository at this point in the history
sbrunato authored May 4, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
1 parent c328060 commit 4c373fd
Showing 26 changed files with 717 additions and 424 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -97,6 +97,7 @@ jobs:
with:
path: artifacts/unit-test-results-python3.11-ubuntu-latest/coverage.xml
repo_token: ${{ secrets.GITHUB_TOKEN }}
pull_request_number: ${{ github.pull_request.number }}
minimum_coverage: 70
fail_below_threshold: false
only_changed_files: true
@@ -108,6 +109,7 @@ jobs:
with:
path: artifacts/unit-test-results-python3.11-windows-latest/coverage.xml
repo_token: ${{ secrets.GITHUB_TOKEN }}
pull_request_number: ${{ github.pull_request.number }}
minimum_coverage: 70
fail_below_threshold: false
only_changed_files: true
4 changes: 2 additions & 2 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
@@ -90,7 +90,7 @@ The documentation of EODAG consists of:
`Markdown markup language <https://jupyter-notebook.readthedocs.io/en/latest/examples/Notebook/Working%20With%20Markdown%20Cells.html>`_
and Python code

`Sphinx <https://www.sphinx-doc.org/en/master/>`_ is used to create the website from these files.
`Sphinx <https://www.sphinx-doc.org/en/master/>`_ is used to create the website from these files.

`nbsphinx <https://nbsphinx.readthedocs.io/en/0.8.3/>`_ is a Sphinx extension that can parse Jupyter Notebooks. It can also execute notebooks.
The notebooks used to document EODAG are not executed by default (see the `conf.py` file), to avoid
@@ -119,7 +119,7 @@ If not, the outputs and the widgets (e.g. progress bar) won't be displayed in th
`sphinx-autobuild <https://pypi.org/project/sphinx-autobuild/>`_ can be installed to rebuild Sphinx documentation on changes, with live-reload in the browser.
Run it from the repository root with ``sphinx-autobuild docs docs/_build/html/``

`Read the Docs <https://readthedocs.org/>`_ is a service that uses Sphinx to build a documentation website,
`Read the Docs <https://readthedocs.org/>`_ is a service that uses Sphinx to build a documentation website,
which it then hosts for free for open source projects, such as EODAG.

Release EODAG
4 changes: 2 additions & 2 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -39,9 +39,9 @@ https://github.com/jswhit/pyproj
https://github.com/kapadia/usgs
https://github.com/jupyter-widgets/ipyleaflet
https://github.com/GeospatialPython/pyshp
https://github.com/flasgger/flasgger
https://github.com/python-visualization/folium
https://github.com/Unidata/netcdf4-python
https://github.com/tiangolo/fastapi


================================================================
@@ -71,7 +71,7 @@ https://github.com/mapbox/rasterio
https://github.com/lxml/lxml
https://github.com/jupyter-widgets/ipywidgets
https://github.com/jupyter/jupyter
https://github.com/pallets/flask
https://github.com/encode/uvicorn

The function slugify, located at eodag/utils/__init__.py is a modified version of the function with the same name from
the Django Project, licensed under the BSD-3-Clause Licence. Follow project link below for more information:
35 changes: 15 additions & 20 deletions README.rst
Original file line number Diff line number Diff line change
@@ -74,7 +74,7 @@ EODAG is available on `PyPI <https://pypi.org/project/eodag/>`_:
python -m pip install eodag
And with ``conda`` from the `conda-forge channel <https://anaconda.org/conda-forge/eodag>`_:
And with ``conda`` from the `conda-forge channel <https://anaconda.org/conda-forge/eodag>`_:

.. code-block:: bash
@@ -124,17 +124,19 @@ An eodag instance can be exposed through a STAC compliant REST api from the comm
Start eodag HTTP server
Set EODAG_CORS_ALLOWED_ORIGINS environment variable to configure Cross-
Origin Resource Sharing allowed origins as comma-separated URLs (e.g.
'http://somewhere,htttp://somewhere.else').
Options:
-f, --config PATH File path to the user configuration file with its
credentials
-d, --daemon TEXT run in daemon mode
-w, --world run flask using IPv4 0.0.0.0 (all network interfaces),
otherwise bind to 127.0.0.1 (localhost). This maybe
necessary in systems that only run Flask [default:
False]
credentials, default is ~/.config/eodag/eodag.yml
-l, --locs PATH File path to the location shapefiles configuration file
-d, --daemon run in daemon mode
-w, --world run uvicorn using IPv4 0.0.0.0 (all network interfaces),
otherwise bind to 127.0.0.1 (localhost).
-p, --port INTEGER The port on which to listen [default: 5000]
--debug Run in debug mode (for development purpose) [default:
False]
--debug Run in debug mode (for development purpose)
--help Show this message and exit.
# run server
@@ -147,31 +149,24 @@ An eodag instance can be exposed through a STAC compliant REST api from the comm
"S1_SAR_SLC"
"S2_MSI_L1C"
"S2_MSI_L2A"
"S3_EFR"
"S3_ERR"
"S3_LAN"
"S3_OLCI_L2LFR"
"S3_OLCI_L2LRR"
"S3_SLSTR_L1RBT"
"S3_SLSTR_L2LST"
# search for items
$ curl "http://127.0.0.1:5000/search?collections=S2_MSI_L1C&bbox=0,43,1,44&datetime=2018-01-20/2018-01-25" \
| jq ".context.matched"
6
# browse for items
$ curl "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
$ curl "http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
| jq ".context.matched"
9
# get download link
$ curl "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
$ curl "http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
| jq ".features[0].assets.downloadLink.href"
"http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
"http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
# download
$ wget "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
$ wget "http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
``eodag-server`` is available on `https://hub.docker.com/r/csspace/eodag-server <https://hub.docker.com/r/csspace/eodag-server>`_:
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -21,6 +21,7 @@ services:
dockerfile: docker/stac-server.dockerfile
environment:
- "EODAG_LOGGING=${EODAG_LOGGING}"
- "EODAG_CORS_ALLOWED_ORIGINS=http://127.0.0.1:5001"
container_name: stac_server
restart: unless-stopped
networks:
2 changes: 1 addition & 1 deletion docs/notebooks/api_user_guide/4_search.ipynb
Original file line number Diff line number Diff line change
@@ -2185,7 +2185,7 @@
],
"source": [
"products, estimated_total_number = dag.search(\n",
" cloudCover=10, # cloud cover Less than 10\n",
" cloudCover=10, # cloud cover Less than 10\n",
" **default_search_criteria\n",
")"
]
2 changes: 1 addition & 1 deletion docs/notebooks/api_user_guide/7_download.ipynb
Original file line number Diff line number Diff line change
@@ -645,7 +645,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The storage status of a product can be obtained from its `storageStatus` field. The status of an `OFFLINE` product is updated by `eodag` to `STAGING` when ordered and to `ONLINE` when found available."
"The storage status of a product can be obtained from its `storageStatus` field. The status of an `OFFLINE` product is updated by `eodag` to `STAGING` when ordered and to `ONLINE` when found available."
]
},
{
2 changes: 1 addition & 1 deletion docs/notebooks/tutos/tuto_search_location_tile.ipynb
Original file line number Diff line number Diff line change
@@ -825,7 +825,7 @@
" tooltip=folium.GeoJsonTooltip(\n",
" fields=[\n",
" \"title\", # The product's title\n",
" \"tileIdentifier\", # The tile number on the MGRS grid\n",
" \"tileIdentifier\", # The tile number on the MGRS grid\n",
" ]\n",
" ),\n",
").add_to(fmap)\n",
30 changes: 16 additions & 14 deletions docs/stac_rest.rst
Original file line number Diff line number Diff line change
@@ -23,24 +23,26 @@ Below is the content of the help message of this command (`eodag serve-rest --he
Start eodag HTTP server
Set EODAG_CORS_ALLOWED_ORIGINS environment variable to configure Cross-
Origin Resource Sharing allowed origins as comma-separated URLs (e.g.
'http://somewhere,htttp://somewhere.else').
Options:
-f, --config PATH File path to the user configuration file with its
credentials, default is ~/.config/eodag/eodag.yml
-d, --daemon run in daemon mode [default: False]
-w, --world run flask using IPv4 0.0.0.0 (all network interfaces),
otherwise bind to 127.0.0.1 (localhost). This maybe
necessary in systems that only run Flask [default:
False]
-l, --locs PATH File path to the location shapefiles configuration file
-d, --daemon run in daemon mode
-w, --world run uvicorn using IPv4 0.0.0.0 (all network interfaces),
otherwise bind to 127.0.0.1 (localhost).
-p, --port INTEGER The port on which to listen [default: 5000]
--debug Run in debug mode (for development purpose) [default:
False]
--debug Run in debug mode (for development purpose)
--help Show this message and exit.
Searching
---------

After you have launched the server, navigate to its home page. For example, for a local
development server launched with ``eodag serve-rest -f <config> --debug``, go to
development server launched with ``eodag serve-rest -f <config> --debug``, go to
http://127.0.0.1:5000/service-doc. You will see a documentation of the interface.

Available operations are:
@@ -86,8 +88,8 @@ EODAG provides additional catalogs that extend browsing/filtering capabilities:

Example URLs:

* http://127.0.0.1:5000/S2_MSI_L1C/country : lists available countries
* http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2019/month/10/cloud_cover/10 : catalog referencing S2_MSI_L1C
* http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country : lists available countries
* http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2019/month/10/cloud_cover/10 : catalog referencing S2_MSI_L1C
products over France, aquired during October 2019, and having 10% maximum cloud cover

Browsing over catalogs can be experienced connecting EODAG STAC API to
@@ -146,14 +148,14 @@ Example
6
# browse for items
$ curl "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
$ curl "http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
| jq ".context.matched"
9
# get download link
$ curl "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
$ curl "http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items" \
| jq ".features[0].assets.downloadLink.href"
"http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
"http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
# download
$ wget "http://127.0.0.1:5000/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
$ wget "http://127.0.0.1:5000/catalogs/S2_MSI_L1C/country/FRA/year/2021/month/01/day/25/cloud_cover/10/items/S2A_MSIL1C_20210125T105331_N0209_R051_T31UCR_20210125T130733/download"
38 changes: 28 additions & 10 deletions eodag/cli.py
Original file line number Diff line number Diff line change
@@ -53,6 +53,7 @@
from importlib_metadata import metadata # type: ignore

import click
import uvicorn

from eodag.api.core import DEFAULT_ITEMS_PER_PAGE, DEFAULT_PAGE, EODataAccessGateway
from eodag.utils import parse_qs
@@ -621,7 +622,11 @@ def serve_rpc(ctx, host, port, conf):
server.serve()


@eodag.command(help="Start eodag HTTP server")
@eodag.command(
help="Start eodag HTTP server\n\n"
"Set EODAG_CORS_ALLOWED_ORIGINS environment variable to configure Cross-Origin Resource Sharing allowed origins as "
"comma-separated URLs (e.g. 'http://somewhere,htttp://somewhere.else')."
)
@click.option(
"-f",
"--config",
@@ -644,9 +649,8 @@ def serve_rpc(ctx, host, port, conf):
is_flag=True,
show_default=True,
help=(
"run flask using IPv4 0.0.0.0 (all network interfaces), "
"run uvicorn using IPv4 0.0.0.0 (all network interfaces), "
"otherwise bind to 127.0.0.1 (localhost). "
"This maybe necessary in systems that only run Flask"
),
)
@click.option(
@@ -677,11 +681,6 @@ def serve_rest(ctx, daemon, world, port, config, locs, debug):
if locs:
os.environ["EODAG_LOCS_CFG_FILE"] = locs

from eodag.rest.server import app, run_swagger, stac_api_config

# run swagger / service-doc
run_swagger(app=app, config=stac_api_config)

bind_host = "127.0.0.1"
if world:
bind_host = "0.0.0.0"
@@ -693,11 +692,30 @@ def serve_rest(ctx, daemon, world, port, config, locs, debug):

if pid == 0:
os.setsid()
app.run(threaded=True, host=bind_host, port=port)
uvicorn.run("eodag.rest.server:app", host=bind_host, port=port)
else:
sys.exit(0)
else:
app.run(debug=debug, host=bind_host, port=port)
logging_config = uvicorn.config.LOGGING_CONFIG
if debug:
logging_config["loggers"]["uvicorn"]["level"] = "DEBUG"
logging_config["loggers"]["uvicorn.error"]["level"] = "DEBUG"
logging_config["loggers"]["uvicorn.access"]["level"] = "DEBUG"
logging_config["formatters"]["default"][
"fmt"
] = "%(asctime)-15s %(name)-32s [%(levelname)-8s] (%(module)-17s) %(message)s"
logging_config["loggers"]["eodag"] = {
"handlers": ["default"],
"level": "DEBUG" if debug else "INFO",
"propagate": False,
}
uvicorn.run(
"eodag.rest.server:app",
host=bind_host,
port=port,
reload=debug,
log_config=logging_config,
)


@eodag.command(
1 change: 0 additions & 1 deletion eodag/config.py
Original file line number Diff line number Diff line change
@@ -15,7 +15,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# import copy
import logging
import os
import tempfile
2 changes: 1 addition & 1 deletion eodag/plugins/apis/base.py
Original file line number Diff line number Diff line change
@@ -48,7 +48,7 @@ class Api(PluginTopic):
is built on the MD5 hash of the product's ``remote_location`` attribute
(``hashlib.md5(remote_location.encode("utf-8")).hexdigest()``) and whose content is
the product's ``remote_location`` attribute itself.
- not try to download a product whose ``location`` attribute already points to an
- not try to download a product whose ``location`` attribute already points to an
existing file/directory
- not try to download a product if its *record* file exists as long as the expected
product's file/directory. If the *record* file only is found, it must be deleted
2 changes: 1 addition & 1 deletion eodag/plugins/download/base.py
Original file line number Diff line number Diff line change
@@ -67,7 +67,7 @@ class Download(PluginTopic):
is built on the MD5 hash of the product's ``remote_location`` attribute
(``hashlib.md5(remote_location.encode("utf-8")).hexdigest()``) and whose content is
the product's ``remote_location`` attribute itself.
- not try to download a product whose ``location`` attribute already points to an
- not try to download a product whose ``location`` attribute already points to an
existing file/directory
- not try to download a product if its *record* file exists as long as the expected
product's file/directory. If the *record* file only is found, it must be deleted
2 changes: 1 addition & 1 deletion eodag/resources/stac.yml
Original file line number Diff line number Diff line change
@@ -35,7 +35,7 @@ landing_page:
href: "{catalog[root]}/api"
- rel: service-doc
type: "text/html"
href: "{catalog[root]}/service-doc"
href: "{catalog[root]}/api.html"
- rel: conformance
type: "application/json"
href: "{catalog[root]}/conformance"
Loading

0 comments on commit 4c373fd

Please sign in to comment.