Skip to content

Application Requirements

Jump to bottom
KoalaGeo edited this page Aug 16, 2021 · 10 revisions

Infrastructure / Frameworks

  • Docker
  • Kubernetes

Standards

Shall conform with OGCAPI-Processes

Implementation Options

  1. Use pygeoapi
  1. Customized FastAPI Docker Image https://hub.docker.com/r/tiangolo/uvicorn-gunicorn-fastapi
  • Make conform with OGCAPI-Processes

Requirements

Offer all functionality of https://gitlab.com/ags-data-format-wg/ags-python-library CLI

Match

AGS Desktop

Upload AGS file / batch of files Validate those files with print out of validation Conversion of AGS to xlsx and xlsx to AGS

GUI Able to be reskinned simply - COPY .html & .css in the dockerfile

BGS Rules

They’ve highlighted a few rules they’d like to see in the AGS Validator/The AGS standard if not already (they’ve not had chance to look at the code yet), mainly as they’re required for our data centre use case and they’d like to see the official AGS validator integrated as part of our own workflows.

  1. basic formatting: number of columns must match number of column headers
  2. groups must include PROJ, LOCA or HOLE, ABBR, TYPE, UNITS (and GEOL for BGS)
  3. FILE is required if the AGS file is submitted with other supporting non-AGS files
  4. DICT is required if any user defined groups or headings are present, otherwise only standard/default settings are used
  5. BGS system populates DICT with standard/default terms if no user defined groups are present.
  6. LOCA/HOLE file must contain a minimum of 75 per cent valid coordinates
  7. If it doesn’t, doesn’t mean we won’t take it, but we would want to check with the submitter why it doesn’t
  8. the spatial referencing sytem or projection format should be specified in LOCA_GREF, LOCA_LREF or LOCA_LLZ columns (or HOLE equivalent)
  9. OSGB coordinates should be in the LOCA_NATE and LOCA_NATN columns(or HOLE equivalent)
  10. local grid reference coordinates should be in the LOCA_LOCX and LOCA_LOCY columns(or HOLE equivalent)
  11. LAT/LON coordinates should be in the LOCA_LAT and LOCA_LON columns(or HOLE equivalent)
  12. local coordinates should not be duplicated in the OSGB columns as this would indicate they have not been converted
  13. If this case the BGS ingestion removes the co-ords from the OSGB columns and assumes local grid
  14. If OSGB and local cords are different they are assumed correct….
  15. LOCA_NATE/N sometimes contain Irish Grid co-ords which is correct, but it will fail stage2 validation (inside UK+NI Boundary check)
  16. In stage 2 validation these are copied to new columns and the OSGB co-ords are regenerated and inserted where possible (SW Ireland isn’t covered otherwise Lat/Long generated)
  17. Check inside GB/Ireland coastline
  18. Check Test & Borehole ID’s match
  • Format - Correct formatting containing no comma or quote errors, which may prevent the file from being processed automatically.
  • Coordinates - Coordinates should be provided for all boreholes in British National Grid.
  • Key fields - All key fields must appear in each group but may contain null data. Key fields are necessary to uniquely define the data.
  • No orphan records - All records should relate to a parent sample or hole.
  • No duplicate records - No duplicate records should be present as this breaks unique identifier rules.
  • Complete information - Important but not mandatory fields should be populated, such as PROJ_CONT or PROJ_ID.
  • Translated codes - All codes used should be present in ABBR.
  • Numerical values - Although text can be stored in all AGS fields, this is not always appropriate for values relating to units, so these should contain numbers with remarks in a remarks field, if required.
  • Lithostratigraphy Codes - Lithostratigraphy should be recorded as a BGS Lexicon code in GEOL_BGS. It should not just be recorded as part of the GEOL_DESC text.
  • Null means null - Where there is no value the field should be null; it should not be padded out with zeros as this may affect statistical analysis.
Clone this wiki locally