Ogcapi records - better "catalog.yaml" objects

[davidblasby]

This improves JSON output for the landing page (/), /collections, and /collections/<collectionid>.

In the output of these I inject the OGCAPI Records catalog.yaml object.

A GN Portal (full portal or sub-portal) is an OGCAPI Catalog.

1. Getting info

This will retrieve the GN ServiceRecord that's linked to a GN Source DB row. This can be done in the GN GUI: Admin Console-Settings-sources for sub-portals.

There is no GUI to set this for the "main" portal - do this in GN GUI: Admin Console-Settings-Settings-Catalog Service for the Web (CSW)-Record to use for GetCapabilities (system/csw/capabilityRecordUuid).

Basically, this PR will retrieve this linked record via the Elastic Index (JSON). This is processed in GN via the index-utils.xsl.

If there isn't a linked service record, then the catalog details will not be added (i.e. unchanged from how GN ogcapi records used to work).

3. New Model Objects

I have put in a bunch of new model objects - OgcApi*. These are 1:1 with the OGCAPI object model - defined in the spec as well as the .yaml files. I've copy-and-pasted in the documentation from the .yaml files for readability.

NOTE: I also moved the old Link object to OgcApiLink for consistency.

4. Landing page (/)

The landing page JSON also includes a new property systemInfo which contains a catalog.yaml object in it.
I'm not sure if this is allowed in the spec, but it allows for more metadata about the entire GN system ("ogcapi-records" server). This is useful for making a nicer landing page (cf. pygeoapi's landing page).

5. A few details

Most of the work is done in ElasticIndexJson2CollectionInfo - this converts the Elastic Index JSON object to the ogcapi catalog.yaml object. There's also a simple test case in ElasticIndexJson2CollectionInfoTest.

6. Example

Underlying XML linked Service Record (xml)
9bac358b-11ec-4293-aeef-5a077b778412.txt

Elastic Index (JSON):
elastic-index-record.txt

OGCAPI catalog.yaml output (JSON):
ogcpai-catalog.json

[davidblasby]

Original PR is here - davidblasby#1

(reviewed by @josegar74 )

@fxprunayre - would love to talk to you about this during the sprint this week.

[davidblasby]

added 2 commits;

1. update CORS to allow connections
   This shouldnt be a problem because the ogcapi is all GET-based (readonly)
2. add collection (catalog/sub-portal) icon

[davidblasby]

@josegar74 @fxprunayre @f-necas I'd like to merge this is this week. I'm working on Queryables and Search right now (#116), and want to finish this up.

There is a Sonar warning - this is because I set the CORS so the ogcapi-records is accessible as a service. They are all GETs at the moment, so i don't think this is too big an issue.

[josegar74]

@davidblasby If the metadata has online resources, the geojson output fails with 406 error. for example with the sample metadata Hydrological Basins in Africa (Sample record, please remove!)

http://localhost:9901/collections/main/items/da165110-88fd-11da-a88f-000d939bc5d8

I see in the log:

com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "hash" (class org.fao.geonet.index.model.gn.Link), not marked as ignorable (9 known properties: "nilReason", "protocol", "applicationProfile", "nameObject", "descriptionObject", "mimeType", "group", "function", "urlObject"])
 at [Source: (String)"{
  "docType" : "metadata",
  "document" : "",
  "metadataIdentifier" : "da165110-88fd-11da-a88f-000d939bc5d8",

Apparently some issue with the index model in GeoNetwork adn the class that models links in this project.

Recently was added a hash property in the links.

@davidblasby to solve this issue, please update https://github.com/geonetwork/geonetwork-microservices/blob/main/modules/library/common-index-model/src/main/java/org/fao/geonet/index/model/gn/Link.java

And add:

  private String hash;
  private String idx;

[josegar74]

About

4. Landing page (/)
The landing page JSON also includes a new property systemInfo which contains a catalog.yaml object in it.
I'm not sure if this is allowed in the spec, but it allows for more metadata about the entire GN system ("ogcapi-records" server). This is useful for making a nicer landing page (cf. pygeoapi's landing page).

You mean that pygeoapi's uses this systemInfo also? or it's completely custom to GeoNetwork implementation?

We should check if there is a validator available, but last time I checked some months ago, that was not the case.

[davidblasby]

You mean that pygeoapi's uses this systemInfo also? or it's completely custom to GeoNetwork implementation?

No - Its unclear where pygeoapi gets its information from (its not in its JSON output).
there is no systeminfo inside pygeoapi. I just added it so an external (i.e. angular) app would have access to this info.

[davidblasby]

@josegar74 - added the hash/idx.

[davidblasby]

I updated this to use the parsed IndexRecord instead directly from the elastic index json.

This is in prep for GeoJSON changes.

[sonarcloud]

Quality Gate failed

Failed conditions
1 Security Hotspot

See analysis details on SonarCloud

[davidblasby]

@josegar74 @fxprunayre - will merge this tomorrow.

[fxprunayre]

Sounds fine to add this @davidblasby. There is some issues
http://localhost:9901/collections/inspire?f=json will return service info
http://localhost:9901/collections/inspire?f=xml will not (related to getRecordAsJson and request headers) but for demo purpose that can be ok.

Also we should plan to move it to GeoNetwork 5 codebase at some point

[davidblasby]

@fxprunayre - I havent been looking at the XML output - just the JSON. I will put that on the list!

Whats involved with moving to the GN5 codebase? I don't know anything about it...

[fxprunayre]

@fxprunayre - I havent been looking at the XML output - just the JSON. I will put that on the list!

Not urgent.

Whats involved with moving to the GN5 codebase? I don't know anything about it...

We are cleaning up the POC with @josegar74 and we will move the repository to GeoNetwork organisation as discussed in Bennekom. Then we can look into it.