Conformance classes

[pvretano]

The purpose of this issues is to discuss the conformance classes that should be in the core. At the moment, I am contemplating the following conformance classes in the core:

1. Core
2. Sorting
3. Collections
4. OpenSearch
5. JSON
6. ATOM
7. HTML

The Core Requirements Class is the minimal useful service interface for an OGC Records API. The requirements specified in this Requirements Class are mandatory for all implementations this standard. The core requirements class includes a set of core properties also referred to as core queryables and a set of core query parameters. The list of core queyrables includes:

NOTE: The requirement column in the following table is meant to indicate whether the corresponding queryables is also a returnable (i.e. a property that MUST appear in the record response).

NOTE: It is STRONGLY recommended that the other queryables also be included as returnables.

Queryable	Requirement	Description
recordId	M	A unique record identifier assigned by the catalogue.
recordcreated	O	The date the records was created in the catalogue.
recordmodified	O	The most recent date on which the record was changed.
title	M	A human-readable name given to the resource.
description	O	A free-text description of the resource.
keywords	O	A list of keywords or tags describing the resource.
type	M	The nature or genre of the resource.
language	O	This refers to the natural language used for textual values (i.e. titles, descriptions, etc) of a resource.
externalId	O	An identifier for the resource assigned by an external entity (i.e. not the catalogue).
modified	O	Most recent date on which the resource was changed.
publisher	O	The entity for making the resource available.
themes	O	A knowledge organization system used to classify the record.
formats	O	A list of available distribution formats for the resource.
contactPoint	O	An entity to contact about the resource.
license	O	A legal document under which the resource is made available.
rights	O	A statement that concerns all rights not addressed by the license, such as copyright statements.
extent	O	The spatio-temporal coverage of the resource.
links	O	A list of links for navigating the catalogue API.
associations	O	A list of links to resources associated with this resource.

The list of core query parameters includes:

Query Parameter	Description
bbox	A bounding box. If the spatial extent of the record intersects the specified bounding box then the record shall be presented in the response document.
datetime	A time instance or time period. If the temporal extent of the record intersects the specified data/time value then the record shall be presented in the response document.
limit	The number of records to be presented in a response document.
type	A resource type. Only records of the specified type shall be presented in the response document.
q	A space-separated list of search terms. If any server-chosen text field in the record contains 1 or more of the terms listed then this records hall appear in the response set.
externalIds	A comma-separated list of external identifiers. Only records with the specified names shall appear in the response document.

The Sort Requirement Class defined requirements needed to perform sorting of search results as per Uwe's proposal (see Issue #136).

The Collections Requirements Class defines requirements for making the /collections endpoint of any OGC Web API searchable using the OGC API Records core queryables and core query parameters. If a server wants to make is /collections end-point queryable it has to do the following:

1. Add the appropriate conformance URIs to its /conformance document. There may be more than one depending on the level of query capability offered. http://www.opengis.net/spec/ocgapi-records-1/1.0/queryable-collections or http://www.opengis.net/spec/ocgapi-records-1/1.0/cql-queryable-collections
2. Add the mandatory core queryables as keys in the collection object if those keys are not already there.
3. Add zero or more optional core queryables to the collection object.
4. Implement the core query parameters of the /collection end-point (core+).
5. The response is exactly what it would be if you fetched the /collections resource but the only collections listed would be the ones that satisfy the query predicates.

The OpenSearch Requirements Class defines the requirements needed to support OpenSearch queries. Specifically this means that the implementation can generate an OpenSearch Description document and optionally generate an ATOM response. The OpenSearch Geo extension has more details.

The JSON-Record Requirements Class defines the requirements for a JSON representation of a standard catalogue record. This is basically GeoJSON.

The ATOM Requirements Class defines the requirements for an ATOM representation of a standard catalogue record.

The HTML-Record Requirements Class defines the requirements for an HTML representation of a standard catalogue record.

[cportele]

Minor comments:

• It should be up to the API to decide which fields to search for the value of q.
• No need to include an OpenAPI 3.0 req class, this is already defined in Common Core.
• We should look at the details of the sort parameters and make sure that the parameters are defined in a way that maps directly to OpenAPI parameter definitions.

[nmtoken]

Does core not include the date the metadata record was created/last updated?
Is identifier the identifier of the metadata ? if so should there also be an identifier for the resource?

[pvretano]

2020;07-13 SWG Meeting:
(1) Carl will crosswalk the core queryables with what was done in CDB.
(2) names core query parameter -> remove; server can implement but it is not mandatory (similar to how offset is handled in OAPIF
(3) type - will be have a controlled vocabulary? Normally, a catalogue would provide the list of valid values for type in its OpenAPI document. Whether that list of values is taken from an authoritative vocabulary is an orthogonal issues (Issue #26
(4) Can the /queryables response include the list of valid values for "type"? Possibly, that depends on the structure of the /queryables response and is an open question.
(5) Note in the specification that "description" is synonymous with "abstract".
(6) Rename "id" to "recordId" and leave externalId as is. Server implementations can decide whether to cast one of the external identifiers to be the recordId.
(7) Sorting should be part of the core. Uwe's proposed specification does not map well to OpenAPI; we need to revisit that.
(8) The collections conformance class should be an extension and not part of the core.
(9) Timeline? Need to discuss this at the next meeting but we are getting closer.
(10) Add a multiplicity column to the core queryables (e.g. externalIds is probably 0..N).

[pvretano]

@nmtoken
(1) No. The modified date is the date the resource being described by the record was last modified. However, there is no restriction on adding a "recordModified" property or something like take to the record.
(2) There are two classes of identifiers. There is the "recordId" which is the id of record in the catalogue. There is also the "externalId" which is the identifier assigned to the resource by some external (i.e. not the catalogue) authority. Implementations of the catalogue can, if they wish, cast one of the externalId's as the recordId or a catalogue might choose, as mine does, to generate a unique recordId each time a record is added to the catalogue.
(3) I think the "externalId" is the resource identifier.

[mhogeweg]

#1 it would make harvesting more efficient if there is a process to retrieve records added/update since a certain date/time (the last time harvesting happened)

[pvretano]

@mhogeweg Agreed. We have not had much/any discussion about transactions or harvesting yet. This capability would almost certainly end up being a different part than Part 1: Core.

[nmtoken]

@pvretano it seems an oversight to not have the metadata date in core, it is a mandatory requirement for ISO 19115

[nmtoken]

For harvesting of metadata, metadata date is important, not just to target records to be harvested from a catalogue, but also (related) to ensure that a newer record is not overwritten by an older version.

And in cases when the same metadata record exists in multiple catalogues, for a user to know which is the most up-to-date version.

[pvretano]

@nmtoken The core is read only. How the catalogue is populated is not in scope for the core. That is the subject of another part of the specification which we have barely discussed. When we get to that, we will include additional metadata in the record to enable efficient harvesting, etc.
F.Y.I. My implementation, for example, uses the Harvest operation from 3.0 to populate my catalogue.

[nmtoken]

@pvretano harvesting aside metadata date is still important, it should be in core. I don't understand why core doesn't include metadata regarded as mandatory by ISO 19115 (there are only 7 and metadata date is one of them).

[mhogeweg]

I think the core queryables and how the query is actually made need some work (unless already described elsewhere). some of the queryables are dates, some may be numbers, phone numbers, etc.

A space-separated list of search terms. If any server-chosen text field in the record contains 1 or more of the terms listed then this records hall appear in the response set.

misses the opportunity to search in specific fields, like title:WHYMAP.

I suggest something like: "if a search term includes a queryable separated by a colon from the search term, the search shall be restricted to the catalog field mapped to the queryable"

[tomkralidis]

Given that this is issue 42 we should ensure that this issue contains all the answers ;)

[pvretano]

LOL ... sure thing Arthur (aka @tomkralidis)!

[pvretano]

@mhogeweg the core has a deliberately limited set of query capabilities in order to make implementations of core simple. The SWG has discussed handling complex queries by a small set of increasingly more capable query extensions such as an OpenSearch extension and CQL (see https://github.com/opengeospatial/ogcapi-features/tree/master/extensions/cql) where the client has more control over fields, operators, expressions, etc. I like your suggestion of "title:WHYMAP" but I think that would go into one of the more capable extensions such as an OpenSearch extension.

[pvretano]

19-OCT-2020 SWG MEETING: The "requirement" of the queryables was meant to indicate whether that queryable must appear in a records response (i.e. a returnables as well). Upon review of the which are mandatory and which are optional we determined that the set of mandatory "returnables" is too large. So, we need to do 2 things ... (1) create a new table called returnables that lists the subset of queryables that must appear is a records (i.e. id, type, title (maybe) (2) include a strong recommendation for the others.

[pvretano]

11-JAN-2020: Keep the set of core queryable properties to the current set (bbox, datetime, q, type, etc.). Add text to the specification to give permission to add additional URL query parameters based on the core queryables BUT indicate that the nature must be the same as defined in the specification.

[tomkralidis]

Perhaps it would be valuable to have suggestive text explaining that while adding additional query parameters is allowed, it is advised to review against the queryables defined in the spec so as to maximize interoperability with clients and other servers.

[bradh]

It looks like /api is considered core as inherited from ogcapi-common (from https://github.com/opengeospatial/ogcapi-records/blob/master/core/standard/clause_7_core.adoc#api-definition).

However there is still a separate oas30 conformance class (https://github.com/opengeospatial/ogcapi-records/blob/master/core/standard/clause_13_oas.adoc) with required conformance statement (https://github.com/opengeospatial/ogcapi-records/blob/master/core/standard/requirements/oas30/REQ_oas-conformance.adoc).

That is a bit confusing. Should the clause_13_oas.adoc part (and direct references) be removed?

I do see some SWGs (e.g. tiles) moving away from a hard dependency on common, as they try to do building blocks. I think that is a bad idea though, partly because it makes the requirements much more vague, and adds "if you are using common, then the way you could do this is..." (e.g. see https://github.com/opengeospatial/ogcapi-tiles/blob/master/core/standard/clause_7_tile_tileSet.adoc#response).

[pvretano]

@bradh HISTORY: the document you are looking at and have been editing was initially copied from COVERAGES which was the first specification that tried to extend from core. We are trying to do the same with RECORDS (i.e. not what tiles is doing). So, there are quite a few artifacts hanging around which I am in the process of removing. I am currently in the MIDDLE of getting the document into better shape for review by the SWG and one of the items on the list is to actually remove clause 13. I think we already discussed this removal a few SWG meetings ago. The next time I push ... sometime in the upcoming week ... Clause 13 should be gone.

[pvretano]

SWG MEETING 08-FEB-2021: Will remove OpenAPI clause and close.

[pvretano]

Removed OAS clause. Closing.