GNIP 81: Add metadata records as models

[gannebamm]

GNIP 81 - Add metadata records as models

Overview

This GNIP shall enable GeoNode to hold just metadata objects, instead of whole datasets with GeoServer services and PostGIS tables. A user can create a new record which describes his/her dataset without uploading the dataset itself. This metadata is findable through the standard search widget of the GeoNode portal.

Proposed By

@gannebamm

Assigned to Release

This proposal is for GeoNode 3.x

State

• Under Discussion
• In Progress
• Completed
• Rejected
• Deferred

Motivation

Our research institute does collaborate with many other institutes which do have several datasets on their own infrastructure. Currently, those datasets can be added as remote services (WMS) but not every dataset is exposed as service or is geodata. At the same time, we would like to use GeoNode as a single entry point for search and need to get the metainformation about those datasets inside our Node.

Proposal

A plain metadata record model as a subtype of resourceBase could be used to get this working. The additional logic in the Layer or Map objects are not needed. It should be very similar to the document model.

Backwards Compatibility

It is a new model which does not interfere with old instances.

Future evolution

This base record model could be used to enable CSW harvesting and OGC records API.

Feedback

Update this section with relevant feedbacks, if any.

Voting

Project Steering Committee:

• Alessio Fabiani:
• Francesco Bartoli:
• Giovanni Allegri:
• Simone Dalmasso:
• Toni Schoenbuchner:
• Florian Hoedt:

Links

• Gitter Discussion

[francbartoli]

Is this overlapping with pycsw and eventually with the next upcoming OARec implementation? By the way, a user is already able to upload a metadata record without the need to pick up the actual dataset AFAIK

[gannebamm]

Is this overlapping with pycsw and eventually with the next upcoming OARec implementation? By the way, a user is already able to upload a metadata record without the need to pick up the actual dataset AFAIK

Since there is an overlap I wanted to discuss this.

I do not know how you are able to upload or create just a metadata object. You can upload a document (eg. a CSW getRecord XML) but it currently does not read its contents to pre-fill the metadata info for the document. Also, the naming of it being a 'document' is strange for a user. If there are good alternatives to fulfil my users feature requests I am willing to take them 😄

[francbartoli]

Is this overlapping with pycsw and eventually with the next upcoming OARec implementation? By the way, a user is already able to upload a metadata record without the need to pick up the actual dataset AFAIK

Since there is an overlap I wanted to discuss this.

I do not know how you are able to upload or create just a metadata object. You can upload a document (eg. a CSW getRecord XML) but it currently does not read its contents to pre-fill the metadata info for the document. Also, the naming of it being a 'document' is strange for a user. If there are good alternatives to fulfil my users feature requests I am willing to take them 😄

You can upload a layer that is just the metadata XML record.

Adding such a new Record object would look like to reinvent a CSW implementation natively in GeoNode IMHO.

[t-book]

@gannebamm as you're saying the new model could inherit from resourceBase you see it models around 19115:2003 as well right? How would a user add a record. Is it just a simple form like when editing metadata?

[gannebamm]

Hi @t-book
Yes I would see it as very simple resourceBase implementation without additional fields and logic. To add a record you should be able to open the metadata wizard and link to a foreign URL like documents do. Furthermore an already created xml file should be able to get uploaded and ingested (like layers do). Later this should be able to harvest csw endpoints like remote layers do with wms.

This is my idea and this should be discussed and shaped to work well. As @francbartoli told above xml files could be used to upload just metadata for foreign layers. I was not able to test this yet. Maybe this could be enough for now.

In the end I would like geonode to be able to harvest csw

[gannebamm]

In the end I would like geonode to be able to harvest csw

This is actually the feature we want. To accomplish that somehow the CSW getRecords response items must be stored.

[gannebamm]

You can upload a layer that is just the metadata XML record.

@francbartoli
Actually, you can not. It will fail with:

Client Error: Bad Request
Shapefile , GeoTIFF , ImageMosaic , ASCII Text File , PNG , JPG , CSV , GeoJSON , KML , KML Ground Overlay , ERDASImg , NITF , CIB1 , CIB5 , CIB10 , GNC , JNC , ONC , TPC , JOG , TLM100 , TLM50 , MrSID , JP2 files are supported. You uploaded a xml file

[t-book]

@gannebamm You've already said that using the documents section might be irritating, however to me it looks quite close to your needs (and the term document describes more or less what it is?):

Let's say next to txt, jpg, etc. we would have a new document type called "external link", choosing that the interface would even not force an upload but give an input field to add the external link.

• After submitting you (already) arrive at the known metadata wizard
• After saving, your record (already) will be indexed for searching and caught by csw

I've tried something similar here with a simple text file but do understand that it's not practical for users to upload an empty file.
https://master.demo.geonode.org/documents/6054

Just my two cents.

[gannebamm]

Good point.

The UI and texts need to get modified for those type of records. Eg. the download this document buttons and texts should say something like open or something similar.

Furthermore if the uploaded document is a XML metadata file like DublinCore or ISO19139 the fields should get read just like an layer XML file. This logic could than be used to implement a CSW harvesting routine?

What do you guys think?

[gannebamm]

After some more discussions and time to think about the GNIP I have come to the following conclusions:

1. Creating a new model to enable features which are mostly already present in GeoNode is overkill and will make the codebase bloaty
2. To close the feature gap between our needs and what's already present in the document model we could:

a) Revise the current document model and add the functionalities
b) Create a contrib-app (like EXIF for images) to close the gap

I would likely vote for a). But this is a different discussion and I would reject and close this GNIP. The document revision GNIP will be a new one. Do the PSC members agree?

[t-book]

yep, I do agree

[gannebamm]

As stated above #6744 (comment)

This GNIP is rejected and withdrawn. There will be a new revised GNIP describing refactoring of XML metadata ingest code to work for Layer and Document models.