Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GNIP 83 - ResourceBase for metadata-only resources #7057

Closed
3 of 5 tasks
etj opened this issue Mar 10, 2021 · 6 comments
Closed
3 of 5 tasks

GNIP 83 - ResourceBase for metadata-only resources #7057

etj opened this issue Mar 10, 2021 · 6 comments
Assignees
Labels
gnip A GeoNodeImprovementProcess Issue
Milestone

Comments

@etj
Copy link
Contributor

etj commented Mar 10, 2021

GNIP 83 - ResourceBase for metadata-only resources

Overview

Flag ResourceBase instances in order not to be displayed in the GUI, but only available through CSW.

Proposed By

  • etj: Emanuele Tajariol

Assigned to Release

This proposal is for GeoNode .

State

  • Under Discussion
  • In Progress
  • Completed
  • Rejected
  • Deferred

Motivation

With this kind of "hidden" ResourceBase, our underlying pyCSW can be decoupled from the data handled in GeoNode/GeoServer, and regain a bit of the functionalities lost when using the internal pyCSW (the only one currenlty maintained) wrt external catalogs.

A good use case is the publishing of service metadata documents that details the WMS, WFS and WCS services offered by our GeoNode/GeoServer instance.

Proposal

The proposal is about adding a new boolean field ResourceBase.metadata_only, defaulting to false.

The queries used to present resources on screen should exclude the ResourceBase records having metadata_only=True.

The CSW endpoint should be able to handle such records, so no changes are needed on that side.

Please note that it's up to future development or external modules to add metadata only records.

Backwards Compatibility

The DB migration process will set the new fields to the default value, so existing data will all be visibile on GUI.
If no external logic is added which creates metadata only records, no changes will be visibile on the GeoNode instance.

External clients that use a generic API that returns all of the ResourceBase fields should be aware of the new attribute.
Anyway the presence of records with the metadata_only attribute set will be well-known in the GeoNode setup (since by default nothing will add such records), so the system architects shall consider how the metadata_only records will be used in the specific context.

External clients that use CSW should be aware that they will get all of the records. Anyway the same recommendations in previous paragraph applies here as well.

Future evolution

We're developing a contrib app that allows the admin to add service metadata to the catalog.

Feedback

Update this section with relevant feedbacks, if any.

Voting

Project Steering Committee:

  • Alessio Fabiani: +1
  • Francesco Bartoli:
  • Giovanni Allegri: +1
  • Simone Dalmasso:
  • Toni Schoenbuchner: +1
  • Florian Hoedt: +1

Links

Remove unused links below.

@gannebamm
Copy link
Contributor

please see #6744 and #6876 which discussed similar concepts.

I do not understand why metadata only Resources should not be part of the search system. Furthermore, I do not understand the use-case:

A good use case is the publishing of service metadata documents that details the WMS, WFS and WCS services offered by our GeoNode/GeoServer instance.

Currently, a dataset is uploaded and stored as 'layer' which itself is:

  • the base item (MD_DATASET)
  • WMS
  • WFS
  • ISO Feature- Catalogue

So what exactly is meant by 'details the WMS, WFS and WCS services'?

@etj
Copy link
Contributor Author

etj commented Mar 10, 2021

@gannebamm

So what exactly is meant by 'details the WMS, WFS and WCS services'?

A layer creates an ISO19115 metadata belonging to the "dataset" hierarchy, which documents data.
As a dataset, in its gmd:onlineResource elements it documents the endpoints to access the layer in its different formats, and with URLs referring the WMS, WFS, ... services, but in relation to how the layer data is distributed.
The metadata also describes the Layer's responsiblePartys, i.e. responsibles for the metadata describing the layer, responsibles for the layers data, and their respective roles (point of contact, author, owner, custodian, publisher, ...)

A service metadata is an ISO19115 metadata which describes the service in its entirety.
The responsible parties are the ones running the service, which are not described in the data.
Service metadata also describe the various operation available in the service, which operations are supported, which are the standards it conforms to, which are other qualitative and quantitative constraints it follows.

So, if I have a CSW catalog, I want to query, for instance, for "view services" to obtain the available WMS endpoints, or for "download services" to get WFS and WCS endpoints, which are not the endpoints provided by the layers metadata, but information about the services themselves.

INSPIRE requires that a catalog should provide service metadata for an organization. If GeoNode can not provide them, then GeoNode can only be used as a slave catalog by some other high level catalog that can include such extra metadata. With this GNIP we can remove the use of other catalogs in the architecture.
Please also note that GeoNode, when using external catalogs, was able to deal with this use case; when using a local pyCSW we are binding CSW records to ResourceBase records, so we are forced to improve the ResourceBase functionalities in order to
deal with records that we want to make available at CSW level, but that are useless (or even confusing) in the UI.

@gannebamm
Copy link
Contributor

@etj
I now understand your concept and like it! Regarding the linked issues #6744 and #6876 the mentioned metadata_only field can be used to mark items that are just metadata entries, eg. as fetched from foreign CSW or other data catalogues.

@t-book t-book added the gnip A GeoNodeImprovementProcess Issue label Mar 11, 2021
@afabiani
Copy link
Member

+1

@gannebamm
Copy link
Contributor

Just a little sidenote and house-keeping:

    Alessio Fabiani: +1
    Francesco Bartoli:
    Giovanni Allegri:
    Simone Dalmasso:
    Toni Schoenbuchner:
    Florian Hoedt: +1

I think this is not a big deal and I am happy this GNIP is worked upon and it is almost finished. But we as PSC (@t-book @afabiani @giohappy @francbartoli @simod) should keep an eye on the voting and remind the other members to give their +1/0/-1 !

@t-book
Copy link
Contributor

t-book commented Mar 22, 2021

Thanks for reminding @gannebamm +1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gnip A GeoNodeImprovementProcess Issue
Projects
None yet
Development

No branches or pull requests

5 participants