DCAT-AP compatibility

[EmidioStani]

Description

By supporting DCAT, DKAN can improve its interoperability.

DKAN already supports DCAT looking at: https://github.com/NuCivic/dkan/blob/935d0c7266f411f641feadda3f4b88188c076784/import/rdf.php

but it seems not updated to the recent standard.

By updating to the recent standard DKAN would be more interoperable with CKAN:

https://github.com/ckan/ckanext-dcat#rdf-dcat-to-ckan-dataset-mapping

User Stories

As CKAN/DKAN site supporting DCAT-AP 1.1 I would like to harvest/be harvested from other portal supporting the latest standard

Tests

Test should be conducted between 2 portals (harvested and harvester)

Documentation

https://joinup.ec.europa.eu/asset/dcat_application_profile/description

Pull Requests

Acceptance Criteria

[akuckartz]

👍

[dafeder]

Thanks for flagging, EmidioStani, we will look at this. We have put more effort recently into supporting Project Open Data and data.json, which is derived from DCAT and used more directly, at least in the US context. But DCAT is important and we'll look into what gaps exist in our implementation.

[acouch]

This tool will be useful https://validator.dcat-editor.com/

[EmidioStani]

I could be biased but I developed this validator on behalf of the European Commission:
http://data.opendatasupport.eu:3030/dcat-ap_validator.html

The code is here: https://github.com/EmidioStani/dcat-ap_validator

[acouch]

@EmidioStani we'll use yours, thanks!

[dafeder]

We're working on adding DCAT-AP 1.1 support to the Open Data Schema Map module. Expect to have it out in a few weeks!

[dafeder]

@EmidioStani we could use some of your expertise to help guide this effort - in particular we're struggling a bit with the lack of canonical examples or machine-readable schemas. For instance, we based our output for project open data on the following:

https://github.com/project-open-data/project-open-data-dashboard/tree/master/schema/federal-v1.1

[marciuz]

Hi,
I'm interested too in the DCAT-AP harvester in DKAN.
There are some news about it?

thank you

[dafeder]

@marciuz we are working on a DCAT-AP endpoint (for harvesting by other catalogs) that will be included in the next point release of DKAN (1.13), due out in October. To be clear, functionality for DKAN to harvest from other DCAT-Ap endpoints is still farther off; the same 1.13 release will ship with a Harvester but will initially only harvest from Project Open Data (data.json) out of the box. Contirbutions always welcome though!

[marciuz]

Dear @dafeder
thank you for your quick reply.
What I would like to do is exactly an harvester for the other DCAT-AP endpoints, with also some additional specification for a local extension. In order to avoid an effort duplication, maybe can I explain better via email what I have to do?

thank you!

[dafeder]

I like to leave these conversations out in the open @marciuz - if you'd prefer we can discuss in our gitter room

[akuckartz]

For those interested in the future of DCAT (and therefore DCAT-AP): The W3C is organising a workshop in Amsterdam 30 November - 1 December 2016 (https://www.w3.org/2016/11/sdsvoc/).

Workshop aims:

The outcome may be a new W3C Working Group chartered to extend DCAT and determine how human and machine-readable metadata profiles are defined and made discoverable.

I intend to participate.

[marciuz]

@dafeder, fine with me!
I have to implement an harvester for the DCAT-AP_IT standard, the italian implementation of DCAT-AP. It is very similar, there are some minor changes about the mandatory fields and some other small difference.

What is in your opinion the best way to organize the work in order to re-use your work and to product code useful also for the DKAN community?

[janette]

Fixed in the 7.x-1.13 release http://dkan.readthedocs.io/en/latest/apis/open-data.html?highlight=dcat#dcat-ap

[dafeder]

@marciuz did you get anywhere with the harvest source type for DCAT-AP?

[acouch]

Closing this ticket as it has not been updated recently and is not on our near-term roadmap. Please re-open if you would like to discuss further or are actively working on it.

[akuckartz]

I would like to mention that dcat-ap.de likely will become a standard to be used by all government levels in Germany.

[akuckartz]

I suggest to reopen this issue because it needs to be resolved for #2595.

[acouch]

@akuckartz thanks for noting this. We will try and put some resources into this as you've noted there is some activity you linked to above.

[dafeder]

@acouch @akuckartz I'm not clear on what the action item is here; the scope of this issue has kind of gotten lost. Can we articulate what's missing in a new issue and close this one?

[akuckartz]

As far as I am concerned the action item is to find out If anything is missing from support for DCAT-AP (and DCAT-AP.de). In addition to that I suggest to follow the discussions in the W3C WG.

[jjalcolea]

Hi. As european DKAN users, we are also very interested on this. What is the situation? We're checking DKAN support of DCAT-AP, maybe we can help in identifying potential incompatibilities? Or is this already solved?

Best,
Juanje.

[jjalcolea]

Hi, by comparing between what is available in the /catalog.xml DKAN URL and the DCAT 1.2.1 standard, we found the following:

1. All mandatory classes and properties seem to be present :-)
2. But the way values are expressed sometimes do not conform to the Range specified in the standard
3. Some values seem to be "hardcoded" and tied to certain site parameters

Here are our detailed findings:

CATALOG
These values seem to be "hardcoded"; Where/how can the following DCAT values be defined in DKAN?

• Description
• Publisher
• Title
• Language

The values of these properties do not conform to the Range specified in the DCAT standard:

• Publisher: value should use foaf:Agent
• homepage: value should use foaf:Document
• language: value should use dct:LinguisticSystem

Also, why "issued" and "modified" always share the same date?

DATASET
The values of these properties do not conform to the Range specified in the DCAT standard:

• Publisher: value should use foaf:Agent
• Theme: value should use skos:Concept
• conformsTo: value should use dct:Standard
• landingPage: value should use foaf:Document
• accrualPeriodicity: value should use dct:Frequency

DISTRIBUTION
The values of these properties do not conform to the Range specified in the DCAT standard:

• format: value should use dct:MediaTypeOrExtent
• license: value should use dct:LicenseDocument
• mediaType: value should use dct:MediaTypeOrExtent
• byteSize: value should be typed as xsd:decimal

AGENT
These values seem to be "hardcoded"; Where/how can they be changed in DKAN?

• Name
• Type

The values of these properties do not conform to the Range specified in the DCAT standard:

• type: value should use skos:Concept

Are these discrepancies true or am I missing something?

Best,
Juanje.

[susisiman]

Closing due to inactivity - feel free to open again!

[akuckartz]

This is not resolved. Please reopen.

[dafeder]

I think the current approach for this would be, if we implement #3761, to provide modules for both DCAT-AP and DCAT-US. I'll keep this open as a reminder for that.