export bald as schema.org #80

jyucsiro · 2018-11-05T02:24:17Z

BALD lib provides the ability to take nc files and obtain an RDF graph in BALD syntax/formalisms.

This provides a pathway to a schema.org profile. It would be good to have an export bald to schema.org function.

jyucsiro · 2018-11-05T02:26:02Z

I've started pulling a mapping table between bald and schema.org in the wiki: https://github.com/binary-array-ld/bald/wiki/Schema.org-mappings

Schema.org only has 2 required fields: title and name. However, there are a range of other 'recommended' properties.

From a first pass, it seems a lot of the predicates may be able to be reused from ACDD, one or two from CF. See https://github.com/binary-array-ld/bald/wiki/Schema.org-mappings

adamml · 2018-11-05T06:48:56Z

Thanks Jonathan. I'm at the International Marine Data and Information Systems conference this week, but will try and fill some of this out in any down time. That also means I'm unlikely to be on the call later.

…

On Mon 5 Nov 2018, 03:26 Jonathan Yu ***@***.*** wrote: I've started pulling a mapping table between bald and schema.org in the wiki: https://github.com/binary-array-ld/bald/wiki/Schema.org-mappings Schema.org only has 2 required fields: title and name. However, there are a range of other 'recommended' properties. From a first pass, it seems a lot of the predicates may be able to be reused from ACDD, one or two from CF. See https://github.com/binary-array-ld/bald/wiki/Schema.org-mappings — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#80 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADNjW1zvhWWFZlbyUSl0CeeItipmW2S_ks5ur6G6gaJpZM4YNqq4> .

jyucsiro · 2018-11-19T05:21:12Z

@adamml @marqh - i've had a go at writing some code to output schema.org JSON-LD (first cut) as a extension to nc2rdf.py.

This uses the mapping file bald2schemaorg_mappings.json, which implements a subset of https://github.com/binary-array-ld/bald/wiki/Schema.org-mappings

A user could run this to get a schema.org description from netCDF file or CDL:
$ python nc2rdf.py -o json-ld --schema-org ../lib/bald/tests/integration/CDL/trajectoryProfile_template.cdl

See #82

Table this to discuss at the next telecon...

jyucsiro · 2018-11-19T10:51:30Z

The output of that example:

{
    "@context": "http://schema.org/",
    "description": "This is an example of the Oceanographic and surface meteorological data collected from the Underwater Slocum Glider RU07 by the National Centers for Environmental Information (NCEI) in the Cordell Bank National Marine Sanctuary from 2015-03-25 to 2015-03-25. The data contained within this file are completely bogus and are generated using the python module numpy.random.rand() function. This file can be used for testing with various applications. The uuid was generated using the uuid python module, invoking the command uuid.uuid4().",
    "http://schema.org/identifier": "NCEI_trajectoryProfile_template_v2.0_2016-09-22_181838.014029.nc",
    "http://schema.org/license": "Freely available",
    "id": "_:N9146fcda1487406b9b977fb4d6641a0f",
    "keywords": "Oceans > Ocean Temperature > Water Temperature, Oceans > Salinity/Density > Salinity",
    "name": "Oceanographic and surface meteorological data collected from the Underwater Slocum Glider RU07 by the National Centers for Environmental Information (NCEI) in the Cordell Bank National Marine Sanctuary from 2015-03-25 to 2015-03-25",
    "type": "Dataset"
}

adamml · 2018-11-20T08:57:24Z

I left a comment on #82...

jyucsiro · 2018-11-20T23:35:08Z

revised example - now with split up keywords and added 'url' if coming from http/https url...

{   
    "@context": "http://schema.org/",
    "description": "The Extended AVHRR Polar Pathfinder (APP-x) version-2 Thematic Climate Data Record (CDR) includes surface temperature, surface albedo, surface and the Top Of the Atmosphere (TOA) shortwave and longwave radiative fluxes, cloud properties (amount, phase, particle size, optical depth,top pressure and temperature, surface and TOA radiative effect), and ice thickness and age. The APP-x CDR has twice daily data at local solar times of 14 and 04(02) for the Arctic(Antarctic) at a spatial resolution of 25 km for both poles.",
    "http://schema.org/identifier": "Polar-APP-X_v01r01_Nhem_1400_d20160801_c20160803.nc",
    "http://schema.org/license": "No restrictions on access or use",
    "id": "_:Na31c8fae6e6e48b986e7d063cd474a28",
    "keywords": [
        "EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > RADIATIVE FLUX",
        "EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SNOW and ICE > ALBEDO",
        "EARTH SCIENCE > CRYOSPHERE > SEA ICE > ICE DEPTH(THICKNESS)",
        "EARTH SCIENCE > CRYOSPHERE > SNOW and ICE > SNOW and ICE TEMPERATURE",
        "EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SNOW and ICE > ICE DEPTH(THICKNESS)",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD TOP PRESSURE",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD TYPE",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD MICROPHYSICS > CLOUD DROPLET PHASE",
        "EARTH SCIENCE > CLIMATE INDICATORS > CRYOSPHERIC INDICATORS > ICE DEPTH(THICKNESS)",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD TOP TEMPERATURE",
        "EARTH SCIENCE > LAND SURFACE > LAND ALBEDO",
        "EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > LONGWAVE RADIATION",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD MICROPHYSICS > CLOUD DROPLET CONCENTRATION(SIZE)",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD RADIATIVE TRANSFER > CLOUD RADIATIVE FORCING",
        "EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > SOLAR RADIATION",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD FRACTION",
        "EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD MICROPHYSICS > CLOUD OPTICAL DEPTH(THICKNESS)",
        "EARTH SCIENCE > CRYOSPHERE > SNOW and ICE > ALBEDO",
        "EARTH SCIENCE > LAND SURFACE > LAND TEMPERATURE",
        "EARTH SCIENCE > OCEANS > SEA ICE > ICE DEPTH(THICKNESS)"
    ],
    "name": "Extended AVHRR Polar Pathfinder Fundamental Climate Data Record (APPx CDR)",
    "type": "Dataset",
    "url": "https://www.ngdc.noaa.gov/thredds/dodsC/arctic/Polar-APP-X_v01r01_Nhem_1400_d20160801_c20160803.nc"
}

jyucsiro · 2018-11-21T06:19:00Z

Been experimenting with landing pages generated from these schema.org descriptions from a thredds harvest process.

using nc files listed in any thredds/opendap catalog... e.g.
--> https://www.ngdc.noaa.gov/thredds/catalog/arctic/catalog.xml
--> schema.org json
--> bald-server (see below)

See the deployed flask app: http://waterinformatics-ext1-cdc.it.csiro.au/bald-server/

So in a view page, the schema.org is rendered, but also the json-ld is embedded in the . This provides a pathway for thredds/opendap hosted nc files to have landing pages for discovery by say, Google Dataset search...

prototype bald-server here: https://github.com/jyucsiro/bald-server

adamml · 2018-11-21T10:47:06Z

Very nice.

The Sructured Data Testing Tool from Google happily parses the JSON-LD...

Attribute	Value
@type	Dataset
description	The Extended AVHRR Polar Pathfinder (APP-x) version-2 Thematic Climate Data Record (CDR) includes surface temperature, surface albedo, surface and the Top Of the Atmosphere (TOA) shortwave and longwave radiative fluxes, cloud properties (amount, phase, particle size, optical depth,top pressure and temperature, surface and TOA radiative effect), and ice thickness and age. The APP-x CDR has twice daily data at local solar times of 14 and 04(02) for the Arctic(Antarctic) at a spatial resolution of 25 km for both poles.
license	No restrictions on access or use
keywords	EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SNOW and ICE > ALBEDO
keywords	EARTH SCIENCE > LAND SURFACE > LAND TEMPERATURE
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD MICROPHYSICS > CLOUD DROPLET PHASE
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD TOP PRESSURE
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD FRACTION
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD MICROPHYSICS > CLOUD OPTICAL DEPTH(THICKNESS)
keywords	EARTH SCIENCE > OCEANS > SEA ICE > ICE DEPTH(THICKNESS)
keywords	EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > SOLAR RADIATION
keywords	EARTH SCIENCE > TERRESTRIAL HYDROSPHERE > SNOW and ICE > ICE DEPTH(THICKNESS)
keywords	EARTH SCIENCE > CRYOSPHERE > SNOW and ICE > SNOW and ICE TEMPERATURE
keywords	EARTH SCIENCE > CRYOSPHERE > SEA ICE > ICE DEPTH(THICKNESS)
keywords	EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > LONGWAVE RADIATION
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD TYPE
keywords	EARTH SCIENCE > CLIMATE INDICATORS > CRYOSPHERIC INDICATORS > ICE DEPTH(THICKNESS)
keywords	EARTH SCIENCE > ATMOSPHERE > ATMOSPHERIC RADIATION > RADIATIVE FLUX
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD PROPERTIES > CLOUD TOP TEMPERATURE
keywords	EARTH SCIENCE > CRYOSPHERE > SNOW and ICE > ALBEDO
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD MICROPHYSICS > CLOUD DROPLET CONCENTRATION(SIZE)
keywords	EARTH SCIENCE > LAND SURFACE > LAND ALBEDO
keywords	EARTH SCIENCE > ATMOSPHERE > CLOUDS > CLOUD RADIATIVE TRANSFER > CLOUD RADIATIVE FORCING
name	Extended AVHRR Polar Pathfinder Fundamental Climate Data Record (APPx CDR)
url	https://www.ngdc.noaa.gov/thredds/dodsC/arctic/Polar-APP-X_v01r01_Nhem_1400_d20160801_c20160803.nc

ddlenz · 2021-05-04T23:11:09Z

Hi, I'm wondering if this feature might still be pending? I get the following using the attached file:

python nc2rdf.py /Users/dlenz/Downloads/foo.nc --schema-org
Traceback (most recent call last):
  File "/Users/dlenz/git/bald/nc2rdf/nc2rdf.py", line 153, in <module>
    nc2schemaorg(args.ncfile, args.format, baseuri=args.baseuri)
  File "/Users/dlenz/git/bald/nc2rdf/nc2rdf.py", line 100, in nc2schemaorg
    schema_g = baldgraph2schemaorg(graph, path=ncfilename, baseuri=baseuri)
  File "/Users/dlenz/git/bald/nc2rdf/nc2rdf.py", line 93, in baldgraph2schemaorg
    schema_org_inst  =  bald.schemaOrg()
TypeError: __init__() missing 1 required positional argument: 'graph'

foo.nc.zip

jyucsiro assigned jyucsiro and adamml Nov 5, 2018

jyucsiro mentioned this issue Nov 19, 2018

Test schema org #82

Merged

jyucsiro mentioned this issue Feb 4, 2019

Initial code for serving bald triples binary-array-ld/bald-server#1

Open

sjskhalsa mentioned this issue Feb 6, 2019

STAC for generic netCDF-type data? radiantearth/stac-spec#366

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

export bald as schema.org #80

export bald as schema.org #80

jyucsiro commented Nov 5, 2018

jyucsiro commented Nov 5, 2018

adamml commented Nov 5, 2018 via email

jyucsiro commented Nov 19, 2018 •

edited

Loading

jyucsiro commented Nov 19, 2018 •

edited

Loading

adamml commented Nov 20, 2018

jyucsiro commented Nov 20, 2018

jyucsiro commented Nov 21, 2018

adamml commented Nov 21, 2018

ddlenz commented May 4, 2021

export bald as schema.org #80

export bald as schema.org #80

Comments

jyucsiro commented Nov 5, 2018

jyucsiro commented Nov 5, 2018

adamml commented Nov 5, 2018 via email

jyucsiro commented Nov 19, 2018 • edited Loading

jyucsiro commented Nov 19, 2018 • edited Loading

adamml commented Nov 20, 2018

jyucsiro commented Nov 20, 2018

jyucsiro commented Nov 21, 2018

adamml commented Nov 21, 2018

ddlenz commented May 4, 2021

jyucsiro commented Nov 19, 2018 •

edited

Loading

jyucsiro commented Nov 19, 2018 •

edited

Loading