JSON-LD 1.1 Feature Request: support @values for describing multidimensional containers (list of lists)

[pietercolpaert]

Problem description

Many JSON specs existed before JSON-LD. A couple of these specs may not be compatible with JSON-LD as they contain multidimensional containers, such as GeoJSON.

Example of a multidimensional array:

[ [3.1,51.06,30],
  [3.1,51.06,20] ]

This issue is a result from the discussion on the GeoJSON-LD repository: geojson/geojson-ld#32. If this issue will not get resolved, the GeoJSON-LD community would suggest creating custom JSON-LD parsers for JSON-LD dialects. This situation would be far from desirable.

Suggested solution

Introduce a new @values keyword, which can be used to describe the values of a @set or a @list container in more detail.

When an array is given in the @values, then the precise amount of objects within this array corresponds with the array in the graph in this order.

When an object is given in the @values, each value of the array in the graph is mapped according to this template.

Example

{
  "@context": {
     "coordinates": {
        "@id": "geojson:coordinates",
        "@container" : "@list",
        "@values" : { 
           "@type" : "geojson:Coordinate",
           "@container" : "@set",
           "@values" : [
               {"@type" : "xsd:double", "@id":"geo:longitude"},
               {"@type" : "xsd:double", "@id":"geo:latitude"}
           ]
        }
     }
  },
  "@graph" : [{
   "@id" : "ex:LineString1",
    "coordinates" : [
          [
            3.1057405471801753,
            51.064216229943476
          ],
          [
            3.1056976318359375,
            51.063434090307574
          ]
    ]
  }]
}

Would transform to (and vice versa):

ex:LineString1 geojson:coordinates _:b0 .
_:b0 rdf:first _:b1 .
_:b1 a geojson:Coordinate ;
      geo:longitude "3.105740547180175E0"^^xsd:double ;
      geo:latitude "5.106421622994348E1"^^xsd:double .
_:b0 rdf:rest _:b2 .
_:b2 rdf:first a geojson:Coordinate ;
      geo:longitude "3.1056976318359375"^^xsd:double ;
      geo:latitude "51.063434090307574"^^xsd:double .
_:b2 rdf:rest rdf:nil .

[gkellogg]

As JSON-LD is a released Recommendation, the only way to resolve such an issue is through an update to the specification. This requires chartering a new Working Group with the goal to update the JSON-LD specs (along with possibly other related goals). Such working group's aren't chartered easily.

There are a number of other similar features (new requirements) that we're also tracking in this repository, along with a goal of sometime publishing JSON-LD Framing.

One issue that this request, and some others faces, is the round-trip nature of JSON-LD Compaction/Expansion. Expanding the example you have above would result in something like the following:

[{
  "@id": "http://example.org/LineString1",
  "http://geojson.org/vocab#coordinates": [{
    "@list": [{
      "@type": ["http://geojson.org/vocab#Coordinate"],
      "http://geojson.org/vocab#longitude": 
        [{"@value": "3.105740547180175E0", "@type": "http://www.w3.org/2001/XMLSchema#double"}],
      "http://geojson.org/vocab#latitude": 
        [{"@value": "5.106421622994348E1", "@type": "http://www.w3.org/2001/XMLSchema#double"}]
    }, {
      "@type": ["http://geojson.org/vocab#Coordinate"],
      "http://geojson.org/vocab#longitude": 
        [{"@value": "3.1056976318359375", "@type": "http://www.w3.org/2001/XMLSchema#double"}],
      "http://geojson.org/vocab#latitude": 
        [{"@value": "51.063434090307574", "@type": "http://www.w3.org/2001/XMLSchema#double"}]
    }]
  }]
}]

When compacting with this context, we'd expect to get back the original. As you can see, such features can make this quite difficult, but of course, not impossible. We could consider relaxing this requirement for certain features in the future.

For now, the best way forward is to do as you suggest and get existing parsers to have a processing mode to support this (as well as other features).

[jasnell]

With everything @gkellogg just said in mind... another possibility would be to introduce a new @container=@json mechanism in the context. In the RDF, this would essentially map to a JSON string.

{
  "@context": {
    "foo": "http://example.org/",
    "coordinates": {
      "@id": "foo:coordinates",
      "@container": "@json"
    }
  },
  "@id": "urn:test",
  "coordinates": [[1,2],[3,4]]
}

<urn:test> foo:coordinates "[[1,2],[3,4]]"^^http://www.w3.org/ns/json-ld#json .

This makes the data in the RDF form less accessible but allows for a much easier round trip to and from the JSON.

[dinizime]

@pietercolpaert So the idea is transform the list of list into a list of objects? Since the transformation that you exemplified is isomorphic to:

{
  "@context": {
     "coordinates": {
        "@id": "geojson:coordinates",
        "@container" : "@list"
     },
    "coordinate": "geojson:Coordinate",
    "longitude": "geo:longitude",
    "latitude": "geo:latitude"
  },
  "@graph" : [{
   "@id" : "ex:LineString1",
    "coordinates" : [
          {
            "@type": "coordinate",
            "longitude": 3.1057405471801753,
            "latitude": 51.064216229943476
          },
          {
            "@type": "coordinate",
            "longitude": 3.1056976318359375,
            "latitude": 51.063434090307574
          }
    ]
  }]
}

In this way, you cannot apply the same idea for [[x11,x12,...],[x21,x22,....],...], since the context would simply be:

"@context": {
     "collection": {
        "@id": "ex:list",
        "@container" : "@list",
        "@values": {
            "@type" : "ex:innerList",
            "@container" : "@set",
            "@values" : {"@type" : "ex:elementType", "@id":"ex:element"}
        }      
     }
}

Which cannot be expanded in the same way you exemplified. Your design only works for a list of lists, where the number of elements in the inner list is fixed and know in advance. No even sets can work since you cannot map correctly which is the latitude and which is the longitude if the order doesn't matters. (so i would guess that you meant the inner container to be @list as well).

In GeoJSON-LD this means that in the proposed way you cannot represent Polygons (with holes), MultiLineString, and MultiPolygons.

[lanthaler]

Is it a strict requirement that GeoJSON-LD looks exactly the same as GeoJSON (modulo the @context property perhaps)? If so, why? What speaks against a standardized (and almost trivial) pre-processor for converting application/vnd.geo+json to application/ld+json?

[sgillies]

@lanthaler it is for me, yes. I want to use JSON-LD to improve the quality of existing GeoJSON data without giving up benefits of ordinary GeoJSON like maps on GitHub.

[pietercolpaert]

Summarizing the above

We now have 3 suggested solutions:

1. Implement @json to say that the JSON object should be stringified and put into 1 RDF object
2. Implement JSON-LD for any kind of hierarchical array as well using a new spec
3. Do nothing at all, but create a custom parser in the GeoJSON-LD repository which is able to expand the multidimensional arrays in GeoJSON to something GeoJSON-LD can parse.

Personal replies

@jasnell Adding a "this is a json string" will need custom parsers to get sense out of the data that is in the JSON string. I'm afraid that this will lead to JSON-LD dialects and forks of the parser

@dinizime Indeed: my suggested solution is not usable. Could we however find a solution that is? When taking @gkellogg's comment into account, I believe that indeed, just extending JSON-LD to any kind of JSON object might make the spec too complex.

@sgillies I think @lanthaler meant that GeoJSON-LD could extend JSON-LD with its own @something, and specify how this can be converted to standard JSON-LD. It would still allow for geojson in github, but it wouldn't be valid JSON-LD in the first place. We basically would have to create a new mime type... Something like application/vnd.geo.ld+json

Personal preference

From a purist perspective, option 2 would be perfect. Yet it will take long to find the nicest solution which will satisfy everyone.

From a pragmatic point of view, I think going for option 3 is the best option without doing any harm: option 2 can still be implemented in the long run. In meantime, @sgillies has already done a proposal to fix it this way: geojson/geojson-ld#37.

[kevinSuttle]

Yeah, I have been wondering about this as well. The current implementation is quite cumbersome. Funny how I just made up an implementation and we arrived at the same solution.

schemaorg/schemaorg#938

[gkellogg]

Quite complicated use case that makes round-tripping impossible, and many questions for multi-dimensional data. Perhaps better preserved using a JSON Literal, and separate rules run in the RDF space.

[gkellogg]

See http://geojson.org/geojson-ld/ for a motivation for use in GeoJSON.

[gkellogg]

See thread on [email protected] for a discussion starting with geojson needs.

[gkellogg]

Deferred to WG due to https://json-ld.org/minutes/2018-04-10/#resolution-3.

[gkellogg]

Closed in favor of w3c/json-ld-syntax#7.

[mitar]

For anyone following this, current plan is to close this as "won't fix", so please give any feedback about this here: w3c/json-ld-syntax#7