Discussion about PR 124

[wiresio]

Since #124 came as a surprise and we didn't want to block the work related to it, let's have a discussion about it in this issue.

Related issues are:

• Handle huge set of Thing Descriptions (pagination, streaming, etc.) #16
• Need information model for directory #98

First discussion point from my side:
A TDD is supposed to return

• One TD via GET /td/{id}
• A bunch of TDs matching several criteria via GET /search/jsonpath or GET /search/xpath or GET/POST /search/sparql
• All TDs stored in the TDD via the new GET /tdall

Having queried for one TD would I expect a response with content type application/td+json such that I receive a TD or enriched TD?

For returning all TDs we discussed that it should be content type application/ld+json. Does the same hold for the search queries? Querying for all TDs seems to be a special case of querying for several TDs. We should fix the response format for these interfaces equally.

[farshidtz]

Thank you for your review.

Unlike the listing, the search APIs does NOT always return a bunch of TDs. The APIs allows queries and IMO should have been named accordingly. The supported query languages respond with what is asked and this is mentioned in the draft spec:

JSONPath/XPath:

contain application/json Content-Type header, and in the body a set of complete TDs or a set of TD fragments.

SPARQL:

Content-Type header application/ld+json for CONSTRUCT and DESCRIBE queries or application/json for SELECT or ASK

Example query response from 2020.09 plugfest which is application/json:
GET /sparql/jsonpath?query=$[:].id OR
GET /sparql/xpath?query=*/id

[
    "de:tum:ei:esi:fp:coffee:1-1",
    "de:tum:ei:esi:fp:coffee:2-1",
    "esi:pantilt:2",
    "urn:com:fujitsu:sensor-f4",
    "urn:dev:mac:b827ebfffe4b6d0b",
    "urn:dev:ops:32473-ConveyorBelt-001",
    "urn:dev:ops:32473-ConveyorBelt-002",
    "urn:dev:ops:32473-DobotMagician-001",
    "urn:dev:ops:32473-HueLight-1",
    "urn:dev:ops:32473-HueLight-2",
    "urn:example:1234",
    "urn:uuid:0800f07a-178d-48ec-9ddf-d8a941071298",
    "urn:uuid:086d490d-90c6-4785-a677-c911bd2af919"
]

Having queried for one TD would I expect a response with content type application/td+json such that I receive a TD or enriched TD?

This is not yet specified. But there are editor notes in place. I'd expect something like:
GET /td/{id}?enriched=true -> one TD, enriched
GET /td?enriched=true -> all TDs, enriched
The default will be enriched=false as agreed, because we prefer to return original TDs and enriched data is not needed by most.

The search APIs shall perform query operations on the enriched data. Need to mention that in the spec.

[relu91]

I think the spec as it now is a good start. I am not sure about the return type of the CONSTRUCT or DESCRIBE. Why is not application/td+jsonld?

As I expressed in different comments users of a TDD expect TDs or (worst case) TD fragments. From the text, it seems that the implementation can just convert the result of the CONSTRUCT or DESCRIBE to jsonld. However, it is super-inconvenient to use this kind of result to consume a TD and starts interacting with the remote Web Thing. We have to remember that the most common use case for a TDD is to search for TDs and consume them.

I know that @AndreaCimminoArriaga worked on ways to improve the framing mechanism so that a TD could successfully be built from json-ld. Should we describe this process? I have also some preliminary work on this topic in my PhD Thesis that we could discuss if anyone is interested.

[wiresio]

Quick question about "id": Since it is optional in the TD, do we expect the TDD to generate it (and in case it is already present in the TD, overwrite it)?

[farshidtz]

PR #124 only added listing. I think this issue is turning into general feedback for the directory API.

Quick question about "id": Since it is optional in the TD, do we expect the TDD to generate it (and in case it is already present in the TD, overwrite it)?

Anonymous TDs, submitted with a POST request, get a system generated UUID. See: #exploration-directory-api-registration-creation
If the ID is already present, then it is not anonymous, and should be sent with a PUT request with the id in HTTP path. The spec doesn't say that the id in /td/{id} must be identical to TD.id. But IMO, they should be.

[AndreaCimminoArriaga]

Answering to @relu91 regarding the mime type, the quick and easy answer is because the SPARQL standard does not have as return type application/td+jsonld. The closest mime type is application/jsonld which specifies a JSON-LD but does not details if it is 1.0 or 1.1 (and definitively is not a framed JSON-LD). Additionally, CONSTRUCT queries could return any RDF following any model, therefore, we can not ensure that the answer in RDF will follow the TD model. Notice that this is not applicable for the DESCRIBE queries. In both cases, CONSTRUCT and DESCRIBE, can out of band return a JSON-LD document (JSON-LD is just another RDF serialisation). What CONSTRUCT and DESCRIBE can not return is a framed JSON-LD. As general reminder, the Thing Descriptions are JSON-LD documents framed, they are not regular JSON-LD documents.

As I expressed in different comments users of a TDD expect TDs or (worst case) TD fragments. From the text, it seems that the implementation can just convert the result of the CONSTRUCT or DESCRIBE to jsonld. However, it is super-inconvenient to use this kind of result to consume a TD and starts interacting with the remote Web Thing. We have to remember that the most common use case for a TDD is to search for TDs and consume them.

I agree 100%, for sure SELECT queries are the most suitable ones for discovery, and later, consume the discovered TDs (or fragments). CONSTRUCT and DESCRIBE can return JSON-LD, what they can't is to return the JSON-LD framed of WoT. I think you are also right regarding the fact that there are easiest ways to retrieve a TD, for instance /td/{id} is something (like) equivalent to DESCRIBE {id}. However, most of users will prefer the first choice. Nevertheless, having both mechanisms can be beneficial and allow us to be aligned with the SPARQL protocol, without increasing coding complexity (remember SPARQL implementation is optional).

I know that @AndreaCimminoArriaga worked on ways to improve the framing mechanism so that a TD could successfully be built from json-ld. Should we describe this process? I have also some preliminary work on this topic in my PhD Thesis that we could discuss if anyone is interested.

For me it is fine, I think a frame or a general mechanism for performing the translation from regular JSON-LD to JSON-LD framed must be provided. Otherwise the Thing Descriptions are disconnected from their RDF representations, and thus, semantic technologies like SHACL shapes for validation (which are also a standard) can not be used. Even using triple stores becomes a complex task since once a TD is translated to RDF (regular JSON-LD) it can not be expressed to the original syntax due to the lack of the frame.

As an additional comment to the thread, I would like to highlight that even if the id is not specified in a description JSON-LD specifies that a blank node is created as id. Therefore, descriptions have always an implicit id, and, according to RDF this id should identify this document and allow anyone to retrieve it. As a result, the id that appears in /td/{id} should be the same of the id that the description has, regardless if it is a blank node (auto generated) or an id provided by a user.

[wiresio]

Obsolete