Skip to content

Implementation Notes

Jump to bottom
Jim Amsden edited this page Aug 22, 2018 · 7 revisions

This is the implementation of the ldp-service storage services (storage.js) using Fuseki and TDB.

These notes will focus on the HTTP POST, GET, PUT and DELETE methods needed to create LDP-RS, and LDPC resources, and get, add and remove members from LDPC basic and direct container interaction models.

These methods are all implemented using the ldp-service/storage.js storage abstraction. ldp-service-jena is the storage.js concrete implementation on Fuseki, using SPARQL update.

ldp-service and ldp-service-jena are implemented together in order to determine the design, use and a reference implementation of the storage.js abstraction.

GET: function get(req, res, includeBody)

includeBody is a boolean to determine if the body should be returned - resource.head calls get(req, res, false), resource.get calls get(req, res, true) to share the common code. The problem is that head still causes the resource to be read and deserialized, even though it isn't sent. This is inefficient, but may be necessary in order calculate the common headers and set the proper link headers for LDP containment.

db.read

GET delegates to the db.read(req.fullURL, function(err, document)). This does a GET on the SPARQL update data endpoint, treating the resource URI as a SPARQL graph.

db.read also determines the interaction model from the triples for the resource: null for an LDP-RS resource, basicContainer, or directContainer. This information, along with the URI for the resource are stored as additional members of the document (an rdflib.js IndextedFormula) which also contains the parsed RDF triples. This is done in db.read because how the containment is implemented is determined by the storage implementation and not in the LDP service.

Error handling

Examines err to set status codes appropriately, does res.sendStatus() and returns if it can't continue.

examines the request accept header to determine what serializer to use. Note the db may use its own request and accept header to interact with the underlying database, but that is database implementation specific and should not be exposed at this level

adds common headers (GET, POST, OPTIONS):

  • sets Allow header to GET, HEAD, DELETE, OPTIONS
  • if it not a container, adds PUT to the allow header
  • if the document is a container, 
  • sets response links type to the document interaction model
  • adds POST to the allow header
  • sets Accept-Post allowed media types (turtle, jsonld, json)

Inserts some calculated triples that aren't stored in the document: insertCalculatedTriples()

Based on preferences, determines how members of a container should be handled. This information is not stored in the DirectContainer resource, a query is used on the membershipResource and hasMemberRelation properties to get the members.

Handles the LDP container membership predicates for ldp:contains, or the LDP membership predicate.

There are two sets of triples added: 

  1. the membership triples defined for the LDPC potentially customized by the Prefer header
  2. the containment and/or membership information about the LDPC, depending on the Prefer header

LDP Best Practices and guidelines

What should be stored depends on what users are more likely to GET. This is probably the domain specific vocabulary, not LDP.

Determine if the resource is a BasicContainer or DirectContainer using the document.interactionModel set when the resource was read.

Determine what the client wants returned using the Prefer header. The Client provides a hint to help the server form an appropriate response from potentially large containers. The Server uses the Preferences-Applied header to indicate what it did.

Representation can be include or omit, and currently the URLs provided only apply to LDPCs

Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMinimalContainer"

Prefer header options

  • no prefer header: include the container properties and its containment and/or membership triples
  • prefer ldp:PreferMinimalContainer: include just the container properties
  • prefer ldp:PreferContainment - the ldp:contains members of a BasicContainer
  • prefer ldp:PreferMembership - the calculated ldp:member triples for a DirectContainer

Use db.getMembershipTriples to get the membership triples for a DirectContainer if needed based on the prefer header.

Remove the containment triples from a BasicContainer if ldp:PreferMinialContainer is specified.

db.getMembershipTriples(container, callback(err, members))

The Fuseki implementation stores the members in the domain specific properties for a DirectConotainer. This storage.js function uses a SPARQL query to calculate the LDPC member resources.

Finish up and send the serialized result

serialize the document (a reflib.js IndexedFormula) based on the content type. Uses rdflib to handle the serialization.

handle Preference-Applied header

generates an eTag and writes the ETag and Content-Type headers.

if includeBody is true, does res.end(newBuffer(content), 'utf-8') to send the response body. otherwise just does res.end.

Examples:

GET an LDP-RS: the University of Maine University

GET <http://http://localhost:3000/univ/umaine>

@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <./>.
@prefix cou: <umaine/courses/>.
@prefix stu: <umaine/students/>.
@prefix te: <umaine/teachers/>.
uni:umaine
    a univ:University;
    univ:courses cou:CS101, cou:EN100, cou:ME201;
    univ:description "A wonderful place to learn";
    univ:name "University of Maine";
    univ:students stu:727175, stu:727188;
    univ:teachers te:P154567.

GET an LDPC: the University of Maine student list

For a DirectContainer, the member can be returned using {DirectContainer ldp:contains } and/or using {membershipResource hasMemberRelation }. The Prefer header can be used to to allow the client to specify what they want.

Prefer header processing:

  • PreferMinimal: only return the triples for the container, not any of its membrers
  • PreferContainment: basic or direct container ldp:contains membrers
  • PreferMembership: for a DirectContainer, provide its calculated members from its membershipResource and hasMemberRelation

If no prefer header is specified, then the result includes containment and membership triples along with the container properties. For a direct container, the containment and membership triples will have the same members, but expressed using different assertions.

Get information about the /umaine/students with no prefer header, container properties and membership triples

 curl --request GET
 --url http://localhost:3000/univ/umaine/students
 --header 'Accept: text/turtle'

Here's an example of a SPARQL query that gets the members of a DirectContainer:

prefix rdf:   <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix univ:  <http://university.org/ns/edu#>
prefix ldp:  <http://www.w3.org/ns/ldp#>
SELECT ?member
WHERE {GRAPH ?membershipResource {
  {SELECT ?membershipResource ?hasMemberRelation
  WHERE {graph **<http://localhost:3000/univ/umaine/students>** {
    <http://localhost:3000/univ/umaine/students>
    ldp:membershipResource ?membershipResource ;
    ldp:hasMemberRelation ?hasMemberRelation .}
    } 
  }
  ?membershipResource ?hasMemberRelation ?member .}
}
  • implement isContainer(req.fullURL, document) by checking if the resource is a BasicContainer or DirectContainer. Use an ASK SPARQL query.

use statementsMatching not any() it doesn't seem to match ?s ?p ?o.

document.interactionModel needs to be set.

  • implement storage.getContainment(document.name, function(err, containment) to return the containment members

This method gets all the members of the basic or direct container. 

Then insertCalculatedTriples determines wether it should include containment and/or membership triples, and puts them in the document.

For our implementation, a BasicContainer will already have its containment triples because they're stored with the BasicContainer graph. (the MongoDB implementation had the resource point to its container).

  • so if includeContainment is false, the {document.url ldp:contains } triples would need to be removed from a BasicContainer

Then the implementation of getContainment() would only be applicable to DirectContainer and could directly add the triples to the document.

For a DirectContainer, we need to know the membershipResource and hasMembrerRelation, handled in isContainer (but this method has side effects and should probably be renamed).

@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <../>.
@prefix stu: <students/>.
@prefix um: <./>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
    a ldp:DirectContainer;
    ldp:contains stu:727175, stu:727188;
    ldp:hasMemberRelation univ:students;
    ldp:membershipResource uni:umaine.
  • Has the right content, but the URLs seem wrong. TBL says these are correct and are relative to the URL of the HTTP resource.

https://gitter.im/linkeddata/rdflib.js?at=5abe80bf2b9dfdbc3a3c8471 

TBL says this is correct, that the serialized resource is relative to the URL in the GET request and no @base would be needed.

And if you want to run some back-end processing with them all in file:// space into the appropriate directories then you can do that too, so log as the links are relative

The web architecture is that the URI of the resource and the content type are both available to make sense of the content.

Suppose you add it and it was different from the Location: header?

Suppose it were different from the URI the user originally sent? Which would take precedence?

HTML files and CSS files use relative URIs without having the absolute URI embedded in them.

Another frequent handy aspect if you might have have an internal test URI which then is picked up by an HTTP firewall proxy to make the same data appear in different spaces ...

This may be correct, the URIs are relative to the request URL, or Location header. But is it no incorrect to use the URLs asserted in the triple store. You can accomplish this by including a base parameter to serialize that would never be used, say "none:".

Then you get:

@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
    a ldp:DirectContainer;
    ldp:contains stu:727175, stu:727188;
    ldp:hasMemberRelation univ:students;
    ldp:membershipResource uni:umaine.

Get information about the /umaine/students container only

curl --request GET  --url http://localhost:3000/univ/umaine/students  --header 'Accept: text/turtle'  --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMinimalContainer"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix uni: <http://localhost:3000/univ/>.

um:students
    a ldp:DirectContainer;
    ldp:hasMemberRelation univ:students;
    ldp:membershipResource uni:umaine.

Get the members of the the /umaine/students container, preferring the containment triples

curl --request GET  --url http://localhost:3000/univ/umaine/students  --header 'Accept: text/turtle'  --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferContainment"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.

uni:umaine univ:students stu:727175, stu:727188 .

um:students
    a ldp:DirectContainer;
    ldp:contains stu:727175, stu:727188;
    ldp:hasMemberRelation univ:students;
    ldp:membershipResource uni:umaine.

Get the members of the the /umaine/students container, preferring the membership triples

curl --request GET  --url http://localhost:3000/univ/umaine/students  --header 'Accept: text/turtle'  --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMembership"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.

uni:umaine univ:students stu:727175, stu:727188 .

um:students
    a ldp:DirectContainer;
    ldp:contains stu:727175, stu:727188;
    ldp:hasMemberRelation univ:students;
    ldp:membershipResource uni:umaine.

Get the members of the the /umaine/students container, preferring the membership triples, omitting the containment triples

curl --request GET  --url http://localhost:3000/univ/umaine/students  --header 'Accept: text/turtle'  --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMembership"; omit="  http://www.w3.org/ns/ldp#PreferContainment"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.

uni:umaine univ:students stu:727175, stu:727188 .

um:students
    a ldp:DirectContainer;
    ldp:hasMemberRelation univ:students;
    ldp:membershipResource uni:umaine.

HEAD

Works without any change. Does a GET, but doesn't return the body.

OPTIONS

Works without any change. Simply does a db.read and uses the content to set the headers, same as GET.

DELETE

Clone this wiki locally