-
Notifications
You must be signed in to change notification settings - Fork 1
Implementation Notes
This is the implementation of the ldp-service storage services (storage.js) using Fuseki and TDB.
These notes will focus on the HTTP POST, GET, PUT and DELETE methods needed to create LDP-RS, and LDPC resources, and get, add and remove members from LDPC basic and direct container interaction models.
These methods are all implemented using the ldp-service/storage.js storage abstraction. ldp-service-jena is the storage.js concrete implementation on Fuseki, using SPARQL update.
ldp-service and ldp-service-jena are implemented together in order to determine the design, use and a reference implementation of the storage.js abstraction.
includeBody is a boolean to determine if the body should be returned - resource.head calls get(req, res, false), resource.get calls get(req, res, true) to share the common code. The problem is that head still causes the resource to be read and deserialized, even though it isn't sent. This is inefficient, but may be necessary in order calculate the common headers and set the proper link headers for LDP containment.
GET delegates to the db.read(req.fullURL, function(err, document)). This does a GET on the SPARQL update data endpoint, treating the resource URI as a SPARQL graph.
db.read also determines the interaction model from the triples for the resource: null for an LDP-RS resource, basicContainer, or directContainer. This information, along with the URI for the resource are stored as additional members of the document (an rdflib.js IndextedFormula) which also contains the parsed RDF triples. This is done in db.read because how the containment is implemented is determined by the storage implementation and not in the LDP service.
Examines err to set status codes appropriately, does res.sendStatus() and returns if it can't continue.
examines the request accept header to determine what serializer to use. Note the db may use its own request and accept header to interact with the underlying database, but that is database implementation specific and should not be exposed at this level
- sets Allow header to GET, HEAD, DELETE, OPTIONS
- if it not a container, adds PUT to the allow header
- if the document is a container,
- sets response links type to the document interaction model
- adds POST to the allow header
- sets Accept-Post allowed media types (turtle, jsonld, json)
Based on preferences, determines how members of a container should be handled. This information is not stored in the DirectContainer resource, a query is used on the membershipResource and hasMemberRelation properties to get the members.
Handles the LDP container membership predicates for ldp:contains, or the LDP membership predicate.
There are two sets of triples added:
- the membership triples defined for the LDPC potentially customized by the Prefer header
- the containment and/or membership information about the LDPC, depending on the Prefer header
LDP Best Practices and guidelines
What should be stored depends on what users are more likely to GET. This is probably the domain specific vocabulary, not LDP.
Determine if the resource is a BasicContainer or DirectContainer using the document.interactionModel set when the resource was read.
Determine what the client wants returned using the Prefer header. The Client provides a hint to help the server form an appropriate response from potentially large containers. The Server uses the Preferences-Applied header to indicate what it did.
Representation can be include or omit, and currently the URLs provided only apply to LDPCs
Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMinimalContainer"
Prefer header options
- no prefer header: include the container properties and its containment and/or membership triples
- prefer ldp:PreferMinimalContainer: include just the container properties
- prefer ldp:PreferContainment - the ldp:contains members of a BasicContainer
- prefer ldp:PreferMembership - the calculated ldp:member triples for a DirectContainer
Use db.getMembershipTriples to get the membership triples for a DirectContainer if needed based on the prefer header.
Remove the containment triples from a BasicContainer if ldp:PreferMinialContainer is specified.
The Fuseki implementation stores the members in the domain specific properties for a DirectConotainer. This storage.js function uses a SPARQL query to calculate the LDPC member resources.
serialize the document (a reflib.js IndexedFormula) based on the content type. Uses rdflib to handle the serialization.
handle Preference-Applied header
generates an eTag and writes the ETag and Content-Type headers.
if includeBody is true, does res.end(newBuffer(content), 'utf-8') to send the response body. otherwise just does res.end.
GET <http://http://localhost:3000/univ/umaine>
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <./>.
@prefix cou: <umaine/courses/>.
@prefix stu: <umaine/students/>.
@prefix te: <umaine/teachers/>.
uni:umaine
a univ:University;
univ:courses cou:CS101, cou:EN100, cou:ME201;
univ:description "A wonderful place to learn";
univ:name "University of Maine";
univ:students stu:727175, stu:727188;
univ:teachers te:P154567.
For a DirectContainer, the member can be returned using {DirectContainer ldp:contains } and/or using {membershipResource hasMemberRelation }. The Prefer header can be used to to allow the client to specify what they want.
Prefer header processing:
- PreferMinimal: only return the triples for the container, not any of its membrers
- PreferContainment: basic or direct container ldp:contains membrers
- PreferMembership: for a DirectContainer, provide its calculated members from its membershipResource and hasMemberRelation
If no prefer header is specified, then the result includes containment and membership triples along with the container properties. For a direct container, the containment and membership triples will have the same members, but expressed using different assertions.
Get information about the /umaine/students with no prefer header, container properties and membership triples
curl --request GET
--url http://localhost:3000/univ/umaine/students
--header 'Accept: text/turtle'
Here's an example of a SPARQL query that gets the members of a DirectContainer:
prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
prefix univ: <http://university.org/ns/edu#>
prefix ldp: <http://www.w3.org/ns/ldp#>
SELECT ?member
WHERE {GRAPH ?membershipResource {
{SELECT ?membershipResource ?hasMemberRelation
WHERE {graph **<http://localhost:3000/univ/umaine/students>** {
<http://localhost:3000/univ/umaine/students>
ldp:membershipResource ?membershipResource ;
ldp:hasMemberRelation ?hasMemberRelation .}
}
}
?membershipResource ?hasMemberRelation ?member .}
}
- implement isContainer(req.fullURL, document) by checking if the resource is a BasicContainer or DirectContainer. Use an ASK SPARQL query.
use statementsMatching not any() it doesn't seem to match ?s ?p ?o.
document.interactionModel needs to be set.
- implement storage.getContainment(document.name, function(err, containment) to return the containment members
This method gets all the members of the basic or direct container.
Then insertCalculatedTriples determines wether it should include containment and/or membership triples, and puts them in the document.
For our implementation, a BasicContainer will already have its containment triples because they're stored with the BasicContainer graph. (the MongoDB implementation had the resource point to its container).
- so if includeContainment is false, the {document.url ldp:contains } triples would need to be removed from a BasicContainer
Then the implementation of getContainment() would only be applicable to DirectContainer and could directly add the triples to the document.
For a DirectContainer, we need to know the membershipResource and hasMembrerRelation, handled in isContainer (but this method has side effects and should probably be renamed).
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <../>.
@prefix stu: <students/>.
@prefix um: <./>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
a ldp:DirectContainer;
ldp:contains stu:727175, stu:727188;
ldp:hasMemberRelation univ:students;
ldp:membershipResource uni:umaine.
- Has the right content, but the URLs seem wrong. TBL says these are correct and are relative to the URL of the HTTP resource.
https://gitter.im/linkeddata/rdflib.js?at=5abe80bf2b9dfdbc3a3c8471
TBL says this is correct, that the serialized resource is relative to the URL in the GET request and no @base would be needed.
And if you want to run some back-end processing with them all in file:// space into the appropriate directories then you can do that too, so log as the links are relative
The web architecture is that the URI of the resource and the content type are both available to make sense of the content.
Suppose you add it and it was different from the Location: header?
Suppose it were different from the URI the user originally sent? Which would take precedence?
HTML files and CSS files use relative URIs without having the absolute URI embedded in them.
Another frequent handy aspect if you might have have an internal test URI which then is picked up by an HTTP firewall proxy to make the same data appear in different spaces ...
This may be correct, the URIs are relative to the request URL, or Location header. But is it no incorrect to use the URLs asserted in the triple store. You can accomplish this by including a base parameter to serialize that would never be used, say "none:".
Then you get:
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
a ldp:DirectContainer;
ldp:contains stu:727175, stu:727188;
ldp:hasMemberRelation univ:students;
ldp:membershipResource uni:umaine.
curl --request GET --url http://localhost:3000/univ/umaine/students --header 'Accept: text/turtle' --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMinimalContainer"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
@prefix uni: <http://localhost:3000/univ/>.
um:students
a ldp:DirectContainer;
ldp:hasMemberRelation univ:students;
ldp:membershipResource uni:umaine.
curl --request GET --url http://localhost:3000/univ/umaine/students --header 'Accept: text/turtle' --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferContainment"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
a ldp:DirectContainer;
ldp:contains stu:727175, stu:727188;
ldp:hasMemberRelation univ:students;
ldp:membershipResource uni:umaine.
curl --request GET --url http://localhost:3000/univ/umaine/students --header 'Accept: text/turtle' --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMembership"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
a ldp:DirectContainer;
ldp:contains stu:727175, stu:727188;
ldp:hasMemberRelation univ:students;
ldp:membershipResource uni:umaine.
Get the members of the the /umaine/students container, preferring the membership triples, omitting the containment triples
curl --request GET --url http://localhost:3000/univ/umaine/students --header 'Accept: text/turtle' --header 'Prefer: return=representation; include="http://www.w3.org/ns/ldp#PreferMembership"; omit=" http://www.w3.org/ns/ldp#PreferContainment"'
@prefix : <#>.
@prefix univ: <http://university.org/ns/edu#>.
@prefix uni: <http://localhost:3000/univ/>.
@prefix stu: <http://localhost:3000/univ/umaine/students/>.
@prefix um: <http://localhost:3000/univ/umaine/>.
@prefix ldp: <http://www.w3.org/ns/ldp#>.
uni:umaine univ:students stu:727175, stu:727188 .
um:students
a ldp:DirectContainer;
ldp:hasMemberRelation univ:students;
ldp:membershipResource uni:umaine.
Works without any change. Does a GET, but doesn't return the body.
Works without any change. Simply does a db.read and uses the content to set the headers, same as GET.
does a DELETE of the graph:
DELETE http://localhost:3030/univ/data?graph=http://localhost:3030/univ/umaine
parse the entity request body read the resource and check its etag uses putUpdate to update an existing resource uses putCreate to create a new resource.
- are both of these needed? putCreate creates the membershipResource for a DirectContainer if it doesn’t already exist. putUpdate is either not allowed, or it removes membership triples before updating a DirectContainer. Yes they are both needed.
default to ldp.RDFSource search for document.uri rdf:type ldp:BasicContainer or ldp:DirectContainer and set the document.interactionModel accordingly if its a direct container, set document.membershipResourcer and document.hasMemberRelation or isMemberOfRelation Don’t override the interaction model if its already set.
updates the interaction model for the document based on the entity request body adjusts the interaction model based on Link headers
- default interaction model should be ldp.RDFSource, not null? maybe only if the Link header says its and LDP-RS checks for valid membership triple pattern, if DirectContainer, must have membershipResource and one of hasMemberRelation or isMemberOfRelation calls db.update to update the document. creates the membership resource if the entity request body is a DirectContainer if necessary so POST methods will work
I simplified putCreate to not create the membershipResource. It might often be already created. Doing it as a side-effect of PUT to create a DirectContainer would not provide a reasonable entity request body to initialize it. So there's little point in creating it here.
Sets the allow header based on the interaction model and sends 405, putUpdate not allowed on a Container.
- this would mean you can’t update the membershipResource or hasMemberRelation properties on a DirectContainer. adds the containment triples for containers, then serializes the result in order to calculate the ETag. Rather use POST to effect the containment of an LDPC.
- why is this being done? wasn’t an error returned to disallow PUT on a container? doesn’t appear to be needed, perhaps this represents incremental code where the error check was added after PUT on an LDPC was already implemented. Not supporting PUT to update an LDPC is after all a should, not must.
- Does LDP spec allow PUT on a Container? No. Section 5.2.4 HTTP PUT says LDP servers should not allot HTTP PUT to update an LDPC’s containment triples. It should respond with 409 (conflict)
calls db.findContainer. - this can be db.read since it already sets the interactionModel returns an error if its not a container, LDP only allows POST on containers determines the Content-Type calls assignURI using the POST URL and the Slug header to assign the URL for the new resource parses the entity request body
- Handling of err vs. status code is not consistent. storage.js isn’t necessarily HTTP, so it might have error codes that don’t correspond to HTTP status codes. On the other hand, since storage and HTTP are both essentially CRUD, the same status codes might be used for both. And other errors, say from parsing or serializing, can be statusCode 500. So be consistent, any err returned from a callback should be considered an HTTP status code.