Perform RDF Merge for POST to existing LDP-RS

[kjetilk]

The way I have previously implemented a POST operation on an existing LDP RDF Source is to perform an RDF Merge between the existing resource and the payload body.

I could have bet I read that in a spec at some point, but I can't find it, and it doesn't seem to be implemented? Intuitively, I think it makes sense, and is a very simple operation.

[melvincarvalho]

Do you have a use case?

Isnt that covered by PATCH with and INSERT the body. As RDF is a set, the dups would be removed.

questions

• What do you return when something cant be merged (like form encoded POSTs)
• What happens when you post an empty body?
• Does it trigger the websocket?

[kjetilk]

Do you have a use case?

The simple case of just adding a few triples to a resource. Like in solid-yo.

Isnt that covered by PATCH with and INSERT the body. As RDF is a set, the dups would be removed.

It is covered by the PATCH, but at the cost of writing SPARQL and firing up a SPARQL engine. It seems overkill and bad DevX to require that for just adding a triple.

You could have servers that don't wish to support PATCH because of SPARQL, but they could support POST and the RDF Merge algorithm. An example of this is a microcontroller, you can't have SPARQL on 64 kB RAM. :-)

Just adding a few triples is the actual thing you would do, tying it to the RDF Merge definition is just for ensuring that it is rigorously defined for the blank node-induced corner cases. :-)

questions

* What do you return when something cant be merged (like form encoded POSTs)

It works if all payloads are RDF, the details need to be worked out if that is not the case, I would think a 400 would be the right response.

* What happens when you post an empty body?

Nothing.

* Does it trigger the websocket?

Dunno :-)

[melvincarvalho]

You could have servers that don't wish to support PATCH because of SPARQL

Not in solid you cant, it's part of the spec. See

https://github.com/solid/solid-spec/blob/master/api-rest.md#alternative-using-sparql-1

Is the proposal to remove this from the spec?

[kjetilk]

Yeah, I know, and no, I don't want to remove it. I'm just saying that POST to do RDF Merge is a "90% solution" that would simplify DevX, and could accomodate for those that cannot do a full implementation.

[melvincarvalho]

We dont run a full SPARQL engine, we just strip out the initial INSERT. Maybe that has lead to some confusion.

My initial concerns

• There doesnt seem to be a use case articulated yet (which is not a show stopper)
• We can do this already in solid

I think this proposal, probably belongs in LDP next?

[kjetilk]

Perhaps it should go in LDP Next, but I think the question, in the interest of DevX, needs to be reversed: What is the use case for using SPARQL, if RDF Merge can do the job?

Ah, BTW, found something, there is something about this in the RDF Graph Store. Lots of stuff in SPARQL is defined in terms of RDF merges.

To answer my own question, PATCH is used to do actual edits, i.e. both deletes and inserts, and we want to be able to extend what the PATCH method can do, right? But still, I think it is important of how the simplest operations are perceived by devs.

[melvincarvalho]

What is the use case for using SPARQL, if RDF Merge can do the job?

Delete?

We are open to extending patch in future, but right now it's just simple insert and delete (I think!).

As long as I can remember POST is up to the implementer to decide what to do with. We only define POST for LDPC's.

What about improving the devx be part of an easy to use library?

What would it do?

• add the INSERT {} around the payload
• set the content type to application/sparql-update
• set the VERB to PATCH

A better route?

[kjetilk]

Yeah, I made that edit about delete right there before your comment. Obviously :-)

Libraries are fine. We can do it in libraries, I just don't see why you would want the extra complexity for something this simple. INSERT DATA {} is doing an RDF merge behind the scenes, it is just extra complexity. That extra complexity needs to be justified. Obviously it can be justified, I just don't see why you should have everyone deal with the extra complexity.

[melvincarvalho]

I've added the tag enhancement to this. Because it's not really an issue in the spec, it's a feature request.

So, I think the next step is to turn this into a proposal.

What strikes me is that the proposal covers half or PATCH (insert and not delete) and is inferior due to lack of extensibility.

All solid servers can do this already, and this is not wanted outside of solid servers. So no new functionality.

It seems the main justification is some syntactic sugar around INSERT DATA {} which comes at the cost of possibly even confusing developers with two ways to do something.

There doesnt seem to be a use case behind it, right now, so I'm not optimistic for this propsoal's chances. Interesting food for thought, tho!

[kjetilk]

It seems the main justification is some syntactic sugar around INSERT DATA {} which comes at the cost of possibly even confusing developers with two ways to do something.

No, no, it is the reverse! INSERT DATA {} is syntactic sugar around this! Just read the SPARQL spec, it is even defined that way. Now, Solid's use of SPARQL isn't compliant with the SPARQL spec, but if it was, it would have to implement RDF merge to implement INSERT DATA {}, and then, implementing the POST as suggested would be 2 lines of code, whereas implementing INSERT DATA {} would use that code after stripping the SPARQL from the RDF, which would require some extra work (not a lot, admittedly)

Simple things should be easy, that should be the main guiding principle. Devs are familiar with the idea that POST adds stuff, PUT replaces it, DELETE deletes it. That intuition should take them 78% of the way. At least :-)

Then, they could move to the next level, where editing requires rudimentary understanding of SPARQL. Some devs my become scared at that point, and not be willing to go there, but those that do might be motivated if they had already been able to do stuff with simple HTTP operations.

[kjetilk]

One may add, the stuff between the {} in INSERT DATA {} are the exact triples that you would just POST, which also illustrates that INSERT DATA {} is extra complexity, and only extra complexity, if your goal is to add triples.

[melvincarvalho]

Got it! It's certainly interesting.

Some Meta Comments

So, we need a process to manage such proposals, and I suggest we chat about that out of band.

Typically with early specs, you can change them, but as solid is maturing now, we need to be more cautious, and create more of a process.

Fast track changes such as typos or the wrong path.

Proposals with wider implications, such as changing the sense of an HTTP verb (like this one).

With some gray areas in between.

I think we can loosely base it on W3C / IETF work flows such as raising issues, and then then ensuring commenters are satisfied.

I find the following doc great guidance in this respect, in case you may not have seen it

https://tools.ietf.org/html/rfc7282

ends meta

[acoburn]

One concern I would have about this is that, with this proposal, POST on an LDP-C would create a new resource while POST on an LDP-RS would update the target resource. Given that an LDP-C is a specialized type of LDP-RS, it would require clients to first check the resource type before issuing a POST. It also makes the semantics of POST inconsistent across different resource types.

What about LDPatch? This is a much lighter-weight "update" mechanism than SPARQL-Update, and it can be used with PATCH operations. The LDP-RS would advertise support for this via: Accept-Patch: text/ldpatch.

[kjetilk]

Given that an LDP-C is a specialized type of LDP-RS, it would require clients to first check the resource type before issuing a POST. It also makes the semantics of POST inconsistent across different resource types.

Right. Then, I suppose you are right, this is not something that is easily doable with LDP.

But you know, you gave me a good argument to say that my hypermedia-approach to this problem is better, because it tells the client what it can do as a part of the metadata of the resource. Now, Tim didn't like the idea of mixing in metadata, and I can see that point, but it reinforces my skepticism towards LDP, it requires too much up-front, unintuitive, out-of-band knowledge.

Anyway, now we have LDP as the foundation, and we need to see where it takes us for now. So, just leave it here as a proposal.

[michielbdejong]

I do like the simplicity, but I don't like that you can't use it to delete/update triples, you can only append. And I think we have enough different PATCH syntaxes as it is, let's not add an extra one? Can this issue be closed?

[kjetilk]

Can this issue be closed?

Not sure I feel about it... I think it is such an obvious way to do stuff that we should have that append with RDF-Merge semantics, everything beyond that could confuse people.

However, since it can't be a part of LDP as it stands now, perhaps it does not belong in solid-spec at all, but in LDP Next.

Perhaps just keep it open until we see if LDP Next will actually happen?

[csarven]

It also makes the semantics of POST inconsistent across different resource types.

It appears to be inconsistent only because the semantics of POST is not strictly locked down to only creating. Still within RFC. LDP seems to leave room for clients to use the interaction model to be more explicit on the intended semantics of POST.

LDP suggests to use PATCH for "sophisticated merge of data" (instead of PUT) and doesn't prohibit the use of POST to accomplish that either - I suppose given RFC's "Appending data to a resource's existing representation(s)."

What I take away from LDP off-the-shelf is that PATCH and POST can potentially be used interchangeably in particular to update to add/append/merge data against an LDP-RS. However, LDP just seems to lean on PATCH because the client can potentially request the server to perform other operations in addition.

POST is exciting but vague. The Solid spec could of course clarify this by making a call (one way or another) that operations like insert/delete should only be initiated with PATCH and/or state that POST should only be used for adding data. That's a question without a question mark.

[kjetilk]

OK, thanks @csarven !

My starting point has always been what I perceive as a strong developer intuition, which is backed by RFC2616 and RFC7231 alike, that POST can be thought of as an append operation. Back before my Ph.d., which is a while, that seemed to be how we used it. It is good to see that this can be reconciled with LDP. :-)

I might have become out of touch with developer communities, and it might be that the HTTP verbs are not that important given libraries anymore, but if possible, I think this belongs in the interaction model.

[csarven]

If POST for RDF merging is popular out there then we can bring more awareness to that but it doesn't entail a new requirement as I see it.

It seems simpler to me to rely on PATCH for graph-based updating in general than to emphasise on the possibility through POST for a specific kind of update. The Solid spec could require one eg. application/sparql-update as MUST support. Other formats eg. text/ldpatch can still be negotiated (as Aaron mentioned) with Accept-Patch.

I don't quite follow what you mean by belonging in the interaction model - do you mean to have a new model? Is there a loss of signal somewhere with POST? Wouldn't PATCH be sufficient? where both are accompanied with preferred format for merging.

[kjetilk]

OK, perhaps the mention of interaction model was not appropriate, but as for the comment to

Is there a loss of signal somewhere with POST? Wouldn't PATCH be sufficient?

the point is really that POST is what I think people would expect for appending stuff to the graph, it has been done that way for decades, so they would just do what they've always done. Then they see it works, then great.

With PATCH only, they first do a POST, see that it doesn't work. Then, one of two things could happen, either they conclude Solid is hard and go somewhere else, or they conclude that Solid is harder than they thought and go look up the spec. Then, they find that they could do the same thing with PATCH, which kinda makes sense, so they learnt something, and they are happy. Or they might be just terrified to see that they might need to learn SPARQL...

I think that we shouldn't make people look up the spec for something like this, it is poor DevX. If we don't need to be Solid-specific, we shouldn't be. But I could be entirely wrong about it, I just don't think we should be risking it.

[csarven]

I acknowledge your core point and generally agree I would just add not looking things up is one way of going at it. Of course the other is to check the spec, the companion primer, tutorials etc to get acquainted. Intending to do RDF merge via POST wouldn't be straight-forward as one would need to know the permitted formats and how to formulate the rest of the request any way. It is not an arbitrary POST and I would even say that it is probably even one of the most complex requests. So, it still results in looking up the spec/primer at one point.