Hypermedia #13
Comments
|
I am surprised that this is marked for 1B, imho there are other, more real-world use issues that would be better implemented in 1B and leave Hypermedia until 1C |
|
I believe Hypermedia elegantly solves many real-world issues such as client decoupling, versioning (along with content negotiation), discoverability to name a few. It also directly contributes to better API design, maintainability and scalability. The idea is to bake its support directly into API Blueprint so you do not even have to think whether you are hypermedia or not. For example this could automatically provide the location HTTP header where applicable or construct the link HTTP header etc. In cooperation with #10 this could go even further utilizing media types such as Collection+JSON, Siren or HAL. Idealistically, should you choose to, I would like you to think about resources, relations and actions not HTTP verbs, URIs and such, but that is definitely even beyond the 1C.. |
|
Has anyone sketched out what this might look like? |
|
@mikekelly, this is how I project it at this very moment: In API Blueprint (Markdown), using some semantic assumptions:
The API Blueprint parser (or its harness) will take the care of injecting the actual model media type and relations (possibly even actions) into the hypermedia affordance provider media type. Using part of @lukestokes foxycart API something along these lines (again, just sketching here): # Resource User [User]
+ Model (application/json)
```
{
"first_name": "Qing",
"last_name": "woldu",
"email": "[email protected]",
"phone": "12345678",
"affiliate_id": 0,
"is_programmer": false,
"is_front_end_developer": false,
"is_designer": false,
"is_merchant": false,
"date_created": "2012-02-29T13:55:09-0800",
"date_modified": "2013-07-11T15:28:56-0700"
}
```
+ Relations
+ self = [User][] ... This User
+ stores = [Stores][] ... Stores for This User
## Retrieve a User [GET]
+ Request (Accept: application/json)
+ Response 200 (application/json)
[User.json][]
+ Request (Accept: application/hal+json)
+ Response 200 (application/hal+json)
[User.hal+json][]
# Resource Stores [Stores]
+ Model (application/json)
...
The API Blueprint parser would take care of injecting the user Note I would also love to have Siren on board alongside with HAL in the first round. Considering other provides later. Also this concept can be extended to auto generate the Link HTTP header. This is in a nutshell how I think about this issue at the moment. There are still few pieces of puzzle missing but that is alright for now. Please let me know what you do think and whether it can work at all. Thank you! Note for @kevinswiber : This is the topic we didn't have the chance to discuss at #apicraft. |
|
I'll let smarter people than myself comment on the approach, but let me just say I'm really excited about seeing progress in this direction! Our sandbox supports both HAL and Siren, so I hope it can be a useful "real world" test bed for trying some of this stuff out. If I can help in a specific way, please let me know. Here are some of the keys I can think of as it relates to the needs client developer:
One thing I was also considering is that not every API has to (or will be) a true REST API. If there's a clear way to distinguish between JSON RPC APIs and REST Hypermedia APIs, that would be fantastic. Great work, Z. I'm really looking forward to where this goes. |
|
@lukestokes I do agree in all points but the very first one – you are very qualified, if not the best, for commenting on this issue! Regarding the clients – as we have discussed with @cainus its the clients where we need to improve the situation next. Not by telling them what they should do but pointing real life benefits as well as offer the best tools to consume our APIs. Now imaging sticking an hypermedia-enabled API Blueprint into a client SDK to configure it for your API and provide realtime request checks etc... (OK now I am totally going both off topic & off the roof :) |
|
I think the only reasonable way of achieving broader adoption is to ship APIs where clients will code against them without even knowing what hypermedia is. This means we need media types that people can treat as plain JSON and they are still just as easy to work with and understand. If I was you, I would pick the media types you initially adopt here carefully. If you offer something that is too complicated, and not intuitive enough - you are going to undermine your larger objective. But I would say that, obviously. |
|
Mike, while I do agree with this line:
However I do believe a broader adoption would be achieved through providing API SDKs and/or tighter framework client integration rather than finding the "holy grail" of media types. I treat a media type just as a mean to transfer the resource representation and, in some cases, hyperlink relations. Idealistically a client should be interested in (and exposed only to) actual resource data and the relations, not in the technical means of the communication itself. In another words as a client you should only need to know your data are and where to get the next. This is exactly what I would like to achieve with API Blueprint. In the first step to sweep the technical means of a hypermedia API under a nice and tidy Markdown carpet, so you do not have to really think HAL, Siren or anything like this – you just focus on your data and relations. Second do the same on the consumer's end – using an API Blueprint on client to hide the HTTP / transfer technicalities and provide directly the data and relations. What do you think? |
|
It's funny, prior to the RESTfest talk by @bsletten , I would have been worried about hiding things behind an SDK or client library. But now, if REST is truly used under the hood, I can see that the format and the datastore don't matter as much, as long as information can be treated as addressable resources which tell you what can be done next. His keynote video isn't up on vimeo yet, or I'd link to it. |
👍 We care more about the media types, because that's what we do. Most people just want to get the job done. The difficult thing when supporting multiple hypermedia types on the same server is transforming capabilities between media types. If one media type only supports links while another supports controls, do you whittle down to the least common denominator? If so, why have additional media types? Do you provide enough information for capabilities of ALL media types supported? If so, do you just discard that information when rendering media types that don't support it? These aren't very easy problems to solve in some cases. I'm curious what route @lukestokes took with FoxyCart. |
Hey @zdne, sorry we didn't get a chance to discuss. Here's how I've been looking at Siren APIs, in terms of documentation. Things to define:
Entities of a particular class may have certain properties, sub-entities, actions, and links. (Note: An entity can have multiple classes.) Defining this is important in documentation. I use class values to route responses to appropriate handlers on the client side. This allows the client to respond to the actual message from the server instead of setting an expectation at request time. For some, this is a different way of thinking, but it's an extremely powerful, reactive pattern. Everything flows from the entity class documentation. Sub-entities have link relations that should be defined. Actions should be defined, and so should expected input fields. Hidden input fields, of course, should not be defined. The Siren spec instructs clients to resubmit hidden fields. As far as functional documentation (docs that are able to submit requests), with Siren, this has to be dynamic. One of Siren's benefits is communicating available actions at request-time, based on resource state. Potential state transitions can change, and so, too, should the active docs. |
We are just at the lowest common denominator right now. I wanted to include actions in our Siren support (and still plan to in the future), but we didn't want to go there until we finalized more important link relationships and property naming conventions. I also want to support Collection, though I'd probably have to do an extension of it because we want to include properties of the collection such as total_items, returned_items, limit, and offset. In the future I may look into XHTML as well. The formatting of the output, imo, isn't nearly as hard as properly designing the API in the first place. As long as the API has a good format layer in place, it's just a matter of moving data around. We did run into some funkiness with how things are done in JSON vs. XML, but for the most part, I think we came up with good approaches that wouldn't appear as compromises. Our hope in supporting multiple formats is that we can be a "real world" API for people to start testing their hypermedia tools against. More tools = more adoption. Feel free to play around with it. I'd love any feedback you have: https://api.foxycart.com/docs Any way, as it relates to this discussion, maybe it would make sense to include @mamund H-Factor ideas when thinking about the documentation system. Just a thought. |
|
I'm just dipping my toes into this hypermedia ocean. The featured apiary apis look great, hadn't discovered it until recently! Apiary looks really powerful! @lukestokes the foxy cart api looks like it already supports hypermedia and consequently serendipitous api navigation discovery! Anywho, as a background, I've been trying to socialize RESTful services and take it to the next level (of maturity) @ The Container Store. As a start, we've been using swagger to "document" our internal API's. Thats a great start, and it seems like a very similar approach to apiary. I've been thinking about how to make the simple API's hypermedia aware, and secondly how to make them discoverable once it is. Like @mikekelly said,
My 2 cents; considerations when it comes to hyper media, documentation and restful services
Glad to help take this discussion forward. |
I'm in the latter camp for sure. Hal happened because I wrote a gist to show what I was building, then bunch of other people got interested so made a little website, then people started asking for a spec... I don't care that much about media types - I care about helping people build effective, usable APIs. There are some media types which I, as someone who's been around this space for a bit, feel are basically over-complicated and solving problems nobody's even sure they have - which is fine in and of itself, but when it comes to projects like this is worth highlighting as a risk for the reason you've outlined here:
Don't get me wrong, I think pushing things further is worthwhile, but I think picking the low-hanging fruit first and then figuring out where to go next is a more reasonable approach. It sounds like this is exactly what @lukestokes has done with the foxycart API - I'll be interested to see where they go next. |
Excellent point Mike! This is my aspiration with API Blueprint. Again looking at FoxyCart HAL Browser: Basically my idea is to describe what is in the left pane in a Markdown syntax and let API Blueprint parser generate what is on the right. Without any prior knowledge of HAL or HTTP / REST. Just tell me about your data and the relations. Simple. |
|
Thanks for your feedback and tips on Siren, much appreciated! Indeed if we decide to go for both the approach of @lukestokes with the smallest common denominator is the way to go. Now, it seems to me the (choice of) (hyper)media type is really more important when you have one end open. Should there ever be something like an API Blueprint driven client consuming an API designed with API Blueprint then the whole (hyper)media type would be really just a 'technicality' under the hood. |
|
Yes that sounds like a really good idea. fwiw, when I spiked the haltalk demo - this is what the link relation documentation ended up looking like: |
|
Yay! This looks almost like an API Blueprint: signupCreate an account [POST]
Very helpful. Thanks for sharing! |
You're in both camps. Everyone has a stake in their work, and hopefully, the belief that it's helpful.
Siren isn't an academic project. It came out of real work, as well. It was solving my problems. Now I'm helping others use it to solve their problems. Siren does have capabilities HAL doesn't. It's okay if you don't need them. Other people do. This conversation between us is a broken record that is being played on 3 different mailing lists. I'd like to show @zdne some respect and not pollute this thread with nerd battle. I'm happy to take this to the Siren list if you'd like to play this song together once more. |
|
So a question I run into from time to time asks if something like this is actually over-specified. The usage of HTTP headers and status codes is already well-defined. You could really just say "Follow HTTP" for a lot of this. Also, there are some things missing that I'd care about as a client developer. What about validation? When might there be an error condition? Are usernames unique? I'm also starting to favor leading with description rather than HTTP, but that's mostly a formatting thing. I'd prefer to highlight "Creates an account" over "POST," because that's the bit client developers are really going to care about. |
great, me too! It's not a personal affront, I'm just offering advice that I think makes sense for this project at this point in time.
Yep - there are many more things this description document could contain, haltalk is a really basic demo - it's not a finished product. The main point of the demo is to show that the various transitions of an API can be broken down into, documented, afforded, and discovered via link relations. It's a no-nonsense strategy for making an API discoverable and moving the attention away from URL patterns. |
I want to say yes, because this sounds interesting, and anything that sounds interesting is worth doing, in my opinion. Innovation is a big, messy pile of ingredients. Sometimes you just need to throw 'em in the cauldron and see what comes out. Media types offer different capabilities, and that's where it gets tricky. No one cares too much about links vs _links. When you're looking at consuming an API, what you care about is features offered by the provider. I think we all agree on this. The question then becomes, as an API provider and potentially someone who will create clients for various platforms, "Which media types help support the features I'm looking to implement?" This is where making a mistake really kind of stinks. You can start off with something minimal and try to bolt-on later, but eventually, the Frankenstein monster emerges... or you intentionally cripple the capabilities of your API. (Those words sound harsh, but it's not always so terrible.) So the argument for starting with a media type that has the most capabilities is there. Hydra[1] is another media type I've taken a liking to lately. I need to examine it more closely, but I would submit Hydra as a candidate for another one of these "capabilities-rich" media types. Of course... you could always come up with whatever format you think would work for you in this situation and birth something like application/blueprint+json. :) You should also take a look at ALPS[2] if you haven't already. It's aiming to add supporting application semantics to APIs. I feel like CCing some people since I'm talking about their work, and surely, they know more about their work than I do. /cc @lanthaler @mamund Now... light that fire and get that cauldron boiling. ;) [1] http://www.markus-lanthaler.com/hydra/spec/latest/core/ |
|
When I started thinking about docs, I figured people building their businesses on our platform will want more information rather than less. I've only got some twig templates in a twitter bootstrap theme, but I thought about having a short paragraph summary of the link relationship along with the affordances (we're a CRUD-heavy api so one relationship does multiple things via various HTTP methods, something OPTIONS can also help discover). Additionally, I want to show the property with detailed descriptions, types and constraints. We also want to show some example outputs in the various formats we support. It's a work in progress, but here are some examples: http://api.foxycart.com/rels/stores When it comes to embedded resources, I also include stuff from /rels/store inside of /rels/stores. Something to think about as users may work with a collection in a slightly different manner than a single resource. I don't think there will ever be a "one-size-fits-all" solution to everyone's needs, but at the very least, focusing on link relationships instead of URLs will be a big step forward, IMO. As long as things are extensible so I could add in my own details, that would be great. Really excited about where this is going, guys. Thanks for encouraging me with your efforts. :) |
Hey, I think this looks pretty good! Capturing all the elements you have outlined there in a more abstract way would get something closer to the hypermedia version of WADL. @lukestokes Thanks for actually implementing a real hypermedia API with public documentation. It's helpful for the whole community and hopefully helpful for FoxyCart, as well. |
Thanks for the nice words and cc'ing me Kevin. Unfortunately, I think I don't have much to add at this moment. I just wanted to say that if there are any questions regarding Hydra I'm here to answer them :-) |
|
I know this thread is getting quite long; good stuff here. I wanted to add one more thing to the mix when considering how to go about documenting a "Hypermedia API" I've recently gotten into the habit of documenting the actions, not the objects. I have a (very) rough example posted in another repo[1]. Right now this approach takes a couple steps (not ideal):
I think that is too unwieldy and can be greatly improved. However, my point here is to pitch documenting the hypermedia controls, not the resources. Its a very diff approach, but something I think can work well. It allows servers to decide their own URIs, their own representation details (what data and which controls appear in responses) and still gives client devs enough to write a working client (that recognizes both the data and the controls when they show up in a response). Anyway, its just another approach to consider. Cheers. |
More features in the media type does not translate to more features of a service. Compromising on media type "features" and instead, folding that complexity into something else (e.g. link rel documentation) does not make a service "inferior". It is a design decision. more can be less. |
|
Defending complexity by offering more "features" is a well proven sales technique, though - I'll grant you that. |
|
@mikekelly I acknowledge your opinions. I have a feeling individual stubbornness on both our parts will prevent compromise. You admit HAL is intentionally minimalist yet claim anything more complex is a waste. For some reason, you think HTML is the only other media type that gets to play in the game. You haven't recognized any positive aspects regarding any other hypermedia type. That is not very constructive to the conversation. I get it. You want everyone using HAL exclusively. That won't happen, in my opinion, because of its minimalist design. I can program with GOTO all day long, but you know what? Sometimes I like IF-ELSE structures, too. You are a very loud minority, and your opinions, though acknowledged, are losing value. |
|
@mamund This approach is very interesting. Are the actions for a link ID always static or could they change at runtime? |
Personally I just take that as a given. Nobody argued otherwise. The problem is what is the right way to approach the "job of enabling people to use the thing you've built"? In my opinion, stating that "more education is needed first" is a huge smell that you are not heading in the right direction. |
What do you think should be done? How can we "sell" these design concepts better? |
|
One thing I've been playing with lately is starting with HTML as the base representation of a resource and then translating that to other hypermedia types. In doing this, you markup the resource using HTML+RDFa (or Microdata, whatever works) along with links and forms for state changes, and then pull that data out of the HTML, either server-side or client-side. Here is a very crude, incomplete example that converts HTML+RDFa to HAL+JSON. My question is, would starting with HTML as your media type allow you to do a lot of this described here? If you used RDFa along with links and forms, couldn't you translate it to pretty much any other hypermedia type? If so, you just made your browser into a browser for any media type you're working with. It would be the ultimate HAL browser or Collection+JSON browser. Also, could you then generate something like Blueprint or ALPS from that HTML with special crawler of your HTML API that doubled as tests for your API? Your API would not only be discoverable, but self-documenting. Developers could follow their nose through your API using their own browser. I see other benefits as well (such as being great for search engine crawlers and noscript users), but I'll leave it at that for now. Just a thought! |
Better designs, basically. It shouldn't have to be so complicated that it requires lengthy specifications, books, blog posts, or github threads. |
@mikekelly So, they simply pull a better design out of their hat because they don't understand design concepts and have not educated themselves on it Hmmm. What am I missing? I think the aspiration here is just enough to help one understand and come up with better hypermedia API designs (and some advanced functionality for the advanced). If you think it is missing that, I think @zdne question was what do you think could change to accomplish that better in what is being done here. |
|
@zdne After looking over the Gists example for the Resource Blueprint I feel like I'm missing something. Gists Resource specifies a state transition to a |
|
@mattupstate We did some editing and missed that one. Will fix. Should be |
|
Do you see another benefit to using HTML as a hypermedia type besides the navigation in browser? Is designing per-resource media types in HTML and chaining them together the API design process you would use? What about making it other way around. Transport from HAL (or SIREN) to HTML5. Sort of HAL+HTML. Do it over a proxy and you can surf a hypermedia API in your browser... Also note API Blueprint is not a media type for hypermedia APIs. But technically, yes, you could compose a blueprint from browsing an existing API (similar to cURL trace parser). |
|
I think the big benefits would come for applications that are meant to be used in the browser, while it seems like it could help in educating developers on how to design hypermedia as has been discussed. Web developers understand HTML links and forms in the browser, whereas they may not iniitally understand hypermedia in JSON that they have to consume in some separate application. Additionally:
So it seems like HTML allows you to be more expressive, even if you are just making your API browseable with it. With that said, it has its limitations, such as only currently supporting GET and POST, so you'd have to hack it with _method inputs like Rails.
I'm not sure yet, but I'm thinking about it! I'm trying to build some tools to see if it simplifies my design/development process, such as the HTML-to-HAL one above. It does seem like it simplifies things when all you have to build is HTML for your API and the rest is automatically done for you. Edit: just to clarify, I don't guess you'd have a bunch of resource-specific media types, but rather use HTML for everything with RDFa to convey your domain semantics. |
|
@smizell Jon Moore would definitely love this idea ;-) I agree you "can" use HTML for anything as it has affordances built in and is very flexible, however it comes at a cost. I think it's real value is I can hand it off to any browser and it can render it. But....You have to work within the existing facilities to express what you want, and that to me is a big downside. If I create a profile for example on JSON, I can tailor JSON it so it really fits my domain and is very obvious as JSON at it's lowest level is just a lightweight way to express dictionaries and lists. If I go down the HTML route I have to work within the constraints, use the attributes it provides like name here, class there. If I see an Also I find that consuming HTML in a JS application is less than ideal. |
|
@glennblock I'm actually working on a library called halpert to address these things you mention, and there is a jQuery version to use for doing this in the browser. This allows you to mark up your resource content using HTML+RDFa, which is much more expressive than using classes/ids, and then parse it to HAL+JSON. Here is an example (view the source), with the parsed HAL+JSON on the right that is derived from the HTML document. I didn't mean to hijack this thread with HTML talk, but my original intent was to mention that starting with HTML as your hypermedia type allows you to address a lot of the issues mentioned higher up, even things like increasing education and awareness on hypermedia design. |
|
I'm totally late to the party, but thank you for the rich discussions so far. Definitely mirrors discussions happening as part of our struggles to define APIs internally, as part of improving both client-side and server-side developer experiences.... Just reviewing the Resource Blueprint. I think it's a step in the right direction, which I haven't seen any other similar initiatives take yet. One thing that I'm either confused or concerned (confuscerned ?) about:
Transition to different states - from the client side this makes sense. Here are the different actions you can possibly undertake, and should look out for as part of a response. The conditions to me are part of the underlying business logic, which I’m trying to shield the client from with an API. Why this affordance appears in the response shouldn’t be a client side concern. From a server-side perspective, the conditions can and should evolve as my system grows, and I should be free to do so without revisiting my resource definition, as I don't think it breaks my client-side contract. From a practical point of view:
|
|
@drewzboto If you look at the use of Whether it is business logic (does the transition even exist) or permissions (the transition is available in the context of the user, role, etc) is all transparent to the client and is encapsulated in the API implementation as you note. Not everything in the blueprint will roll over to the human readable documentation generated by it. It is a good question whether conditions should. Presumably somewhere if there are permissions that should be set up, you would want to document their applicability. Not sure that would be via Given the blueprint is itself a design tool, thinking about permissions separate from the existence of a transition in a state is a useful abstraction, and thus the point of its inclusion. @zdne may have other thoughts. |
No, I would say the party is just about to start – we are only at the dawn of the "API design" age.
No. The resource blueprint is a permissive language. The minimum reasonable blueprint would include just a resource with one attribute, perhaps one affordance (other than "self"). Documenting conditions clearly depends on the API designer. Should I document a condition that controls whether I can "create" a resource? Probably yes. Should I document a condition that controls whether I can perform a "checkout"? I would say no if it is obvious (e.g. cart is empty clearly implies checkout does not make sense). But again it is up to the designer(s) to decide. The option is here. As for rendered documentation – it is a good question whether and how it should capture the conditions. My feeling is it should. Important thing here is that the blueprint shouldn't be written in the respect "how rendered documentation looks" but with the focus on the particular API design and the use of blueprint in the API lifecycle process. Rendered documentation is just a tool using it, and there are hopefully many, not just the one from Apiary. |
|
Are there any updates to this? Would be great to see a more efficient way of using hypermedia links/embeds in blueprint soon! |
|
@stefanpearson Yes, we are making quite a progress towards Resource Blueprint with the MSON. MSON is the building stone for Resource Blueprint as it will enables the description of Resource attributes (semantic descriptors) and also of the affordances. I also plan to update Resource Blueprint concept soon. I will keep you posted in this thread. |
|
That's good news :), I'm having to do some very questionable pre-processing to get around this! |
@stefanpearson can you share more details on this? perhaps there is something we can learn from it |
|
Well I haven't implemented it yet, but my end-goal at the moment is to:
It's worth noting that I'm also using the resource/action schemas (JSON schema) to validate incoming request bodies, and validate outgoing response bodies within my tests. In lieu of this, the JSON schema output needs to actually include and represent the relevant hypermedia. At the moment, as a workaround, my immediate thought is to:
|
zdne
added
scope (1): Language
status (1): Idea
type (2): Feature
and removed
Hypermedia
labels
Direct support for hypermedia and modeling actions & relations.
The text was updated successfully, but these errors were encountered: