Roy Fielding, the inventor of REST, wrote a blog post recently titled REST APIs must be hypertext-driven. It's quite hard to understand, being written in pure academese, but I think I get the gist.
The gist is that for an API to be properly RESTful it must be discoverable. Specifically, you should be able to point a client at the root URI (/) and have it find all the resources that the API exposes. This is a cool idea, in theory, but very problematic in practice.
A consequence of this restriction is that any sort of documentation that contains a list of URIs (or URI templates, more likely) and documentation on accepted parameters is verboten.
Presumably, if I had a sufficiently smart client that understood the media types used in the application, I'd point it at the root URI, it'd discover all the URIs, and I could manipulate and fetch data along the way.
That's a nice theory, but has very little with how people want to use these APIs. For a simple example, let's take Netflix. Let's assume that I want to use the Netflix API to search for a movie, get a list of results and present it back for a human to pick from, and add something from that list to my queue.
Without prior documentation on what the URIs are, how would I implement my client? How do I get those search results? Does my little client appgo to the root URI and then looks at the returned data for a URI somehow "labeled" as the search URI? How does my client know which URI is which without manual intervention?
If I understand correctly this would somehow all be encoded in the definition of the media types for the API. Rather than define a bunch of URI templates up front, I might have a media type of x-netflix/resource-listing, which is maybe a JSON document containing label/URI/media type triplet. One of those label/URI pairs may be "Search/http://...". Then my client POSTS that URI using the x-netflix/movie-search media type. It gets back a x-netflix/movie-listing entity, which contains a list of movies, each of which consists of a title and URI. I GET each movie URI, which returns an x-netflix/movie document, which contains a URI template for posting to a queue? Okay, I'm lost on that last bit. I can't even figure this out.
Resource creation and modification seems even worse. To create or modify resources, we would have a media type to describe each resource's parameters and type constraints, but figuring out how to create one would involve traversing the URI space (somehow) until you found the right URI to which to POST.
Of course, this all "just works" with a web browser, but the whole point of having a web API is to allow someone to build tools that can be used outside of a human-clicks-on-things-they're-interested-in interface. We want to automate tasks without requiring any human interaction. If it requires human intervention and intelligence at each step, we might as well use a web browser.
I can sort of imagine how all this would work in theory, but I have trouble imagining this not being horribly resource-intensive (gotta make 10 requests before I figure out where I can POST), and very complicated to code against.
Worse, it makes casual use of the API much harder, since the docs basically would say something like this ...
"Here's all my media types. Here's my root URI. Build a client capable of understanding all of these media types, then point it at the root URI and eventually the client will find the URI of the thing you're interested in."
Compare this with the Pseudo-REST API Fielding says is wrong, which says "here is how you get information on a single Person. GET a URI like this ..."
Fielding's REST basically rules out casual implementers and users, since you have to build a complete implementation of all the media types in advance. Compare this to the pseudo-REST API he points out. You can easily build a client which only handles a very small subset of the APIs URIs. Imagine if your client had to handle every URI properly before it could do anything!
In the comments in his blog, Fielding throws in something that really makes me wonder if REST is feasible. He says,
A truly RESTful API looks like hypertext. Every addressable unit of information carries an address, either explicitly (e.g., link and id attributes) or implicitly (e.g., derived from the media type definition and representation structure). Query results are represented by a list of links with summary information, not by arrays of object representations (query is not a substitute for identification of resources).
Look at last sentence carefully. A "truly RESTful API", in response to a search query, responds not with the information asked for, but a list of links! So if I do a search for movies and I get a hundred movies back, what I really get is a summary (title and short description, maybe) and a bunch of links. Then if I want to learn more about each movie I have to request each of 100 different URIs separately!
It's quite possible that I've completely misunderstood Fielding's blog post, but I don't think so, especially based on what he said in the comments.
I'm not going argue that REST is something other than what Fielding says, because he's the expert, but I'm not so sure I really want to create true REST APIs any more. Maybe from now I'll be creating "APIs which share some characteristics with REST but are not quite REST".
Cross-posted from House Absolute(ly Pointless) - permalink.
I thought you of all people understood REST.
Presumably, if I had a sufficiently smart client that understood the media types used in the application, I’d point it at the root URI, it’d discover all the URIs, and I could manipulate and fetch data along the way.
That’s a nice theory, but has very little with how people want to use these APIs.
That’s not theory. That’s the web. Also, have you heard of something called AtomPub? That’s like that, too. There are a lot of desktop weblog publishing client that use it.
Without prior documentation on what the URIs are, how would I implement my client?
Yeah, how? How did these people write weblog clients against AtomPub?
How does my client know which URI is which without manual intervention?
Because it saw something like <link rel="search" href="..."/> (in the case of Atom) in some other response body it already received.
I can’t even figure this out.
Read the AtomPub spec.
Fielding’s REST basically rules out casual implementers and users
Nonsense, I’m afraid. (Quite the opposite, actually, it invites reuse in the small in a way that RPC-ish systems cannot – Roy terms this “engineering for serendipity”.)
since you have to build a complete implementation of all the media types in advance.
How do you write a program that can do something with information it has no way of parsing?
A “truly RESTful API”, in response to a search query, responds not with the information asked for, but a list of links! So if I do a search for movies and I get a hundred movies back, what I really get is a summary (title and short description, maybe) and a bunch of links. Then if I want to learn more about each movie I have to request each of 100 different URIs separately!
You’re twisting Roy’s words in his mouth. Atom feeds can contain the full data for each entry, and if you ask Roy what he thinks about AtomPub, he will tell you it is a great example of getting REST right. The salient point is that the Atom feed contains the URI of any entry that has one, and that is what you use to manipulate the entry if you want to do so.
Re:Et tu, Brutus?
autarch on 2008-10-24T03:08:37
I thought you of all people understood REST.
To be honest, I'd totally missed this part of it. I understand why Roy thinks it's important, but I'm not sure that I think it's important.
My goals for doing something REST-like are several fold
... First, share the same URI space between what a browser sees and what a non-browser would see. This works quite well for everything that should be POST, PUT, and DELETE. For GETs, you end up with a bunch of stuff like
/person/1/edit_formbecause browsers kind of suck, but that's ok.Second, by sharing that URI space, I can re-use all my non-GET code. The GET code I can't, because the HTML I deliver to a browser is nothing like what an application client would want, but again, browser sucks, so that's ok.
I do like the idea of providing actual "media types", although I imagine my media types would mostly be well-defined JSON "blobs". This just makes sense, and should be part of one's spec for an API anyway, since it basically defines input and output parameters.
However, I'm not convinced it would be better to not document the URI space.
I get the concept Roy is talking about, and it sounds cool. OTOH, if I just wanted to write a client to work with a small subset of a server's entities, I'd be damn annoyed if I couldn't find a direct way to get what I want.
As far as AtomPub goes, it does seem like one would have to implement a full client to do anything with an AtomPub server that doesn't document it's URI space. You might point out that many AtomPub clients are exist, which is great for AtomPub, but not great for some other application which uses its own one-off media types.
You're twisting Roy's words in his mouth. Atom feeds can contain the full data for each entry, and if you ask Roy what he thinks about AtomPub, he will tell you it is a great example of getting REST right. The salient point is that the Atom feed contains the URI of any entry that has one, and that is what you use to manipulate the entry if you want to do so.
I'll take your word for it, but I was not twisting his words. He said that queries are represented by "a list of links with summary information". I'm not sure how else to interpret that.
If he's just saying that results should contain URIs, then no shit. I thought that was obvious. But I'd also expect results to come with whatever information I asked for, not just a summary and a link.
I blame ORA's REST book for my misunderstanding, at least in part. It really never discusses this aspect of REST. It does make the point that results should contain links, but it doesn't ever quite state the whole discoverability concept. It talks a lot about coming up with URIs, but never says that to document them as part of the API would be wrong.
Unfortunately, it's the only thing I've found that talks about large parts of REST clearly, unlike Roy's writing, which is ridiculously academic. Much of the other REST stuff on the wiki (like on the REST wiki) is similar.
Re:Et tu, Brutus?
Aristotle on 2008-10-24T06:14:28
I imagine my media types would mostly be well-defined JSON “blobs”.
Sure, that’s fine. Really, you don’t necessarily have to provide a specific media type; the core issue is that you document the response format, and that that format use links rather than application-specific IDs that the client must slot into URI templates which are never part of the in-band communication, and generally have to be built into the client at programming time.
The other thing is, hopefully you would mostly use well-known, extensible media types (eg. Atom) as the base content type for most of what you serve, so that clients actually don’t need to reimplement this parse-and-follow-links bit over and over. (Using a documented subset of XHTML works well when the structure of AtomPub does not quite match your needs: it’s XML and therefore easily parsed, but it’s also browser-navigable at the same time – very nice.)
However, I’m not convinced it would be better to not document the URI space.
It looks that way from the point of view of any one service, yes. You have to take a larger view to see that the point is not to have to write clients entirely from scratch for every single service, which ultimately benefits all of the individual services as well. (Imagine what would happen to the web if you had to use different browsers for different sites, or even just for different kinds of sites. Yet that is how we build programmatic agents today. AtomPub is the only large-scale exception I know of so far.)
but not great for some other application which uses its own one-off media types.
And then the question becomes, does it really need them? Most web services reinvent wheels in areas where that is not at all necessary.
It really never discusses this aspect of REST.
Yes. This criticism has been levelled against the book by many of the people in the REST circle. Sam’s response has always been along the lines of progressive disclosure – which I kind of agree with, however I still think they needed to expound on this aspect at more length. It is really the cornerstone of REST – all of the other parts are in a sense groundwork laid down to enable hypermedia, which is the actual end goal, since that is how you achieve real decoupling.
