News You Can Bruise for 2007 June 4

(12) Mon Jun 04 2007 19:25 It's Just A Hypermedia Format: There's a children's book about marijuana called "It's Just a Plant". The title always sounded disingenuous to me because if marijuana had no interesting features apart from plantness, nobody would write a book about it. Maybe the lesson of the book is along the lines of: "It's a very special plant, a plant that Mommy and Daddy smoke to get high, but it's not ontologically disjunct from other plants."

In that spirit, here is my defence of WADL, called "It's Just a Hypermedia Format". Sometimes people say they'd like to do strange things with WADL, things I don't approve of and things they wouldn't think of doing with HTML. But WADL is just a hypermedia format. I like WADL because it's a better hypermedia format than HTML or plain XML.

WADL is a way of describing the infinite variety of HTTP requests a client might make in the future. WADL also helps the client figure out which of those infinitely many requests they actually want to make next. One term for this is "service description". Another is "hypermedia". When I say "hypermedia" I just mean links and forms, like you have in HTML.

I'm going to talk about several common objections to WADL, and then put a new objection into circulation which I think is better. A lot of this has been said already, back and forth, but I wanted to gather a bunch of thoughts into one place. I'm taking most of these these from entries in the weblogs of Mark Baker, Aristotle Pagaltzis, Dare Obasanjo, John Heintz, and Patrick Meuller, but also from general murmuring I've heard on the web.

Disclaimer: despite the fact that I wrote a frickin' book about this stuff, I don't think of myself as a "REST expert" and I don't have anything in particular invested in WADL. I just like it as a hypermedia format.

WADL encourages/requires/represents the RPC style.

I'm putting this one first because I think something like this underlies many of the ideas that some standard part of HTTP is what you really want instead of WADL. We all take it for granted that you should rarely if ever need to expose an RPC-style interface through HTTP. It follows that if I fall into the RPC style, I'm probably not using HTTP correctly.

So, here's some WADL for your delectation, a partial description of a service that supports the Atom Publishing Protocol. It's from RWS but it's very similar to stuff in section A.2 of the WADL standard.

  <resource id="atom_collection" path="{feedID}">
    <resource href="#member" />
    <method href="#getCollection" />
    <method href="#postNewAtomMember" />
  </resource>
...
  <method name="POST" id="postNewAtomMember">
    <request>
      <representation href="#entry" />
    </request>
  </method>

You might look at that and think "They've taken a RESTful POST and renamed it 'postNewAtomMember'! Those RPC bastards!" Except "postNewAtomMember" isn't the name; it's the unique ID. The name is still "POST". An HTML form can have an ID, and so can a WADL method. The ID is optional, and its job is to distinguish this POST method from other POST methods so you can reference it from elsewhere. Same as with HTML forms. That "RPC-style" method (once you multiply out the reference to #entry) has the same meaning as this HTML form:

<form action="post" id="postNewAtomMember" 
      enctype="application/atom.entry+xml">
</form>

OK, I'm not naive. I know that lots of people are going to want to make tools that take that WADL file and generate something ugly like an AtomService class which defines a method postNewAtomMember. When they should be doing something non-ugly like defining a "resource" object or class which responds to post.

But the solution is to educate people about REST so they don't make bad tools (and bad services). There's nothing here that's not in HTML forms.

The uniform interface is good enough.

If the uniform interface was good enough, we'd have no hypermedia. The uniform interface restricts the methods you might apply to a resource. It doesn't tell you which methods are applicable to a specific resource, what the representation to a POST or PUT request should look like, which resource the client wants, or how to get from one resource to another. We use hypermedia for that.

There is an HTTP method called OPTIONS, which is supposed to advertise the capabilities of a resource. The first problem is that nobody uses OPTIONS. The second problem is that the only defined behavior for OPTIONS is to say which methods of the uniform interface are supported (in the Allow header). I'd like to try a system where sending OPTIONS to a URI gets you a hypermedia description of that resource, and see where that goes.

(The uniform interface plus) Media types are good enough.

They're usually not good enough to describe the representation for a POST or PUT request. Think about HTML forms. Specifying the enctype as application/form-encoded doesn't save you from having to define the form fields. The client needs some way of mapping its internal state onto the media format the server expects. This can be described in English text, or it can be described in hypermedia, like WADL.

If there's a custom media type for the specific XML vocabulary you're using, the media type might be good enough. The catch is that your XML vocabulary must also be usable as a hypermedia format. Atom is a good hypermedia format because it's got the atom:link element. atom:link has attributes like "rel" and "type" that let the server add semantic meaning to a link for a client to pick up on. But if you need to specify a hypermedia form (see below) instead of just a link, you'll need to interpolate some WADL or XHTML into the Atom file.

You can't get away from hypermedia. Neither the uniform interface nor media types help you describe the levers of state or connect resources to each other.

Links are good enough.

Links certainly are good, which is why as a hypermedia format WADL describes them. If the XML vocabulary you're serving isn't a very good hypermedia format, if your "links" are just URIs stuck somewhere in an XML document, a WADL addendum can explain what the URIs are pointing to and what they mean. WADL links have useful semantic attributes like rel.

But links are not good enough. They're not good enough for the WWW. Lots of RESTful, resource-oriented websites make their clients do things other than follow links. If links were good enough, Google would look like this:

The joke is, there's just a bunch of links to the first letter of your query.

Hypermedia consists of links and forms. XHTML is one way of expressing forms, and WADL is another. AFAIK they are the only two XML vocabularies that can describe hypermedia forms. [Update: I forgot XForms, OpenSearch, RDF Forms, and Poland.]

There are two kinds of forms: forms for manipulating application state (where the action is GET) and forms for manipulating resource state (where the action is POST, PUT, or DELETE).

An application form helps the client figure out the next URI it should GET. This is how a search engine form works. There are a set of data structures that need to be filled in ("q should be a string") and a set of rules for turning that data into a URI (http://www.google.com/search?q=jellyfish). WADL does the same thing, with some additional features like URI templates.

The client is not making assumptions about the URI space. It's calling in structured promises that the server made earlier, possibly in the response to the previous GET. These promises were represented in hypermedia, in WADL as they are in HTML. If HTML 5 was out and had URI Templates, the hypermedia could always be HTML, but right now there are cases where it has to be WADL.

A resource form helps the client figure out how to format the representation it sends along with POST or PUT to a particular resource. This includes the media type (eg. application/form-encoded) as well as a description of what the document should look like, possibly with reference to some XML schema. Lots of people (inc. me) hate XSD, but that's how you describe what an XML document should look like.

RFC2616 is enough.

It's not enough for the WWW. We also have a hypermedia standard: HTML. HTML is not just another media format like PNG. Media formats convey state. Hypermedia formats convey the levers of state. They show the client how to turn its internal desires ("I want the next item", "I want to spawn a subordinate resource with properties x,y,z") into a next request whose URI and representation the server will understand.

Okay, RFC2616+HTML is enough.

Maybe. I haven't heard anyone say this, but I've thought it. If you grant the HTML5 improvements and URI Templates, HTML is good enough for a large number of real-world services.

I know this because in chapters 5 and 6 of RWS I designed a fairly complicated web service that was completely described in machine-readable ways, and I didn't need a line of WADL to do it. Somebody else could write WADL for that service, just as someone else can write their own link-filled HTML guide to crummy.com, but I didn't need any WADL because I chose a different hypermedia format: HTML.

HTML is missing some of WADL's nice features, but if people start serving HTML representations instead of XML representations, then WADL might die out.

It's better to program directly to the HTTP client library.

A properly written (ie. not sneakily RPC) WADL library is an HTTP client library. My Ruby WADL library is pretty lame, but it's an HTTP library. The methods that trigger HTTP requests are get, post, put, and delete. It doesn't look like most other HTTP libraries, because its metaphors are resources and representations and links, where most existing libraries are based on the request/response cycle. But it's not hiding HTTP under something else, any more than Restlet's client library is.

WADL files are monolithic, like WSDL files.

I haven't heard anyone say this explicitly, but it's a reasonable complaint, and it's one of the things that worries me. That's why in RWS I proposed WADL "snippets" that describe a resource's behavior under the uniform interface. These snippets would be interspersed in an XML representation, the way links and forms are interspersed in an HTML representation.

Monolithic "site map" hypermedia files are a problem because they tend to be cached a long time and treated as unchanging. When hypermedia snippets are served along with representations, a client will hear about any changes in the hypermedia description of a resource. It will also hear about any changes in the relationships between one resource and others. A client can automatically adapt to some changes, assuming the semantic cues stay constant. But when the client does all its work off of an old WADL file which contains all hypermedia for the service, it won't know if the WADL file changes.

To sum up, I'd like to respond to something Aristotle wrote:

So this is how you describe a REST service: you explain the type of response body your server will send, where to find links in it, and what sort of semantics and verbs a particular link implies about the resource it points to.

This is exactly what WADL is for. You can describe a RESTful service in English text so that a human being can write a custom (and probably procedural) client. Or you can describe it in hypermedia so that a client library can present it in terms of the resources you were thinking about when you designed it. Right now your options for rich hypermedia are HTML and WADL.