I guess it's time to announce my next book project. Mike Amundsen and I are working on a book for O'Reilly called
RESTful Web APIs, a follow-up to 2007's RESTful Web
Services. Our deadline is the end of March 2013, and the book
should come out a few months later. (Here's Mike's announcement of the project.)
Here's my announcement
of RESTful Web Services from about six years
ago. You can see how far we've come since then.
In 2008 I came up with the now-famous "maturity
heuristic" (aka the "RMM") for quickly judging the quality of an API. If you go back and
look at RWS through the lens of the maturity heuristic, you'll
see that it spends a lot of time trying to get the reader to grasp the
concepts behind Level 1 services: that URLs identify resources,
and that resources are the proper targets of HTTP requests. That's
because back then, Level 1 was about the best you could expect from a "REST" design.
RWS spends a whole chapter treating Amazon S3 as though it
were a great example of RESTful design, because in RMM terms it made
it all the way to... Level 2. RWS spends another chapter
arguing for the legitimacy of REST against a thriving part of the industry that's
content at Level 0.
We don't live in that world anymore. I'd say the modal RMM level of
a new "REST" API today is 2. Most of the 2007-era arguments have been
rendered irrelevant (SOAP vs. REST, the usefulness of PUT) or been
given generally accepted best practices (URL formats). Restful Web
Services was the first book-length treatment of REST; there are
now over ten books on REST from O'Reilly alone. There are lots of
development tools that make it really easy to crank out a Level 2
service.
In short, a lot of gravel- and rock-sized problems have been excavated,
revealing two boulder-sized problems. I mentioned these two problems
in my QCon talk "How to Follow Instructions" (available as video in hour-long version and in a half-hour condensed version
presented at RESTFest; see also my QCon slide deck and speaker notes). Now I'll state them explicitly as the two
major problems I want to address with RESTful Web APIs.
Duplication of effort
Here's an article from May on Programmable Web, a great site
that tracks the world of public APIs: "53
Microblogging APIs: Twitter, TwitPic and What The
Trend". Fifty-three was really the number of organizations
publishing microblogging APIs, and some of them are utility APIs rather than "microblogging APIs" per se, but after doing some spot checks I do
believe there were approximately fifty-three distinct APIs.
Should there really be fifty-three distinct APIs in this field?
What if there were, like, five? We're not talking about something
complicated, like insurance policies or regulatory compliance. We're
talking about posting a little bit of text to a user account. Can we
at least get down to ten?
No, apparently not. Like I said, that number was from May. There
are now fifty-seven
microblogging APIs. Well, damn. What if we got together and
hammered out a standard, interoperable microblogging API? Sounds
great, except nobody's interested. Evan Prodromou tried that with
OStatus, and nothing came of it. You could use AtomPub as a
microblogging API, pretty much unmodified, but nobody seems to want to.
The original RESTful Web Services sets out a design
procedure for taking your problem domain and turning it into an
API. But there's a big problem with this procedure: it doesn't
consider the possibility that fifty-six other people may have already
done the work for you.
Back in 2007, that possibility was remote. There was one
microblogging API. I had to scrape the barrel pretty thin to find
enough quasi-RESTful example APIs to fill Appendix A. Now the big problem
is too many APIs. When you design the fifty-eighth
microblogging API you're limiting your audience and wasting your
users' time.
This is a really huge problem and we won't solve it with a book. But we can point out that it's a problem and take the first step towards mitigation. In RESTful Web APIs we present a new design procedure that focuses on reusing existing designs, coming up with new stuff to fill any
gaps, and then publishing the new stuff for the benefit of a) your users
and b) API developers who come after you.
Hypermedia
Whatever influence RESTful Web Services has had, it stops dead at RMM level 3, the hypermedia level. In 2012, the modal new "REST" API is at level
2. It has been level 2 for a while, and will continue to be level 2
unless something changes. Most architectures don't support hypermedia
very well (or at all), and a lot of developers still don't understand it.
That's understandable. To the programmer brain, hypermedia is weird. When you read
RWS it's possible to just skip the parts about hypermedia, and
I get the feeling a lot of people did. There's stuff in RWS
that is now outdated, and stuff that was useful for dealing with
2007's problems but that I now consider just plain wrong. Most of that
stuff has to do with hypermedia.
RESTful Web APIs is tightly focused on hypermedia--you will
not be able to skip the "hypermedia parts"--and it deals with 2013's
problems. Some specific shortcomings of RWS we'll address:
- We as developers have a strong desire to export our server-side
object models as APIs. This is the idea behind JAX-RS. It's the
idea behind lazr.restful, the framework I developed to make the
Launchpad web service. But it's also the idea behind SOAP, and it's how we got fifty-seven microblogging
APIs.
This desire isn't technically at odds with the use of hypermedia,
but it does tend to use hypermedia only to describe safe state
transitions (i.e. to link resources together), and RWS didn't
really go further than that.
- In 2007 there were no hypermedia formats based on JSON, but
JSON-based representations were becoming overwhelmingly popular. RWS does not offer advice for this scenario. Developers
either made up their own one-off hypermedia formats (as I did with the
Launchpad web service), or, more commonly, ignored the hypermedia
constraint. Today JSON has a number of general-purpose hypermedia
formats (like Siren, Collection+JSON, and HAL), and we'll cover them
in RWA.
- In RWS I was very dogmatic about avoiding overloaded
POST. I felt strongly about this because I really needed to yank
people out of the SOAP-RPC mindset. But I've come to accept that overloaded POST exists for a
reason: to convey state transitions that don't map naturally onto one
of the other HTTP methods.
These state transitions are legitimate if they are described by
hypermedia controls, as they are on the Web. They're not just hacks to get around the fact that HTML can't describe PUT and DELETE methods. Prohibiting
overloaded POST in the name of the uniform interface forces designers
into complicated workarounds to minimize the difference between their application semantics and the protocol semantics of
GET/PUT/POST/DELETE. Those workarounds usually reflect the object
model the designers already defined for their CRUD databases. Or, the
designers give up, use overloaded POST without using hypermedia
to describe the state transitions, and feel guilty about it.
Besides which, forbidding overloaded POST means that you can't use
HTML, the world's most popular hypermedia format. That's a big
problem.
The title
As I said, this is a brand new book, not a second edition of
RESTful Web Services. The only major things we're reusing are the
appendices dealing with HTTP headers and response codes. To signify
the handoff, we're changing the name.
RESTful Web Services was a great name in 2007, but it's not
so good now. See, when SOAP went down, it took the term "Web Service"
with it. "Web Service" is now an enterprise software term. Everywhere
else it's all "API API API".
As I mentioned last
year, this is backwards. The SOAP approach is to make a remote
service look like a local library. (This idea is not limited to SOAP,
of course.) Whereas the RESTful believe that a remote service should
look like the Web. (Whatever that means.)
But I'm not gonna re-litigate that argument. "Web Service" is dead,
we're stuck with "API", and the name of the book needs to change.
"RESTful Web APIs" is a direct translation of "RESTful Web Services"
into 2013-speak. It keeps continuity with the old book, but it's a new
title for a new book.
Interested?
If you want to be a beta reader, let me know (via comment or email to leonardr@segfault.org). I'll get in touch with you around the end of March, or possibly earlier. Update: We've got a lot of beta readers and we're no longer accepting applications.
Once in a while I add a section to a chapter called "The Hypermedia Zoo", which takes a not-too-detailed look at a lot of hypermedia-aware data formats to demonstrate their diversity. I'm always on a low-level search for more hypermedia formats, and if you name your favorites in the comments I'll take a look. Don't worry about duplicating my list; I also want to see what springs to mind when you think "hypermedia format".
There's also a shorter section called "The Semantic Zoo" which talks about interesting domain-specific data standards. I'm mostly interested in reusable profiles (the IANA registry of link relations, microformats, schema.org microdata, Dublin Core), but I'm also interested in pointers to standalone data formats like Activity Streams. There's a whole lot of stuff out there and I'm not confident that I've seen even half of it.