News You Can Bruise for 2006 November 6

Mon Nov 06 2006 11:30 open-uri ought to be a REST client: I spent a lot of time and used a lot of pages in the Ruby Cookbook showing how to use Net:HTTP to build more and more capable HTTP clients that hid the hairiness of Net:HTTP. But in truth the open-uri library, which we only covered a little, supports almost every feature you could want from an HTTP client library: custom headers, proxies, transparent HTTPS with cert validation, even automatic handling of the redirects it's okay to handle automatically. And the interface couldn't be simpler: it overloads Kernel#open so that you can open an HTTP request like a file, and read the response like data from a filehandle.

I'd like to recommend open-uri for everything, including as a client for REST web services, but there are two big features it's missing. You can't use any HTTP method other than GET, and you can't send any body data (which makes sense if GET is the only supported method). Lately I've been building on my Cookbook work, coming closer and closer towards a general HTTP client library for Ruby, but really all I'm doing is reinventing open-uri with two extra features. It would be a lot better to just add those features to open-uri.

I submitted a patch for open-uri on this topic a while back but have yet to get any response, even from people saying it's stupid. I'm writing REST clients with open-uri under the assumption that that'll work by the time the book is published. For instance, here's some code from my Amazon S3 client:

  # Saves this bucket to S3.
  def save
    open(uri, :method => :put)
    return self
  end

  # Deletes this bucket and all its contents.
  def delete
    open(uri, :method => :delete)
  end

I guess you could argue that these features break the metaphor, that PUTting to a URI isn't really "opening" it, or that it's like opening a file for write access and you should have to write the body data inside a code block the way you do for a file.[0] But c'mon, let's have that argument. I don't want to have to keep telling everyone about fake reimplementations of open-uri just so they can write REST clients in Ruby. There's also precedent, like XMLHttpRequest.open in Javascript.

I'd also like to add caching support to open-uri, like the caching support in Joe Gregorio's httplib2 for Python. But I'll wait on that until the fate of this simpler patch is decided.

[0] I doubt this is really the intent of open-uri, since right now it doesn't support HEAD either. I think the intent of open-uri is to make it trivial to make an HTTP request from Ruby.

(2) Mon Nov 06 2006 21:21 Nomenclature: Some good pushback on the Venn diagram and ROA/SOA dichotomy, primarily here at Labnotes. But also in some emails saying what amounts to: "Why are you putting 'services' in the name of your book if you then go and classify REST outside the Service-Oriented Architecture, today's hot buzzword that has 'service' in its very name? Why are you even saying 'web services' at all if you think REST is different from anything service-oriented?"

Good points, and highlighting an increasing source of uneasiness on my part. Which is good, despite the fact I generally don't like uneasiness, because it shows it's paying off to embarrass myself by formulating this book in public. People are interested enough in the topic to form uncoordinated revolts around ideas that are wrong or need work.

First a disclaimer about how much this matters: not a whole lot. This is a book about REST and the ROA. The name of the ROA might change yet for marketing reasons, but we've got its technical features pretty well defined. There is a chapter (#1) that covers the programmable web as a whole, a chapter (#3) whose first half covers services popularly considered RESTful that aren't really, and a chapter (#8) that compares REST/ROA to "the other guys".

I was calling "the other guys" "SOA" but that's starting to seem wrong. My current wild hare is to call them "Message-Oriented Architectures", though that has a bit of a collision with Message-Oriented Middleware. That way "Service-Oriented Architecture" is freed up to describe a higher-level architecture based on providing and consuming web services (ROA or MOA or both). Which seems closer to existing usage of "SOA", insofar as "SOA" means anything besides "your new issue of BusinessWeek is here."

Point is, our taxonomy could change completely and we'd only have to make real changes to those three chapters (and #8 isn't even started yet, partially because we're trying to sort this stuff out first). More important, much more detailed, and hopefully less contentious, will be our description of what goes on inside the "REST" part of the Venn diagram.

Anyway, our task here is the Confucian task of the rectification of names. We're trying to classify the set of techniques and technologies used on the programmable web. If it turns out to be best described with gradients instead of subsets, it might be a 2D or 3D map instead of a Venn diagram or taxonomy.

This doesn't have to be like the Linnean classification system where everyone signs onto it for all time, but it should be done with respect for the way "the other guys" see themselves. Sometimes in the text of the book we'll poke fun at what I call Big Web Services⁰. But we need to have a credible story about what Big Web Services really are, how they differ from RESTful web services, and what lives in between. Ideally someone who disagrees might say "Well, I wouldn't call it that, but I see how you draw the distinction."

So here are some unsolved mysteries of our taxonomy:

I don't think anyone will dispute that there's at least one rival web service architecture to REST/ROA: the RPC architecture that shows up in many web applications, in web services like Flickr's, and in XML-RPC. Can we consider this architecture as similar to the message processing architecture that rarely shows up in web applications, and that is commonly associated with SOAP? That is, granted there are differences, do they matter for a book about REST, or do they both differ from REST in a way we can capture in some snappy term like MOA?

Are all these architectures mutual opposites, or points on a gradient?

It it useful to distinguish between the typical robotic SOAP/WSDL interfaces autogenerated from Java or .NET code at the touch of a button, and the in potentia SOAP architecture that can be anything, even RESTful?

Consider the services popularly considered RESTful but seriously non-thesis-compliant. We currently term those things "HTTP+POX" in deference to common (kind of common) usage. Is that just another case of conflating architecture and technology? Is it worth coming up with another term? And given that there are multiple rival architectures for web services, do these not-quite-RESTful services consistently deviate from REST in the direction of some other architecture, or are they just freakish one-off mutants? Are they the missing links on a gradient between the rival architectures?

⁰ Said like in one of those political ads with lots of black-and-white closeups: "Senator Bedfellow doesn't want you to know about his millions in contributions from Big Web Services. When the FBI came knocking, he started shredding the XML documents. But he forgot to shred the envelopes. Senator Bedfellow: wrong on URIs. Wrong for the web."