News You Can Bruise

Beautiful Soup 4.2.0

Tue, 14 May 2013 14:04:17 GMT

My work on RESTful Web APIs is pretty much done, so I went through the Beautiful Soup bug tracker and fixed everything I could. The result is a new, stoner-iffic release of Beautiful Soup.

Here are the release notes. The main new features are a much more capable CSS selector engine, and a diagnostics module that should help with tech support.

Story Bundle

Wed, 01 May 2013 18:55:56 GMT

Constellation Games is featured in the current video game-themed StoryBundle. It's a pay-what-you-want, like the Humble Indie Bundle. This means that if you're the ultimate cheapskate, you can get my book and six others for the Steam-sale-level price of three bucks. Pay ten bucks, and you also get three bonus books, including Jordan Mechner's "The Making of Prince of Persia and a Ralph Baer memoir which--just guessing here--is probably enjoyably cranky.

And for people who discover Constellation Games based on this bundle, this is my occasional notification that there are tons of free extras: four bonus stories, in-character Twitter feeds, and an episode guide with commentary.

Side note: the bundle was assembled by Simon Carless, who is the reason I wrote Constellation Games in the first place.

April Film Roundup

Wed, 01 May 2013 14:59:00 GMT

Another month, another few movies. RESTful Web APIs is almost done, but not quite, so once again there's not a whole lot here. The theme of this month is "really loving a movie, seeing a different movie on that basis, and being very disappointed."

The Face You Deserve (2004): This movie caused a rare Siskel and Ebert-type rift between me and Sumana. I thought the first part of the movie was boring, but that it picked up once it turned into a surreal fairy tale. Sumana thought the first part showed promise and hated the Michel Gondry-esque manchildren in the fairy tale part. I don't recommend this movie, either, but I was engaged for the fairy tale.

The General (1926): This is one of the best movies I've ever seen. It's so well put together. The movie is basically two chase scenes, and each chase scene is made entirely of inventive Jackie Chan-style action gags. People in the theater were cheering, which I've never experienced before, and laughing to an extent not heard since The Whole Town's Talking. The General has all the good things Chaplin put into his films, but none of the treacly sentimentality. The one bit of sentimentality is deflated by its co-occurance with the one bit of corny dated-looking special effects.
No surprise, then, that this was Keaton's Ishtar: a way-too-expensive flop that cost him his creative control. You can Watch The General on the Internet Archive, but as always with silent film the problem is finding an appropriate soundtrack. We heard an amazing live soundtrack performed by Viola Dana, and they have a CD available, but the CD only has "selections". So maybe try Stravinsky's "Chamber Works", as suggested by a comment on this page? I bet some peppy chiptunes would also work.
Die Hard (1988): Not one of the best movies I've ever seen. It deserves a lot of credit as the pinnacle of the 80s action movie, but at this point I've seen some action movies from the 70s, and it feels like movie execs saw Die Hard and said "well, we found it!" and the genre never advanced again. Not really Die Hard's fault, but it's hard not to be bitter.
For improvisational comedy-violence, The General is better. Not just my idiosyncratic opinion! The General's IMDB rating is 8.4, versus Die Hard's 8.3, and at the high end of the distribution, 0.1 IMDB star is worth a lot.
Steamboat Bill, Jr. (1928): A big disappointment after The General. Buster Keaton has lost his creative control, and it shows. The film lacks a through-line (unlike The General, which was literally on rails), and promises a "snobs vs. slobs" rivalry that never gets going. Partially redeemed by great stunts. This, too, can be seen on the Internet Archive.
Tai Chi Zero (2012): Zany anti-colonialist steampunk kung fu movie that annotates events with video game-style infographics and otherwise breaks the fourth wall all the time. It's kind of China's Scott Pilgrim Versus the World, and judging from online reviews it's just as divisive. We liked it a lot. (I also liked Scott Pilgrim.) It's got big problems, notably the acting, which is very stiff. But most of the actors were chosen for their martial arts ability, and martial arts are happening about seventy percent of the time. The only time the fourth-wall-breaking got out of control was a scene at the beginning of the third act, which blends together "we're planning this heist" shots and hypothetical "this is what it will look like when you carry out the heist" shots, and then starts mixing in "this is how the heist actually went down" shots! It took about thirty seconds before I realized that I was now watching the actual heist.
Sometimes the problems made me enjoy the movie more! The awkward English scenes gave me an experience similar to what I imagine a Mandarin speaker feels watching a Mandarin scene in an American movie. There's a steampunk tank with an English instruction manual, which was supposedly written by Brits but which reads exactly like the instruction manuals that come with Chinese-manufactured kitchen appliances. I thought the villain was a more complex character than he actually was, because I assumed that if everyone derides a character as a wimp, that makes him the underdog and you're supposed to have some sympathy for him. But no, apparently not in this movie.
Oh, and you know how they say "there ought to be a law?" Well, the special effects supervisor for Tai Chi Zero is credited as "A Law". So now there is A Law!
But I gotta tell you that this is not a standalone movie. It could have been, but about three minutes from the end it turns into The Good, the Bad, and the Ugly, introduces a whole bunch of new characters, sets up a sequel and leaves you hanging.
Tai Chi Hero (2012): Fortunately, the sequel is playing in Times Square right now, so we went and saw it the next day. Aaaand... we were very disappointed. If you liked all the steampunk and fourth-wall-breaking from the first movie, then too bad, because there's no steampunk until the second act and no fourth-wall-breaking until the third. (The steampunk, when it finally happens, is still great.) On the other hand, if you hated all that nerd shit from the first movie, you'll love the by-the-numbers soap opera they replaced it with.
On top of everything else, the title of this movie retroactively makes the title of the previous movie dumb. I can't believe they got Peter Stormare to... wait, he was in Armageddon, never mind.

Board Game Dadaist Improvements

Sat, 27 Apr 2013 21:08:32 GMT

I've finally relented to Adam's demands and made some improvements to the Board Game Dadaist RSS feed. He broke his kneecap recently and I figured this would be a good way to cheer him up. Every game that shows up in the feed now has a permalink (here's "Plue"), and that page has a very basic link for posting your find to Twitter.

In Search of the Beautiful Soup Double-Dippers

Thu, 18 Apr 2013 17:05:17 GMT

Recently I noticed that certain IPs were using distribute or setuptools to download the Beautiful Soup tarball multiple times in a row. For one thing, I'm not sure why distribute and setuptools are downloading Beautiful Soup from crummy.com instead of using PyPI, especially since PyPI registers almost 150k downloads of the latest BS4--why are some people using PyPI and not others?

If anyone knows how to convince everyone to use PyPI, I'd appreciate the knowledge. But it's not a big deal right now, and it gives me some visibility into how people are using Beautiful Soup. Visibility which I will share with you.

Yesterday, the 17th, the Beautiful Soup 4.1.3 tarball was downloaded 2223 times. It is by far the most popular thing on crummy.com. The second most popular thing is the Beautiful Soup 3.2.1 tarball, which was downloaded 381 times. The vast majority of the downloads were from installation scripts: distribute or setuptools.

1516 distinct IP addresses were responsible for the 2223 downloads of 4.1.3. I wrote a script to find out how many IP addresses downloaded Beautiful Soup more than once. The results:

Downloads from a single IP	Number of times this happened
55	1
35	1
15	1
13	1
11	1
5	2
4	12
3	43
2	453
1	1001

Naturally my attention was drawn to the outliers at the top of the table. I investigated them individually. The IP address responsible for 55 downloads is a software company of the sort that might be deploying to a bunch of computers behind a proxy. The 35 is an individual on a cable modem who, judging from their other traces on the Internet, is deploying to a bunch of computers using Puppet. The 15, the 13, and the 11 are all from Travis CI, a continuous integration service.

One of the two 5s was an Amazon EC2 instance. Five of the twelve 4s were Amazon EC2 instances. Thirty-seven of the forty-three 3s were Amazon EC2 instances. And 395 of the 453 double-dippers were Amazon EC2 instances. Something's clearly going on with EC2. (There was also one download from within Amazon corporate, among other BigCo downloaders.)

I hypothesized that the overall majority of duplicate requests are from Amazon EC2 instances being wiped and redeployed. To test this hypothesis I went through all the double-dippers and calculated the time between the first request and the second. My results are in this scatter plot. Each point on the plot represents an IP address that downloaded Beautiful Soup twice yesterday.

For EC2 instances, the median time between requests is 11 hours and 45 minutes. So EC2 instances are being automatically redeployed twice a day. For non-EC2 instances, the median time between requests is 51 minutes, and the modal time is about zero. Those people set up a dev environment, discover that something doesn't work, and try it again from scratch.

March Film Roundup

Mon, 01 Apr 2013 13:25:24 GMT

Okay, look. I don't see movies just for their entertainment value. I dig film as an art form. But my permit to dig is premised on an amateur understanding of film as a narrative art form. If you want to present an endless stream of disconnected images, let's do an installation piece, because I want to decide for myself when I've had enough. I'm not going to be your captive for fifty minutes. (I'm looking at you, Andy Warhol.) And all that aside, I'm not gonna see a movie called Trash Humpers (2009), when the nicest thing the folks doing the screening can say is that it "rewards the open-minded viewer with moments of astonishing and unexpected poignancy."

Which is to say that I skipped most of the museum's highly avant-garde March offerings. I also got this book I have to work on. So not many movies in this roundup. Let's-a go:

Fallen Angel (1945): Decent noir with a fake mystery and an interesting twist at the end (in terms of which characters got what they wanted and how, not in terms of plot). John Carradine appeared as a classic comedic noir conman, but he had to appear in twenty other movies, so he left after the first reel, much to Fallen Angel's detriment.
Turns out a noir film is my popcorn movie. I'll go see any number of them but I'm not expecting great things from them. PS: there is no popcorn allowed in the museum theater.
Horse Feathers (1932): Still really funny, but this is the first viewing where I noticed that the Marx Brothers' general disrespect for society encompasses a lot of misogny. It's not just Harpo chasing the choir girls. In fact, Groucho's the worst. It doesn't help that there's no Margaret Dumont here to take up the flag of society and fight back. But Chico filling bottles in the speakeasy will never get old.
Ikiru (1952): Watched on Hulu during the free Kurosawa weekend. Highly recommended. A little heavy-handed at the beginning, but it really started paying off when the main character died. (Not a spoiler.) At that point I saw a masterful display of one of the most difficult and most important things that fiction can bring to our attention: the mechanics by which we all construct narratives for our lives in which we're the good guy making good things happen.
Bonus: everyone referred to Takashi Shimura's character as "Kacho", deepening my belief that Game Center CX is a workplace satire in the vein of Ikiru.
Wreck-it Ralph (2012): I'm not sure who gave Disney the idea that it's okay to use other people's intellectual property in their movies, but it gives good results. Wreck-it Ralph is a by-the-numbers Disney narrative, but the fact that it's a movie about arcade games and their by-the-numbers narratives leaves quite a bit of room for subversion and criticism, in service of the larger goal of feel-good entertainment. As with Who Framed Roger Rabbit?, I thought the original characters were a lot more interesting than the famous cameos, to the extent that I continually wished the famous cameos would butt out and let the original characters get on with it.
(The worst cameo was Sonic the Hedgehog's infodumpy PSA near the beginning. Awful! But! What if it was a sly reference to those dumb PSAs at the end of the old Sonic cartoons? Does an obscure reference deserve respect even when deployed as a cheesy infodump? OH THE DILEMMA)
The museum showed this in 3D, and I was apprehensive about the extra D. I can report that it neither caused me headaches nor made me want to see all movies in 3D from this point on.
2 or 3 Things I Know About Her (1967): The French New Wave eludes me again. There's one great scene in this, in which the conspiratorially-whispering narrator (Godard himself) deconstructs the subject-object distinction to the extent that he loses the ability to make directorial decisions, and lets the camera linger on some trees for a while. There's a few other good bits, and lots of Ballardian imagery. Makes me want to watch Alphaville even more. But... eh. Eh, I say!

Tue, 26 Mar 2013 12:00:13 GMT

From an interview with Ken Liu, recent Hugo/Nebula/WFA winner:

I went to law school, started a new job, and kind of gave up on writing for a while due to a supreme act of stupidity. I wrote this one story that I really loved, but no one would buy it. Instead of writing more stories and subbing them, as those wiser than I was would have told me, I obsessively revised it and sent it back out, over and over, until I eventually gave up, concluding that I was never going to be published again.
And then, in 2009, Sumana Harihareswara and Leonard Richardson bought that story, "Single-Bit Error," for their anthology, Thoughtcrime Experiments. The premise of the anthology was, in the editors' words, "to find mind-breakingly good science fiction/fantasy stories that other editors had rejected, and release them into the commons for readers to enjoy."
I can't tell you how much that sale meant to me. The fact that someone liked that story after years of rejections made me realize that I just had to find the one editor, the one reader who got my story, and it was enough. Instead of trying to divine what some mythical ur-editor or "the market" wanted, I felt free, after that experience, to just try to tell stories that I wanted to see told and not worry so much about selling or not selling. I got back into writing—and amazingly, my stories began to sell.

Case closed, I'd say.

February Film Roundup

Mon, 04 Mar 2013 18:38:07 GMT

The second in the 2013 series, as promised. Note: I draw no distinction between information about a movie that's a "spoiler" and information that's not.

Roots (1977): I'd never seen Roots. We watched it so we could reasonably go to a reunion interview at the museum featuring Levar Burton, Louis Gossett Jr., Leslie Uggams, and Ben Vereen. Here's Sumana's take. Non-surprise: Roots is really good! I have only one complaint: it was disconcerting to see Levar Burton abruptly replaced by John Amos in a "nine years later" flash-forward, since we now know that age cannot change the essential Levar Burton-ness of Levar Burton.
My other complaint was going to be that some (not all) of the villains were one-dimensional villains defined entirely by their hatred for the main characters. I'm actually okay with this in certain kinds of pieces, and Roots definitely qualifies, so my criticism was going to be somewhat muted. But then something magical happened that cancelled my criticism altogether.
You see, in a decision that probably made sense to whichever ABC exec was trying to backpedal from having greenlit Roots in the first place, it was decided to cast "television's most likeable white actors" in the roles of the villains, to sort of tone it down a little. At least that's what it said in the little mini-documentary on Roots that they showed before the reunion interview. If an actor's natural "likeability" tones down the evil of their character, that means they're not a very good actor, so it's a good thing that the whole "likeability" thing was a bust.
But then! The most despicable villain of the entire show, the final boss of Roots, the very face of evil, was played by genial, honey-voiced Burl Ives. Amazing! It was like watching Burl Ives play Saruman. I don't think that's what they were going for, but it was great. And I can confirm that Roots as a whole deserves all the praise given to it over the years. Ben Vereen, in particular, is amazing.
Killer of Sheep (1979): A sad movie about a man so beat-down by the grim meathook future that he can't appreciate the odd moments of grace when they pop up. It's always been tough for me to stay engaged in a movie that has no through-line, but after the first half of Celine and Julie Go Boating I've learned to treat it as the dramatic equivalent of sketch comedy. And although this movie is not a comedy, its brand of existential despair gives it something in common with sketch comedy, so it wasn't as much of a mismatch as you might think.
Metropolis (1927): The first of several movies we saw on Hulu when they made their Criterion movies free to watch over the course of a weekend. I'd seen it before; Sumana had not. My current opinion is an amplified version of my old, uninformed opinion. As a story, Metropolis is terrible, but if you treat it as an opera it's a pretty good opera, with lots of awesome stuff to look at. And the robot's wink is one of my favorite film shots ever.
Modern Times (1936): Sumana has zero tolerance for protagonists whose incompetence is supposed to be endearing. Since endearing incompetence is the Little Tramp character's stock in trade, I probably should have anticipated her reaction to Modern Times. I wouldn't say I have zero tolerance for such protagonists, but I don't have a lot, and it really dampened the mood of the movie. Bright moments include the feeding-machine scene, the Tramp getting high on coke, and the one line of silent-film dialogue that redeems his incompetence (paraphrased): "I'll do whatever it takes to get you a home, even if I have to work for it!"
Diabolique (1955): The cream of Criterion Weekend, an exciting thriller that Hitchcock wanted to direct but Henri-Georges Clouzot got there first. So the authors of the novel on which Diabolique was based wrote another novel with a similar twist but which was probably a lot worse, because that novel became Vertigo. I guess back then you couldn't just write a screenplay, you had to try it out as a novel first.
While watching this movie we noticed that it's effectively a Columbo episode. A little while later we watched the movie that serves as the pilot episode of Columbo, and there were quite a few similarities to Diabolique! Coincidence? I really don't know. The play that introduced the Columbo character premiered in 1960.
Bonus: Hulu kept interrupting the movies with commercials, creating the bizarre experience of watching subtitled French films on a local TV station in 1993. They've clearly got an algorithm for determining how many commercials they can cram in before people stop watching, because near the end of Diabolique the commercial breaks started coming once every six minutes. It's so suspenseful, they know no one's going to tune out.
It was the same three or four commercials the whole weekend, and one of the commercials included the perfect iambic-tetrameter line "In every segment we compete," which we mashed up with Sydney Smith's Answer to an Invitation to Dine at Fishmongers Hall.
In every segment we compete
The monsters of the deep to eat
Shoot the Piano Player (1960): Hulu called this "Truffault's most playful film", and I misinterpreted this statement as implying that the movie would be a comedy. It is not a comedy, but it's not bad. I dunno about this whole French New Wave thing, though. I just don't know.
I Married A Witch (1942): This was a comedy, and it was terrible. You know how sometimes a movie gets into Criterion more because it's representative of a genre than because it's good? I suspect that happened here. This is the perfect "stupid black-and-white non-musical comedy." After a decent opening it went downhill fast, and by mutual agreement Sumana and I skipped the middle 45 minutes of this 77-minute movie. Fortunately, this movie ends with a witch stealing a gubernatorial election through brainwashing and magical vote fraud! Stupendous! But please don't take that as an endorsement of the movie as a whole. I Married A Witch has an IMDB rating of 7.1, a rating that rightfully belongs to Ishtar.
Foreign Correspondent (1940): Hitchcock's propaganda thriller. I was definitely caught up in it but I don't have a lot to say about it in the cold light of day. Some of the twists were not that surprising, others were good examples of Hitchcock thinking outside the cinematic box. E.g. most of the main characters get on a passenger plane to America and you're in the middle of some piddling drama that pits Group A against Group B, but then the real twist happens which is the plane is shot down by a German submarine!
The Great Dictator (1940): I rewatched this after Sumana went to sleep, in an effort to get one more free Criterion movie out of Hulu. I remember really liking The Great Dictator and I wanted to bring my recently honed film-watching skills to bear on it. And it's... uneven. Chaplin's tramp-like character is as genially incompetent as ever, and his author-mouthpiece speech at the end, which expresses a lot of nice sentiments, makes no sense in terms of plot and does nothing to fulfil the incredibly tense dramatic situation that the rest of the movie has been building up. (Compare the speech at the end of Foreign Correspondent, in which Hitchcock puts away all his cinematic tricks and gimmicks and simply begs the American audience for help.) So I guess I have zero tolerance for sappy melodrama in comedies, especially comedies about horrible wars.
But when Chaplin plays Hitler it's amazing. This is the portrayal I can imagine getting under the dictator's skin. "Adenoid Hynkel" is a petty, insecure, puffed-up, blustering asshole, the opposite of Chaplin's tramp, a man whose legendary incompetence threatens to ruin the world. You can tell that Chaplin wasn't aware of how evil Hitler really was (or else you can understand why he claimed he hadn't been aware). Even though this is the most scathing satirical portrayal of Hitler I've ever seen--and well-timed to boot--you'd always feel bad for even deploying satire as a weapon instead of buying more government bonds.
Emma Mae (1976): Kind of an unintentional pun on "MMA" there. Director Jamaa Fanaka tried to make a funny action film for a black audience, only to see it dubbed "blaxploitation" and released on DVD as "Black Sister's Revenge". Well, it's a very fun movie, but if you're expecting "Black Sister's Revenge" you're gonna be disappointed.
IMDB trivia says Fanaka was a fan of Billy Wilder, and it shows in this story of a country girl who comes to L.A. to live with her aunt and uncle. The plot, the action and the comedy are all driven by Emma Mae's tendency to take the most direct approach to any problem. A guy calls Emma Mae a hick, so she hits him. Her boyfriend gets arrested, so she starts a car wash to raise money to pay his bail. Someone else thinks a car wash is a dumb idea, so Emma Mae hits her. The Man shuts down her car wash, so Emma Mae robs a bank. And so on.
Jerri Hayes, who played Emma Mae, came to the showing and said that before Fanaka died last year, they'd been talking about doing a sequel to Emma Mae. That would have been really fun to see.
J.S.A.: Joint Security Area (2000): March bonus! I don't have the stomach for Oldboy, so I'm skipping most of the Chan-wook Park festival, but I figured I could handle a thriller about the Korean DMZ. I mean, the worst that can happen is nuclear war. And it was pretty good! I was analyzing the movie while watching it (occupational hazard) and thinking how its use of symbolism and callbacks was corny and heavy-handed, but then the very last shot of the movie tied together two earlier scenes, which initially seemed to be nothing but comic relief, into the movie as a whole. Tied it all together in a way that made those scenes transcend comic relief to create something moving. The very last shot changed my opinion of the movie from "pretty good" to "very good". I didn't think that stuff happened in real life.
My attempt to explain this experience to Sumana revealed that this is very much a "you had to be there" thing, but if you have the chance to be there, I think you should take it. I'm not gonna say there's no disturbing violence in this movie, but it's no Oldboy.

Ragtime Synchronicity

Thu, 28 Feb 2013 13:49:33 GMT

"Bugs," said Krakowski. "In-tell-i-gence gathering devices. The Constellation loves recording things. Now they're going to record every conversation anyone ever has."
"I think you might be projecting a little."

100 Years of Markov Chains

Fri, 22 Feb 2013 14:44:01 GMT

Back in January I took a little trip to Boston and hung out with Kirk. Among other things, we attended an event at Harvard celebrating the 100th anniversary of the paper that kicked off the Markov chain craze. I only wish Adam had been there. I've held off on talking about the event because I've been waiting for Harvard to put the video of the talks online. But that's a sucker's game, and now I have something better!

See, the first talk, by Brian Hayes, covered the amazing history leading up to the publication of Markov's seminal paper. He's now turned his talk into an article in American Scientist. The first few pages of that article are a basic introduction to Markov chains; the history starts on page four. Basically, Markov was a cranky old man who liked picking fights.

Markov’s pugnacity extended beyond mathematics to politics and public life. When the Russian church excommunicated Leo Tolstoy, Markov asked that he be expelled also. (The request was granted.) In 1902, the leftist writer Maxim Gorky was elected to the Academy, but the election was vetoed by Tsar Nicholas II. In protest, Markov announced that he would refuse all future honors from the tsar... In 1913, when the tsar called for celebrations of 300 years of Romanov rule, Markov responded by organizing a symposium commemorating a different anniversary: the publication of Ars Conjectandi 200 years before.

As acts of political protest go, the well-timed symposium is pretty great. At that symposium Markov revealed the Markov chain, which he'd invented as a way to smack down the dumb theological arguments of rival mathematician Pavel Nekrasov. His paper wasn't called "Markov Chains: Future Basis for Art and Scientific Discovery, Named After Me, A. A. Markov." It was called called "An Example of Statistical Investigation of the Text 'Eugene Onegin' Concerning the Connection of Samples in Chains".

Markov had manually gone through the first 20,000 characters of Pushkin's "Eugene Onegin", looking at every pair of letters, writing down whether the letters were both vowels, both consonants, vowel-consonant, or consonant-vowel. Then he'd modelled the transitions between those four states with a Markov chain. The result disproved an assumption about the law of large numbers, an assumption crucial to Nekrasov's mathematical argument for free will. There's something about this mindset that always gets me--inventing the sledgehammer so you can use it to kill a fly.

The other two talks were a lot more technical. I was mostly able to follow them, but I don't think I got much out of them. Here's a summary of all three talks from someone else who was there. But I strongly recommend Hayes's article to anyone who reads this weblog.

Welcome BoingBoing Readers

Wed, 20 Feb 2013 17:14:20 GMT

If you're coming here from Cory Doctorow's review of Constellation Games, you might like to know about my web page for the book. The book was originally a serial, and I wrote chapter-by-chapter commentary as it was serialized. I also wrote four bonus stories set before, during, and after the novel, which I've released under CC-BY-SA. All that stuff is here.

You might also be interested to know that you get a DRM-free PDF version of the novel by buying direct from the publisher.

Now would also be a great time to mention that Constellation Games is eligible for this year's Hugo.

Fundamental Indeed

Thu, 14 Feb 2013 17:04:54 GMT

I could spend all day just posting games that Board Game Dadaist comes up with. I forbear, for the sake of you, my readers, but Adam Parrish and I will email each other when we find an especially good one. And I think you should know about the best game BGD ever came up with (found by Adam back in December):

Fantasy Fundamental Rails (2005)
Players divide themselves into two teams.

What's New in "RESTful Web APIs"

Tue, 12 Feb 2013 18:43:50 GMT

We're ahead of schedule, which is good because we have a lot of work to do that isn't part of the book manuscript. Yesterday I sent out over forty copies of the manuscript to beta readers. That is too many beta readers, so at this point I must refuse anyone else who wants to be part of the beta, unless they have/had a hand in one of the standards we discuss, and they want to specifically critique our coverage of that standard.

With the beta closed I think it's a good time to go into a little detail about the structure of the book. My guiding principle was to write a book that will be as useful now as RESTful Web Services was in 2007. Like RWS, RESTful Web APIs has a main storyline that takes up most of the book. My inspiration for the main storyline were a few books that followed RWS, notably REST in Practice and Mike's Building Hypermedia APIs with HTML5 and Node.

RWS focused on the HTTP notion of a "resource", and despite the copious client-side code, this put the conceptual focus clearly on the server side, where the resource implementations live. RWA focuses on representations, and thus on hypermedia, on the interaction between client and server, which is where REST lives. The stuff you remember from RWS is still here, albeit rewritten in a pedagogically superior way. Web APIs work on the same principles as the Web, here's how HTTP works, here's what the Fielding constraints do, and so on. But the focus is always on the interaction, on the client and server manipulating each others' state by sending messages back and forth.

We've also benefited from a lot of tech work done by others. The IANA registry of link relations showed that state transitions don't have to be tied to a media type. The RFC that established that registry also showed how to define custom state transitions (extension relation types) without defining yet another media type to hold them.

Insights like these inform the new parts of RWA's main storyline. What makes your API different from every other RESTful API in existence? That's the only part you really need to buckle down and design. Everything else you can reuse, or at least copy.

In particular, you shouldn't have to design a custom media type. Your API probably isn't that different from other APIs, and a ton of hypermedia formats and protocols have been invented since 2007. We cover a few of the most promising ones in the book's main storyline. We cover even more of them afterwards, mostly in the big "Hypermedia Zoo" chapter. Here's the book-wide list:

Maze+XML
Collection+JSON
AtomPub
OpenSearch
HTML
HAL
Siren
JSON-LD
The HTTP headers Link, Link-Template, Location, and Content-Location
URL lists
JSON Home Documents
Problem detail documents
Hydra
OData
WADL
Web Intents
XLink
XForms
SVG
VoiceXML
GeoJSON (which is only a hypermedia format on a bizarre technicality!)
RDF
XRD/JRD
Web host metadata documents
WebFinger
CoRE Link Format

After the main storyline and the hypermedia zoo, RWA continues the RWS tradition of giving an API-centric view of the HTTP standard. We have a "crash course in advanced HTTP" chapter, some of which is an update of Chapter 8 from RWS. (Look-before-you-leap requests never caught on, but I still feel like I have to describe them in RWA because I have no other source to refer you to!) Appendix A is an updated version of Appendix B from RWS, with the addition of these exciting new status codes:

308 ("Permanent Redirect")
428 ("Precondition Required")
429 ("Too Many Requests")
431 ("Request Header Fields Too Large")
451 ("Unavailable For Legal Reasons")
511 ("Network Authentication Required")

Appendix B is an update of appendix C from RWS, with these API-licious new HTTP headers:

Link and Link-Template
Prefer and Preference-Applied

The amount of reused material in RWA is really small, because the main storyline is completely rewritten for 2013. And I haven't even mentioned our coverage of profiles, partly because I can't yet think of a way to talk about profiles at less length than what we say in the book.

Hire Aaron DeVore

Tue, 12 Feb 2013 12:26:11 GMT

I don't often use the NYCB bully pulpit to tell you to hire someone (apart from myself), but folks, you should hire Aaron DeVore. He was effectively the maintainer of Beautiful Soup during the period when I wasn't working on it. He answered tons of questions on the mailing list and sent me bugfix patches. When I started work on Beautiful Soup 4, he gave me a lot of feedback that helped stabilize the API.

Aaron did all this while a college student in Portland, Oregon. Now he's about to graduate, and he's looking for a job. Send him an email and let him know what you've got going on.

Constellation Games Interview in Bookslut

Thu, 07 Feb 2013 17:14:59 GMT

Hey folks, CG fan Jeanne Thornton interviewed me a couple months back, creating a text that has now been published on Bookslut. (There's also an interview with Saladin Ahmed in the same issue.) The interview ranges over the CG publication process, games as an art form, space exploration, and so on.

One thing the published interview doesn't include is a question about Tetsuo Milk, which Jeanne cut before submitting the interview because it was kind of inside-baseball. But hey, inside baseball is the whole point of News You Can Bruise, so with Jeanne's permission I've reproduced the original question and my answer here:

I would feel remiss in not asking you about Tetsuo Milk, a character whom you’ve said (in your really, really mind-blowingly extensive commentary on the novel) essentially ran away with the book. Tetsuo is a brilliant character, but also feels at times like a heterogeneous element. I like this effect a lot, but I’m curious as to where this guy came from, what you’re saying through him, and how you see him fitting into the overall mix.
Maybe this will help: Tetsuo Milk is the ET version of Ariel. His silly mistakes and misunderstandings are mirror images of the mistakes Ariel makes trying to understand the Constellation. We don't laugh because we're not the ones being misunderstood. When Tetsuo does it to us, it's funny.
Here's a spoiler-free example. One of Ariel's post-contact hobbies is posting reviews of alien computer games to his blog. There's one really important scene that reverses the roles: Tetsuo writes a review of a game Ariel worked on as a developer, Brilhantes Poneis 5. Brilhantes is a stupid Farmville-type mobile game where you have a pet pony and do pointless tasks to earn coins to buy accessories for it. Tetsuo tackles the game from a post-scarcity Marxist perspective, putting a lot of work into understanding how a game's economy can work when the player is the employee of an animal. He gets a lot of it right (i.e. he recognizes that the game demeans both its players and its developers), but he's operating from completely the wrong framework.
That's the kind of mistake Ariel makes. He brings his human assumptions to everything, whether he realizes it or not, whether or not Tetsuo or someone else calls him on it.

(This is why there's a reference to "Tetsuo-like ideas" later in the interview; we shoulda cut that reference.)