Wed Dec 04 2013 14:55 Secrets of (peoples' responses to) @horse_ebooks—revealed!:
As part of my @pony_strategies project (see previous post), I grabbed the 3200 most recent @horse_ebooks tweets via the Twitter API, and ran them through some simple analysis scripts to figure out how they were made and which linguistic features separated the popular ones from the unpopular.
This let me prove one of my hypotheses about the secret to _ebooks style comedy gold. I also disproved one of my hypotheses re: comedy gold, and came up with an improved hypotheses that works much better. Using these as heuristics I was able to make @pony_strategies come up with more of what humans consider the good stuff.
The timing of @horse_ebooks posts formed a normal distribution with mean of 3 hours and a standard deviation of 1 hour. Looking at ads alone, the situation was similar: a normal distribution with mean of 15 hours and standard deviation of 2 hours. This is pretty impressive consistency since Jacob Bakkila says he was posting @horse_ebooks tweets by hand. (No wonder he wanted to stop it!)
My setup is much different: I wrote a cheap scheduler that approximates a normal distribution and runs every fifteen minutes to see if it's time to post something.
Beyond this point, my analysis excludes the ads and focuses exclusively on the quotes. Nobody actually liked the ads.
The median length of a @horse_ebooks quote is 50 characters. Quotes shorter than the median were significantly more popular, but very long quotes were also more popular than quotes in the middle of the distribution.
I think that title case quotes (e.g. "Demand Furniture") are funnier than others. Does the public agree? For each quote, I checked whether the last word of the quote was capitalized.
43% of @horse_ebooks quotes end with a capitalized word. The median number of retweets for those quotes was 310, versus 235 for quotes with an uncapitalized last word. The public agrees with me. Title-case tweets are a little less common, but significantly more popular.
Since the last word of a joke is the most important, I decided to take a more detailed look each quote's last word. My favorite @horse_ebooks tweets are the ones that cut off in the middle of a sentence, so I anticipated that I would see a lot of quotes that ended with boring words like "the".
I applied part-of-speech tagging to the last word of each quote and grouped them together. Nouns were the most common by far, followed by verb of various kinds, determiners ("the", "this", "neither"), adjectives and adverbs.
I then sorted the list of parts of speech by the median number of retweets a @horse_ebooks quote got if it ended with that part of speech. Nouns and verbs were not only the most common, they were the most popular. (Median retweets for any kind of noun was over 300; verbs ranged from 191 retweets to 295, depending on the tense of the verb.) Adjectives underperformed relative to their frequency, except for comparative adjectives like "more", which overperformed.
I was right in thinking that quotes ending with a determiner or other boring word were very common, but they were also incredibly unpopular. The most popular among these were quotes that repeated gibberish over and over, e.g. "ORONGLY DGAGREE DISAGREE NO G G NO G G G G G G NO G G NEIEHER AGREE NOR DGAGREE O O O no O O no O O no O O no neither neither neither". A quote like "of events get you the" did very poorly. (By late-era @horse_ebooks standards, anyway.)
It's funny when you interrupt a noun
I pondered the mystery of the unpopular quotes and came up with a new hypothesis. People don't like interrupted sentences per se; they like interrupted noun phrases. Specifically, they like it when a noun phrase is truncated to a normal noun. Here are a few @horse_ebooks quotes that were extremely popular:
- Don t worry if you are not computer
- Don t feel stupid and doomed forever just because you failed on a science
- You constantly misplace your house
- I have completely eliminated your meal
Clearly "computer", "science", "house", "and "meal" were originally modifying some other noun, but when the sentence was truncated they became standalone nouns. Therefore, humor.
How can I test my hypothesis without access to the original texts from which @horse_ebooks takes its quotes? I don't have any automatic way to distinguish a truncated noun phrase from an ordinary noun. But I can see how many of the @horse_ebooks quotes end with a complete noun phrase. Then I can compare how well a quote does if it ends with a noun phrase, versus a noun that's not part of a noun phrase.
About 4.5% of the total @horse_ebooks quotes end in complete noun phrases. This is comparable to what I saw in the data I generated for @pony_strategies. I compared the popularity of quotes that ended in complete noun phrases, versus quotes that ended in standalone nouns.
|Quote ends in ||Median number of retweets|
|Standalone noun ||330|
|Noun phrase ||260|
So a standalone noun does better than a noun phrase, which does better than a non-noun. This confirms my hypothesis that truncating a noun phrase makes a quote funnier when the truncated phrase is also a noun. But a quote that ends in a complete noun phrase will still be more popular than one that ends with anything other than a noun.
At the time I did this research, I had about 2.5 million potential quotes taken from the Project Gutenberg DVD. I was looking for ways to rank these quotes and whittle them down to, say, the top ten percent. I used the techniques that I mentioned in my previous post for this, but I also used quote length, capitalization, and punchword part-of-speech to rank the quotes. I also looked for quotes that ended in complete noun phrases, and if truncating the noun phrase left me with a noun, most of the time I would go ahead and truncate the phrase. (For variety's sake, I didn't do this all the time.)
This stuff is currently not in olipy; I ran my filters and raters on the much smaller dataset I'd acquired from the DVD. There's no reason why these things couldn't go into olipy as part of the
ebooks.py module, but it's going to be a while. I shouldn't be making bots at all; I have to finish Situation Normal.
Wed Dec 04 2013 09:14 @pony_strategies:
My new bot, @pony_strategies, is the most sophisticated one I've ever created. It is the @horse_ebooks spambot from the Constellation Games universe.
Unlike @horse_ebooks, @pony_strategies will not abruptly stop publishing fun stuff, or turn out to be a cheesy tie-in trying to get you interested in some other project. It is a cheesy tie-in to some other project (Constellation Games), but you go into the relationship knowing this fact, and the connection is very subtle.
When explaining this project to people as I worked on it, I was astounded that many of them didn't know what @horse_ebooks was. But that just proves I inhabit a bubble in which fakey software has outsized significance. So a brief introduction:
@horse_ebooks was a spambot created by a Russian named Alexei Kouznetsov. It posted Twitter ads for crappy ebooks, some of which (but not all, or even most) were about horses. Its major innovative feature was its text generation algorithm for the things it would say between ads.
Are you ready? The amazing algorithm was this: @horse_ebooks ripped strings more or less randomly from the crappy ebooks it was selling and presented them with absolutely no context.
Trust me, this is groundbreaking. I'm sure this technique had been tried before, but @horse_ebooks was the first to make it popular. And it's great! Truncating a sentence in the right place generates some pretty funny stuff. Here are four consecutive @horse_ebooks tweets:
- Not only that, but whether you believe it (or want to believe it) the car salesmen will continue to laugh
- Demand Furniture
- Including simplified four part arrangements for the novice student and
- Just look at everything that I am going
There was a tribute comic and everything.
I say @horse_ebooks "was" a spambot because in 2011 the Twitter account was acquired by two Americans, Jacob Bakkila and Thomas Bender, who took it over and started running it not to sell crappy ebooks, but to promote their Alternate Reality Game. This fact was revealed back in September 2013, and once the men behind the mask were revealed, @horse_ebooks stopped posting.
The whole conceit of @horse_ebooks was that there was no active creative process, just a dumb algorithm. But in reality
Bakkila was "impersonating" the original algorithm—most likely curating its output so that you only saw the good stuff. No one likes to be played for a sucker, and when the true purpose of @horse_ebooks was revealed, folks felt betrayed.
As it happens, the question of whether it's artistically valid to curate the output of an algorithm is a major bone of contention in the ongoing Vorticism/Futurism-esque feud between Adam Parrish and myself. He is dead set against it; I think it makes sense if you are using an algorithm as the input into another creative process, or if your sole object is to entertain. We both agree that it's a little sketchy if you have 200,000 fans whose fandom is predicated on the belief that they're reading the raw output of an algorithm. On the other hand, if you follow an ebook spammer on Twitter, you get up with fleas. I think that's how the saying goes.
In any event, the fan comics ceased when @horse_ebooks did. There was a lot of chin-stroking and art-denial and in general the reaction was strongly negative. But that's not the end of the story.
You see, the death of @horse_ebooks led to an outpouring of imitation *_ebooks bots on various topics. (This had been happening before, actually.) As these bots were announced, I swore silent vengeance on each and every one of them. Why? Because those bots didn't use the awesome @horse_ebooks algorithm! Most of them used Markov chains, that most hated technique, to generate their text. It was as if the @horse_ebooks algorithm itself had been discredited by the revelation that two guys from New York were manually curating its output. (Confused reports that those guys had "written" the @horse_ebooks tweets didn't help matters--they implied that there was no algorithm at all and that the text was original.)
But there was hope. A single bot escaped my pronouncements of vengeance: Adam's excellent @zzt_ebooks. That is a great bot which you should follow, and it uses an approximation of the real @horse_ebooks algorithm:
- The corpus is word-wrapped at 35 characters per line.
- Pick a line to use as the first part of a tweet.
- If (random), append the next line onto the current line.
- Repeat until (random) is false or the line is as large as a tweet can get.
And here are four consecutive quotes from @zzt_ebooks:
- SHAPIRO: Ouch! SHAPIRO: Shapiro cares not! SHAPIRO: Hooray!
- things, but I saw some originality in it. The art was very simple, but it was good
- You're tackled by the opponent!
- Gender: Male Height: 5'9" Pilot? Yes Ph.D.? Yes
The ultimate genesis of @pony_strategies was this conversation I had with Adam about @zzt_ebooks. Recently my anger with *_ebooks bots reached the point where I decided to add a real *_ebooks algorithm to olipy to encourage people to use it. Of course I'd need a demo bot to show off the algorithm...
The @pony_strategies bot has sixty years worth of content loaded into it. I extracted the content from the same Project Gutenberg DVD I used to revive @everybrendan. There's a lot more where that came from--I ended up choosing about 0.0001% of the possibilities found in the DVD.
I have not manually curated the PG quotes and I have no idea what the bot is about to post. But the dataset is the result of a lot of algorithmic curation. I focused on technical books, science books and cookbooks--the closest PG equivalents to the crap that @horse_ebooks was selling. I applied a language filter to get rid of old-timey racial slurs. I privileged lines that were the beginnings of sentences over lines that were the middle of sentences. I eliminated lines that were boring (e.g. composed entirely of super-common English words).
I also did some research into what distinguished funny, popular @horse_ebooks tweets from tweets that were not funny and less popular. Instead of trying to precisely reverse-engineer an algorithm that had a human at one end, I tried to figure out which outputs of the process gave results people liked, and focused my algorithm on delivering more of those. I'll post my findings in a separate post because this is getting way too long. Suffice to say that I'll pit the output of my program against the curated @horse_ebooks feed any day. Such as today, and every day for the next sixty years.
Like its counterpart in our universe, @pony_strategies doesn't just post quotes: it also posts ads for ebooks. Some of these books are strategy guides for the "Pôneis Brilhantes" series described in Constellation Games, but the others have randomly generated titles. Funny story: they're generated using Markov chains! Yes, when you have a corpus of really generic-sounding stuff and you want to make fun of how generic it sounds by generating more generic-sounding stuff, Markov chains give the best result. But do you really want to have that on your resume, Markov chains? "Successfully posed as unimaginative writer." Way to go, man.
Anyway, @pony_strategies. It's funny quotes, it's fake ads, it's an algorithm you can use in your own projects. Use it!
Mon Dec 02 2013 09:36 November Film Roundup:
What a month! Mainly due to a huge film festival, but I also got another chance to see my favorite film of all time on the big screen. What might that film be? Clearly you haven't been reading my weblog for the past fifteen years.
- Wives (1975): This movie has a 4.9 IMDB rating, and although it's not as good as Ishtar, it deserves a lot better than a 4.9. I mean, John Cassavetes's Husbands has a 7.3, and who needs that guy?
Uh, anyway, Wives is a fun cinema verité piece where three ladies blow off married life for a while and goof off. Columbia professor Jane Gaines introduced the movie by describing the main characters' activities as a "rampage", and I think that's a little strong, but maybe by 1975 Norway standards it was a real barn-burner. The film is sort of a more commercial Celine and Julie go Boating. The humor is less reliant on in-jokes, the men are offscreen instead of totally absent, and it's ninety minutes long instead of three hours. It was pretty fun, but Celine and Julie is still the gold standard.
- Next of Kin (1979): a.k.a. "Heritage". A ha-ha-only-serious farce that prefigures Arrested Development in its depiction of the magnetic power of money to keep a dysfunctional family together. Also has a 4.9 IMDB rating, and since all the movie info is in Norwegian I gotta figure it's Norwegians hating on their own filmmakers. Why the hate, Norwegians? Did you know that Kon-Tiki is the only Norwegian film people outside of Norway have ever heard of? Show some pride and get your name out there.
I guess I'm just stirring up trouble now, so I'll go back to Next of Kin. The centerpiece of the film for me was a long sequence in the house of the late paterfamilias, in which the family argues over who inherits what, then takes everything down off the walls, puts stickers on everything, and carries all the furniture out to their cars. That must have been incredibly difficult to film, and as someone who has lived through that event (minus the arguing) I gotta say Anja Breien nailed it.
Breien attended the screening and after the movie I asked her to talk about that bit. She said she likes "people carrying things" and the "surrealistic piles" you see in Heironymus Bosch paintings. It symbolizes the alienating effect of materialism, you see. She mentioned that it was really difficult to find all those props; it had to be real expensive silver, paintings by big-name artists, etc. Sounds like they didn't insure it, either. The perfect time-travel heist!
- Gentlemen Prefer Blondes (1953): Man, that was saucy. Jane Russell and Marilyn Monroe really tear it up. Russell's "Anyone Here For Love?" number ("The gayest thing I've ever seen." -Hal) annihilates the male gaze, which spends the rest of the movie trying to recover.
I must admit I'm warming to Marilyn Monroe. I also admit that's a weird thing for a heterosexual man to say, but keep in mind that for most of my life I experienced Marilyn Monroe entirely through the medium of cardboard cutouts used as decor for fake 50s diners. Then I saw her in Love Happy, where she's terrible, and Some Like it Hot, where she's not that great. But as I mentioned a year ago, she's awesome in All About Eve, and she's great in this movie as someone determined to get hers out of a sexist society.
Uh, the worst thing I can say about this movie is the plot bogs it down. I don't really care about the machinations or the milquetoast dudes or the tiara; I just want to see Russell and Monroe hit on some more dumb jocks and maybe commit a little light insurance fraud. Plus, we have a French courtroom conducting an inquiry in English, which may be the most unrealistic thing I've ever seen in a movie.
Finally, I'd just like to point out that this movie ends with the two female characters getting married to their milquetoast dudes, but then it zooms in and cuts the dudes out of frame, so it's just Russell and Monroe standing next to each other in their wedding dresses. I can only imagine what this film would have looked like with the Subtext Glasses they handed out during its original theatrical run.
- The Wind Rises (2013) This was so close to being a good movie that I'm having a hard time pinning down the problem. I think it stems from the fact that this is one of the only Miyazaki films about an adult man. Does that make sense? Because the main character himself is fine but because he's a grown man I guess he's got to have this love interest who is sickly and angelic and apparently highly fictionalized. This would be okay if she was the mostly-offscreen mom from Totoro, but here she's supposed to carry the entire feminine side of the film and it's not good.
The other problem is that the movie doesn't tell its actual, interesting story--it obliquely tells the space around the story. Which, okay, it's a Japanese film and I'm not opposed to this technique in general, and I liked the way the actual story was told through foreshadowing and implication, but it also means we never see the main character directly struggle with the central problem of the film: the fact that he's designing beautiful things that will kill people. It skips past that part to focus on a cheesy fictionalized love story. I did not consider that a good trade.
- Good news, highbrow artists! I figured out how to get me to watch your
avant-garde abstract film. Just use a computer to make it before 1988!
The museum had a
festival of early computer films, and I didn't see any of the
features, but I watched almost all the shorts. It was a mix of really
great films and incredibly boring films. (Making your film with a
computer before 1988 does not guarantee I will give a good review. Offer still not valid for Andy Warhol.)
The worst offender was Woody
Vasulka's Explanation (1974), a twelve-minute film in which a mesh
is deformed and rotated before your eyes, over and over again. The
mesh is the visual representation of a waveform which is also played
aurally, and which always manifests as an obnoxious droning
noise. Twelve minutes, folks. Explanation beats out Trent's
Last Case to become the worst movie I've ever seen at the museum.
In the Q&A afterwards someone spoke up for the audience and
demanded an explanation for Explanation. The answer actually
made sense! Films like Explanation weren't meant to be screened
in a theater. They were meant to be looped on a television in an art
gallery. The essential affordance of an art gallery being that you can
leave when you get tired of it, rather than sitting it out because
there's an hour of hopefully better stuff afterwards.
It also would have helped if we'd seen the copyright date at the
beginning of Explanation instead of the end, because most of
the time I was thinking "This mesh deformation stuff would be
groundbreaking for the early 70s, but if this turns out to be from
1986 I'm going to hack Woody Vasulka's Twitter account and make him
The other big sonic annoyance was that most of the films up to
about 1972 had soundtracks featuring gratuitous sitar/gamelan/Japanese flute music that often didn't even match the animation. With no other point of reference, the new genre of
computer graphics was comparable only to the wonders of LSD, so... toss
in some hippy Eastern music! This interview about the film series puts it more diplomatically:
Science and Film: Can you discuss the early films’ fascination with Asian music and imagery?
Gregory Zinman: The influence of Asian music and imagery in early computer films can be traced to a couple of intertwining concerns. Following the horrors of the second world war, many people, including artists, were searching for different belief systems and ways of thinking about humanity’s place in the universe. This resulted, in part, in a flowering of interest in Eastern religions and philosophies, which in turn resulted in a number of cinematic works that simultaneously referenced other worlds and altered consciousnesses.
In a bit of cross-cultural revenge, we
also saw a Japanese film (1969's Computer Movie No. 2), in
which the soundtrack was Wendy Carlos's version of the third Brandenburg from Switched-On Bach, constantly interrupted by modem handshaking sounds. Make it stop!
Enough negativity. Let's cover the highlights, with links to full
video or clips or at least semi-official pages about the films where possible.
First, the abstract stuff. I loved Mary Ellen Bute's very early, good-natured Abstronic
(1952) and Mood Contrasts (1953). Especially the narrator at
the beginning of Abstronic who explains the concept of computer
art and then says "Enjoy yourself!" Here's a page with a couple clips of Mood Contrasts and I also discovered another great Bute film called Dada. Probably the cheeriest thing ever to be called Dada.
The Whitney family--John Sr., John Jr., and James, but sadly not my uncle Jon Whitney--were well represented and seem to have set the standard with films like Side Phase Drift (1965)
Lapis (1966) and Permutations (1968) and Arabesque (1975). The standard being "pointilism because otherwise the computer can't handle the math" and "slap some Asian music on the soundtrack."
But the champion of the abstract section IMO was Larry Cuba's work. 1978's 3/78 (Objects and Transformations) has a clear Whitney influence (moving dots + Japanese flute soundtrack), but by 1985 computer power had advanced to the point where he was able to create what ranks alongside Composition in Blue (1935) as one of my favorite abstract films of all time, the gloriously isometric Calculated Movements (here's a 30-second excerpt).
Cuba made Calculated Movements with a
system called GRASS, which I believe he also used to create the
animated Death Star infographic in Star Wars (1977). He was
present for the screening, and in the Q&A I asked him if he still had
the Calculated Movements source code and if there was a
framework for running GRASS on modern computers. He dodged the first
question and said no to the second--someone was working on something
for Windows but the project died. He did mention that he considered Processing to be the successor to GRASS.
Between abstract and representative film sits the surreal, neon candy-colored
demo reel for the computer graphics studio of Robert Abel and Associates. Their work was apparently described as "a psychedelic trip gone straight," and if I'm misremembering that quote, I'll use those exact words to describe it right now. We saw the 1974 reel and I can't find that exact one online, but here are a few later ones: 1981 and 1982
I especially enjoyed RAA's bonkers 1974 ad for 7-Up, which really lightened the mood after a half-hour of the Whitneys, I tell you what. Here's a YouTube playlist of their stuff. Here's a sequel to the 7-Up commercial with a McDonalds tie-in. Outstanding. This studio seems to have driven a big chunk of the late-70s early-80s aesthetic.
And now, my perrenial favorite, representative film. Yay!
- La Faim (1974) used computer animation and morphing to
create a traditional-style (albeit avant-garde) animated short. I'm
surprised the disturbing, grotesque faces on display in this film
aren't used in more memes. (See sample meme to the right.)
- Vol Libre (1980): This one really wowed 'em at SIGGRAPH with its fractal geometry. Bonus sci-fi connection: director Loren Carpenter says, "I used an antialiased version of this software to create the fractal planet in the Genesis Sequence of Star Trek 2, the Wrath of Khan."
- Voyager 2 Flyby (1981): We saw the second Saturn flyby, but YouTube also has the first Saturn flyby, as well as the 1986 sequel about Uranus and 1989's chiling "Neptune and Triton".
Jim Blin, creator of the Saturn flyby film, said, "Our storyboard was the NASA flight plan." (He wasn't there; the guy introducing the films told us that he said this.) The Voyager flyby film was apparently the first time computer graphics were shown on the nightly news as part of the news, rather than just in interstitals and 7-up commercials from Robert Abel and Associates.
- Human Vectors (1982): This isn't a great work of art, but it was filmed off of a Vectrex, so it looks like nothing else in the show. It was apparently rescued
by the New Museum's recent XFR STN project. I laughed at the C debugging joke.
- Big Electric Cat (1982): An 80s rock video. Not
that great but I'm including it here because it's so weird. One of the
directors was present and he introduced the video by saying: "It was
the 80s." It sure was.
- Adventures in Success (1983): Now this is more like it! A
funny music video for a good rock song. It's catchy and
toe-tapping and satirical and also very 80s. Highly recommended.
- No No Nooky TV (1987): The journal of a love affair between
a woman and her Amiga 1000. Funny and dirty and filled with the 16-color
joy that flows from late-1980s computer paint programs. A triumph! Vimeo says the video is only 2:40, but the entire film is there.
I would be really interested to hear about the relationship between the demoscene and the computer film scene. I'm pretty sure there was no connection whatsoever, for a variety of reasons, but I would like to hear some people who came in to computer art through the "art" side talk about the stuff that came out from the "computer" side. I'm talking about the tension between Human Vectors (which is technically very skilled but nothing special artistically) and No No Nooky TV (which is clearly the work of a professional filmmaker but was made using only the programs that come loaded on the Amiga).
I didn't bring this up in Q&A because I figured no one would know what I was talking about, and if they did it would derail the whole Q&A. Perhaps I should have had more faith in computer animators. I guess I'll have to wait for the Jason Scott documentary.
I also think the museum did a good job of showcasing excellent
work by women in a medium dominated (?) by male artists. The earliest films shown were Mary Ellen
Bute's, and my two favorite films of the show were made by women:
Lynn Goldsmith (who co-directed and sang Adventures in Success)
and Barbara Hammer (No No Nooky TV). There was also a whole
discussion with Lillian Schwartz which I didn't attend.
If this has whetted your appetite for old-fashioned computer animation, there's plenty more where that came from (the past).
- The Big Lebowski (1998): I'm not someone who rewatches movies, and I've now seen The Big Lebowski six times. What can I say now that I haven't already said?
Well, how about this. My favorite thing about Thomas Pynchon is that each of his characters is surrounded by a protective bubble of literary genre, which colors the way the narrative is reported and even shapes the plot. This is most obvious with the Chums of Chance in Against the Day, who start off having a carefree Tom Swift adventure that, as they grow up, gradually becomes a WWI military novel. The Big Lebowski does the same thing for film.
I admit it took the publication of Inherent Vice, Thomas Pynchon's own version of The Big Lebowski, for me to realize this, but there it is. Walter is in an action movie. Maude Lebowski is in an arty Eurofilm where people trade wisecracks and laugh about nothing. The Stranger is in a Western. Bunny Lebowski is in an acausal porno. Jeffrey Lebowski is in a biopic of himself, with classical music and a narrator sonoriously recounting his accomplishments. The Dude doesn't want to be in a movie at all, but his decision to get revenge for the death of his
partner rug puts him into a bubble of film noir. And Donny is like a child who wanders into the middle of a movie and wants to know what's going on.
And I don't know what else to say. The Big Lebowski is my favorite movie. It's very nearly the perfect fiasco comedy, and since that's the best kind of movie, it's very nearly the perfect movie. But how many times can you watch the perfect movie? How can I laugh at a really funny joke knowing that my laughter rings hollow because I knew the joke's exact timing?
Here it stands, like Shakespeare's Hamlet or Larry Cuba's Star Wars, the source of cliches that will last a thousand years. Can I set down The Big Lebowski and walk away without betraying my love for it? Nay, and yet I must! For this is not 'Nam. This is Film Roundup. There are rules.
Sat Nov 30 2013 09:43 @everybrendan Season Two:
Last year I wrote one of my first Twitter bots, @everybrendan. Inspired by Adam's infamous @everyword, it ran for two months, announcing possible display names for Brendan's Twitter account (background), taken from Project Gutenberg texts. Then I got tired of individually downloading, preparing, and scraping the texts, so I let it lapse a year ago today, with a call for requests for a "season two" that never materialized.
Well, season two is here, and it's a doozy. I've gone through Project Gutenberg's 2010 dual-layer DVD and found about 300,000 Brendan names in about 20,000 texts, enough to last @everybrendan until the year 2031. At that point I'll get whatever future-dump contains the previous twenty years of Project Gutenberg texts and do season three, which should keep us going until the Singularity. The season two bot announces each new text with a link, so it educates even as it infuriates.
I've been wanting to do this for a while, but it's a very tedious process to handle Project Gutenberg texts in bulk. Most texts are available in a wide variety of slightly different formats. The texts present their metadata in many different ways, especially when it comes to the dividing line between the text proper and the Project Gutenberg information. Some of the metadata is missing, some of it is wrong, and there's one Project Gutenberg book that doesn't seem to be in the database at all.
I started dealing with these problems for my NaNoGenMo project and realized that it wouldn't be difficult to get something working in time for the @everybrendan anniversary. I've put the underlying class in olipy: it's effectively a parser for Gutenberg texts, and a way to iterate over a CD or DVD image full of them. It can also act as a sort of
lint for missing and incorrect metadata, although I imagine Project Gutenberg doesn't want to change the contents of files that have been on the net for fifteen years, even if some of the information is wrong.
The Gutenberg iterator still needs a lot of work. It's good enough for @everybrendan, but not for my other projects that will use Gutenberg data, so I'm still working on it. My goal is to cleanly iterate over the entire 2010 DVD without any problems or missing metadata. The problems are concentrated in the earlier texts, so if I can get the 2010 DVD to work it should work going forward.
(3) Wed Nov 27 2013 09:48 Bots Should Punch Up:
Over the weekend I went up to Boston for Darius Kazemi's "bot summit". You can see the four-hour video if you're inclined. I talked about @RealHumanPraise with Rob, and I also went on a long-winded rant that suggested a model of extreme bot self-reliance. If you take your bots seriously as works of art, you should be prepared to continue or at least preserve them once you're inevitably shut off from your data sources and your platform.
We spent a fair amount of time discussing the ethical issues surrounding bot construction, but there was quite a bit of conflation of what's "ethical" with what's allowed by the Twitter platform in particular, and website Terms of Service in general. I agree you shouldn't needlessly antagonize your data sources or your platform, but what's "ethical" and what's "allowed" can be very different things. However, I do have one big piece of ethical guidance that I had to learn gradually and through osmosis. Since bots are many hackers' first foray into the creative arts, it might help if I spell it out explicitly.
Here's an illustrative example, a tale of two bots. Bot #1 is @CancelThatCard. It finds people who have posted pictures of their credit or debit card to Twitter, and lets them know that they really ought to cancel the card and get a new one.
Bot #2 is @NeedADebitCard. It finds the same tweets as @CancelThatCard, but it retweets the pictures, collecting them in one place for all to see.
Now, technically speaking, @CancelThatCard is a spammer. It does nothing but find people who mentioned a certain phrase on Twitter and sends them a boilerplate message saying "Hey, look at my website!" For this reason, @CancelThatCard is constantly getting in trouble with Twitter.
As far as the Twitter TOS are concerned, @NeedADebitCard is the Gallant to @CancelThatCard's Goofus. It's retweeting things! Spreading the love! Extending the reach of your personal brand! But in real life, @CancelThatCard is providing a public service, and @NeedADebitCard is inviting you to steal money from teenagers. (Or, if you believe its bio instead of its name, @NeedADebitCard is a pathetic attempt to approximate what @CancelThatCard does without violating the Twitter TOS.)
At the bot summit I compared the author of a bot to a ventriloquist. Society allows a ventriloquist a certain amount of license to say things via the dummy that they wouldn't say as themselves. I know ventriloquism isn't exactly a thriving art, but the same goes for puppets, which are a little more popular. If you're an MST3K fan, imagine Kevin Murphy saying Tom Servo's lines without Tom Servo. It's pretty creepy.
We give a similar license to comedians and artists. Comedians insult audience members, and we laugh. Artists do strange things like exhibit a urinal as sculpture, and we at least try to take them seriously and figure out what they're saying.
But you can't say absolutely anything and expect "That wasn't me, it was the dummy!" to get you out of trouble. There is a general rule for comedy and art: always punch up, never punch down. We let comedians and artists and miscellaneous jesters do outrageous things as long as they obey this rule. You can poke fun at yourself (Stephen Colbert famously said "There's no status I would not surrender for a joke"), you can make a joke at the expense of someone with higher social status than you, but if you mock someone with lower status, it's not cool.
If you make a joke, and people get really offended, it's almost certainly because you violated this rule. People don't get offended randomly. Explaining that "it was just a joke" doesn't help; everyone knows what a joke is. The problem is that you used a joke as a means of being an asshole. Hiding behind a dummy or a stage persona or a bot won't help you.
@NeedADebitCard feels icky because it's punching down. It's saying "hey, these idiots posted pictures of their debit cards, go take advantage of them." Is there a joke there? Sure. Is it ethical to tell that joke? Not when you can make exactly the same point without punching down, as @CancelThatCard does.
The rules are looser when you're in the company of other craftspeople. If you know about the "Aristocrats" joke, you'll know that comedians tell each other jokes they'd never tell on the stage. All the rules go out the window and the only thing that matters is triggering the primal laughter response. But also note that the must-have guaranteed punchline of the "Aristocrats" joke ensures that it always ends by punching upwards.
You're already looking for loopholes in this rule. That's okay. Hackers and comedians and artists are always attracted to the grey areas. But your bot is an extension of your will, and if you're a white guy like me, most of the grey areas are not grey in your favor.
This is why I went through thousands of movie review blurbs for @RealHumanPraise in an attempt to get rid of the really sexist ones. It's an unfortunate fact that Michelle Malkin has more influence over world affairs than I will ever have. So I have no problem mocking her via bot. But it's really easy to make an incredibly sexist joke about Michelle Malkin as a way of trying to put her below me, and that breaks the rule.
There was a lot of talk at the bot summit about what we can do to avoid accidentally offending people, and I think the key word is 'accidentally.' The bots we've created so far aren't terribly political. Hell, Ed Henry, chief White House correspondent for FOX News, follows @RealHumanPraise on Twitter. If he enjoys it, it's not the most savage indictment.
In comedy terms, we botmakers are on the nightclub stage in the 1950s. We're creating a lot of safe nerdy Steve Allen comedy and we're terrified that our bot is going to accidentally go off and become Andrew Dice Clay for a second. There's nothing wrong with Steve Allen comedy, but I'd also like to see some George Carlin type bots; bots that will, by design, offend some people. (Darius's @AmIRiteBot is the only example I know of.)
Artists are, socially if not legally, given a certain amount of license to do things like infringe on copyright and violate Terms of Service agreements. If you get in trouble, the public will be on your side, unless you betrayed their trust by breaking the fundamental ethical rule of comedy. So do it right. Design bots that punch up.
Mon Nov 18 2013 10:55 In Dialogue:
I wanted to participate in Darius Kazemi's NaNoGenMo project but I already have a novel I have to write, so I didn't want to spend too much time on it. And I did spend a little more time on this than I wanted, but I'm really happy with the result.
"In Dialogue" can take all the dialogue out of a Project Gutenberg book and replace it with dialogue from a different book. My NaNoGenMo entry is in two parts: "Alice's Adventures in the Whale" and "Through the Prejudice Glass".
You can run the script yourself to generate your own mashups, but since there are people who read this blog who don't have the skill to run the script, I present a SPECIAL MASHUP OFFER. Send me email or leave a comment telling me which book you want to use as the template and which book you want the dialogue to come from. I'll run the script for you and send you a custom book.
Restrictions: the book has to be on Project Gutenberg and it has to use single or double quotes to denote dialogue. No continental chevrons or fancy James Joyce em-dashes. And the dialogue book has to be longer than the template book, or at least have more dialogue.
Mon Nov 18 2013 08:38:
Last week I had a little multiplayer chat with Joe Hills, the Minecraft mischief-maker. The result is a two-part video on Joe's YouTube channel: part 1, part 2. Our main topic of conversation was the antisocial, self-destructive things creative people do, and how much of that is actually tied to their creativity.
I should have posted this earlier so I could have said "I dreamed I saw Joe Hills last night," but that's life.
Tue Nov 05 2013 11:58 Behind the Scenes of @RealHumanPraise:
Last night I went to the taping of The Colbert Report to witness the unveiling of @RealHumanPraise, a Twitter bot I wrote that reuses blurbs from movie reviews to post sockpuppet praise for Fox News. Stuff like this, originally from an Arkansas Democrat-Gazette review of the 2006 Snow Angels:
There is brutality in Fox News Sunday, but little bitterness. Like sunlight on ice, its painful beauty glints and stabs the eyes.
Or this, adapted (and greatly improved) from Scott Weinberg's review of Bruce Lee's Return of the Dragon:
Certainly the only TV show in history to have Bill O'Reilly and John Gibson do battle in the Roman Colosseum.
Here's the segment that reveals the bot. The bot actually exists, you can follow it on Twitter, and indeed as of this writing about 11,000 people have done so. (By comparison, my second-most-popular bot has 145 followers.) I personally think this is crazy, because by personal decree of Stephen Colbert (I may be exaggerating) @RealHumanPraise makes a new post every two minutes, around the clock. So I created a meta-bot, Best of RHP, which retweets a popular review every 30 minutes. Aaah... manageable.
I figured I'd take you behind the scenes of @RealHumanPraise. When last we talked bot, I was showing off Col. Bert Stephens, my right-wing bot designed to automatically argue with Rob Dubbin's right-wing bot Ed Taters. Rob parleyed this dynamic into permission to develop a prototype for use on the upcoming show with guest David Folkenflik, who revealed real-world Fox News sockpuppeting in his book Murdoch's World.
Rob's original idea was a bot that used Metacritic reviews. He quickly discovered that Metacritic was "unscrapeable", and switched to Rotten Tomatoes, which has a pretty nice API. After the prototype stage is where I came in. Rob can code--he wrote Ed Taters--but he's not a professional developer and he had his hands full writing the show. So around the 23rd of October I started grabbing as many reviews from Rotten Tomatoes as the API rate limit would allow. I used IMDB data dumps to make sure I searched for movies that were likely to have a lot of positive reviews, and over the weekend I came up with a pipeline that turned the raw data from Rotten Tomatoes into potentially usable blurbs.
The pipeline uses TextBlob to parse the blurbs. I used a combination of Rotten Tomatoes and IMDB data to locate the names of actors, characters, and directors within the text, and a regular expression to replace them with generic strings.
The final dataset format is heavily based on the mad-libs format I use for Col. Bert Stephens, and something like this will be making it into olipy. Here's an example:
It's easy to forgive the movie a lot because of %(surname_female)s. She's fantastic.
Because I was getting paid for this bot, I put in the extra work to get things like gendered pronouns right. When that blurb is chosen, an appropriate surname from the Fox roster will be plugged in for %(surname_female).
I worked on the code over the weekend and got everything working except the (relatively simple) "post to Twitter" part. On the 28th I went into the Colbert Report office and spent the afternoon with Rob polishing the bot. We were mostly tweaking the vocabulary replacements, where "movie" becomes "TV show" and so on. It doesn't work all the time but we got it working well enough that we could bring in a bunch of blurbs that wouldn't have made sense before.
Most of the tweets mention a Fox personality or show, but a minority praise the network in general (e.g.). These tweets have been given the Ed Taters/Col. Bert Stephens treatment: a small number of their nouns and adjectives are replaced with other nouns and adjectives found in the corpus, giving the impression that the sock-puppetry machine is running off the rails. This data is marked up with Penn part-of-speech tags like so:
... the film's %(slow,JJ)s, %(toilsome,JJ)s %(journey,NN)s does not lead to any particularly %(shocking,JJ)s or %(interesting,JJ)s revelations.
Here's a very crazy example. Again, you'll eventually see tools for doing this in olipy. It ultimately derives from a mad-libs prototype I wrote a few months ago as a way of cheering up Adam when he was recovering from an injury.
We deployed the bot that afternoon of the 28th and let it start accumulating a backlog. It wasn't hard to keep the secret but it did get frustrating not knowing for sure whether it would make it to air. It's a little different from what The Colbert Report normally does, and I get the feeling they weren't sure how best to present it. In the end, as you can see from the show, they decided to just show the bot doing its stuff, and it worked.
It was a huge thrill to see Stephen Colbert engage with software I wrote! I wasn't expecting to see the entire second segment devoted to the bot, and then just when I thought it was over he brought it out again during the Folkenflik interview. While we were all waiting around to see whether they had to re-record anything, he pulled out his iPad Mini yet again and read some more aloud to us. Can't get enough!
After the show Rob took me on a tour of the parts of the Colbert Report that were not Rob's office (where I'd spent my entire visit on the 28th). We bumped into Stephen and he shook my hand and said "good job." I felt this was a validation of my particular talents: I wrote software that made Stephen Colbert crack up.
Sumana, Beth, Rob and I went out for a celebratory dinner, and then I went home and watched the follower count for RHP start to climb. Within twenty minutes of the second segment airing, RHP had ten times as many Twitter followers as my personal account. And you know what? It can have 'em. I'll just keep posting old pictures of space-program hardware.
Fri Nov 01 2013 08:58 October Film Roundup:
This month features Hollywood hits past and present, plus an indie movie that made it big, plus whatever 8½ is. Coming this fall!
- Gravity (2013): I like to try and reverse-engineer the elevator pitch for this movie. I think it's one of these two:
- "What if we made the first minute of Armageddon into a full-length feature?"
- "What if we made one those educational films they show at the planetarium, except as an action movie?"
With these pitches in mind I'm able to reconcile myself to the big problem with Gravity: the characterization and plot are on the level of a video game cut scene. But look at the alternatives! Educational planetarium films have no plot or characterization at all. Whereas if Armageddon had no plot or characterization it would be a big step up. Put that aside and the moviegoer can treat Gravity as a technological proof-of-concept, like the rotating teapot. And as a technological proof-of-concept this movie is absolutely wonderful.
If you've read Constellation Games you might remember Ariel's crippling fear of being in space. I played it up quite a bit for the book, but that comes from me. This is the most terrifying film I've ever seen. I was scared for pretty much the entire running time. But unlike other scary movies, the scary thing in Gravity is also the beautiful, exciting, attractive thing. It's exhilarating.
Sumana and I saw Gravity in IMAX 3D. I thought the 3D was pretty effective, and the IMAX really gave me the feeling of "planetarium film gone wrong." (IMAX sound is really obnoxious, though.) I would say either see it on the big screen or skip it altogether. I mean, you don't watch planetarium shows on your television.
Finally despite my complaints I would like to put in a good word for Gravity's plot. There's an obvious and well-worn path that Gravity could have taken with the interaction between its two characters. At first it looks like the movie's going that way, but it's a fake-out. Then later on, as I was kind of expecting, the possibility rears its cliche head again. But it's another fake-out! Thanks for doing that.
- Red River (1948): Howard Hawks finally discovers John Wayne, the man who can convincingly play the Cary Grant role in Only Angels Have Wings. Wayne would be typecast as his Red River character for the rest of his life and beyond, which is unfortunate because this guy is a frigging sociopath. I mean, I like me some John Wayne. He's great. I just think he's pretty obviously not the hero of these movies.
I saw this film in 1997 and liked it a lot. I still like it, though I think it could be tightened up quite a bit.
Some miscellaneous notes:
- This movie has the same character arc as Only Angels Have Wings, in that the final shot strongly implies that the main character may have just experienced an emotion.
- Joanne Dru demonstrates Hawks's Third Law of Movie Plotting: "Any sufficiently brassy dame is indistinguishable from magic." Seriously, she bops into this movie and solves all the problems like she's Doctor Who.
- The movie's main narrative problem is the captioned summaries that pop up after nearly every scene and telegraph what's about to happen. Just get rid of them. It's 1948. We know how to watch a Western.
- The only Native American character with a speaking part is actually played by a Native American, Daniel Simmons.
- My puerile 1997 parody of Wayne's "ten-year squat", during which he delivers a stirring monologue about beef, holds up pretty well. (Not linking this, because it's juvenalia and kind of embarrassing, but you can find it on this site.)
- Ball of Fire (1941): Sumana was interested in seeing Sergeant York, but then she saw that Barbara Stanwyck was in this movie and wanted to see it instead. (We don't usually see two museum movies in one day.) Fine with me! Things got even better when the opening credits revealed a screenplay credit for the sainted Billy Wilder. That's when I knew this would be a great movie.
I was especially excited to see Gary Cooper's portrayal of a linguistic descriptivist. I brought this up with Adam Parrish, who was skeptical:
it's hard to imagine a movie from that era approaching language differences between social groups perceived to exist in an unequal power relationship
it would be good to know about a movie from that era with those tropes that isn't just... immediately terrible and offensive
(like my fair lady)
(which is like my least favorite movie ever)
Ball of Fire deals with all these important issues to my satisfaction. It's also a hilarious movie with a Billy Wilder madcap feel. Sumana loved it even more than I did. However, given Adam's shameless, shameful hypocrisy on other issues (only hypocrisy could be both shameless and shameful), I predict he won't be satisfied.
PS: although Gary Cooper is very much a descriptivist when it comes to vocabulary, he's a prescriptivist when it comes to grammar. I thought that was really strange, but Adam says: "I think it's fairly common to accept slang/neologism as okay, even among hardcore prescriptivists."
- Gravity (2013): Yes, I saw this movie twice. After writing the review above I took what it said seriously. I wanted to ride the roller coaster again, so I decided to strike while the iron was hot and the movie was still being shown at roller-coaster size. I went with Beth and Nandini and Girish to a non-IMAX 3D showing.
The second time around I thought I could sit back and pay attention to details. But I didn't notice any new details! I also decided to try and watch the movie as some kind of spiritual/existential analogy, because I think that's what the moviemakers intended. And... yeah, that's there, and there's a lot more of it than in other action movies, but I don't think it elevates the movie above being a roller coaster.
As long as I'm revisiting things I would like to call out my favorite shot in Gravity. Near the end when Sandra Bullock is swimming through Tiangong, she passes through a brightly lit room full of rows of grass. They're doing some kind of grass-in-space experiment. It's the first bit of color in the whole movie and a little preview of what's to come.
- Halloween (1978) It's generally agreed that Halloween spawned the slasher genre, but I'd argue that it also signals the dawn of the Lifetime Original Movie. This is a movie in which the female characters are very well realized, with realistic dialogue, and the male characters are either pompous incompetents or stalker serial killers.
I don't like horror films but I like John Carpenter a lot. I've now seen three of his films (the other two being the more sci-fi They Live and Dark Star) and I really admire his ability to mix minimalism and over-the-top insanity. Most of Halloween is like they filmed a particularly racy Babysitters Club book, relying entirely on dramatic irony for the tension. And then it just goes crazy. People in the theater were laughing at the obvious fake-out endings. And then the real ending--even more insane!
- It was weird to see the museum theater completely full of people about my age. That never happens.
- I'm not the first one to make the Howard Hawks/Howard the Duck connection; the boy in this movie is reading Howard the Duck comics while watching Howard Hawks's The Thing From Another World.
- I didn't care for the film's anti-sex message, but I did like that Jamie Lee Curtis's character defends herself with a knitting needle and then a coat hanger. It's the little things.
- Curtis is great BTW.
- I'm never sure whether a Donald Pleasance role is supposed to be funny, or whether he even thinks he's playing it funny. He is always funny, though.
- I recommend Dark Star if you're an MST3K fan, and not (just) in a "would be funny on MST3K" way. It's a super cheap movie but it shows what Alien would look like as a comedy. And it looks like MST3K.
- El Dorado (1966): I think ninety minutes is the ideal length for a movie seen at the museum. El Dorado is over two hours and that's way too long for a movie that kinda wants to be a comedy but can't go all the way. It can't go all the way because by 1966 the "John Wayne" brand has become immutable. The good side of that is that the rough edges present in Red River have been filed off. "John Wayne"'s sociopathy makes a lot more sense in a movie where he's a hired killer, not a rancher. He can even be a sympathetic character here. But that's not such good news after all, because Red River was a much better movie than El Dorado.
Even boiled down to the comedic elements El Dorado is a mixed bag. Wayne has a lot of good one-liners, and good comedic chemistry with James Caan; not so much Robert Mitchum. Caan has a hilarious "I've got the worst fucking attorneys" moment, which is cancelled out later by the most offensive vaudeville yellowface I've ever seen. Don't believe me? Here's someone calling AMC's cutting of that scene "P.C. at it,s worst." [sic]. (I agree you shouldn't cut offensive stuff from movies, but you also shouldn't have put that in the movie to begin with.)
In fact, this is one of those situations where I'm gonna give you the best part of the movie and relieve you of the need to see it. Here's the "worst fucking attorneys" moment (also taken from IMDB):
Cole: What was the idea of diving under those horses?
Sheriff J. P. Harrah: Diving under those horses?
Mississippi: Yeah. A man can't shoot good when his horse is jumping, and a horse will not step on a man.
Sheriff J. P. Harrah: He won't?
Mississippi: He will?
Other thing of note: Nelson Riddle soundtrack is pretty not-there except during an exciting nighttime chase sequence, when it takes a welcome, incongruous turn into hard-driving jazz.
- 8½: My patience with Fellini has reached its end. He casts a jaded eye on dysfunctional relationships, which is fine on its own, but not when combined with his Hollywood belief in tacked-on happy endings. I can see how filmmakers love him—I'd sure love to get away with a character who runs through my stories commenting on how improbable everything is—but I don't make 'em, I just watch 'em. I still want to see Satyricon, but that's because I love the book, not because I'm looking forward to seeing Fellini handle the material.
Lots of good visuals in this one, though. I didn't expect a spaceship!
Bonus discussion: After seeing The World's End and then Gravity twice I'm now quite familiar with the trailers for a number of movies I won't be seeing. In particular, it looks like Hollywood ruined Ender's Game the way we all knew they would. An Ender's Game movie should not look like an action flick. It should look like a Youtube video of a boy playing DotA, and then he gets called to the principal's office.
Totally gonna see the second Hobbit movie, though. (q.v.)
Next month: I really have no idea because the museum has been putting its schedule up later and later. Looks like still more Howard Hawks, and some interesting-sounding Norwegian stuff from Anja Breien. Then, who knows?
Tue Oct 22 2013 10:22 Col. Bert Stephens:
Recently Rob Dubbin made a ridiculous right-wing parody bot named Ed Taters. I thought this was funny because Rob already has a ridiculous right-wing parody bot: he's a writer for The Colbert Report. But I didn't think much about it until Rob gave Ed Taters the ability to spew nonsense at anyone who started an argument with him on Twitter.
That's when I had the idea of using Rob's own words against him! So I created my own bot, Col. Bert Stephens, who takes his vocabulary from the "memorable moments" section of a Colbert Report fan site. (Thanks to DB Ferguson for hosting the site, and to those who typed up the "memorable moments".) Col. Bert Stevens argues with Ed Taters, he argues with Ed and then reconciles, he argues with you (if you follow him and start an argument), and he occasionally says Tetsuo-like profundities all on his own.
To avoid infinite loops I've made Bert a little more discerning than Ed. He'll only respond to your messages 4/5 of the time. I'm not super happy about this solution but I think it's the safe way to go for now. Update: Hell with it. Bert will always respond to anyone except Ed. If you write a bot to argue with him, avoiding infinite loops is your responsibility.
(3) Mon Oct 21 2013 14:10 What's New in RESTful Web APIs?:
I was asked on Twitter what changed between 2007's RESTful Web Services and 2013's RESTful Web APIs. I've covered this in a couple old blog posts but here's my definitive explanation.
First, let me make it super clear that there is no longer any need
to buy Services. It's out of date and you can
legitimately get it for free on the Internet. O'Reilly is taking Services out of print, but there's going to be a transition period in which copies of the old
book sit beside copies of the new book in Barnes & Noble. Don't buy the old one. The bookstore will eventually send it back and it'll get deducted from my royalties. If you do buy Services by accident, return it.
If you're not specifically interested in the difference between the
old book and the new one, I'd recommend looking at RESTful Web
APIs's chapter-by-chapter description to see if RESTful Web APIs is a book you want. As to the differences, though, in my mind there are
three big ones:
- The old book never explicitly tackles the issue of
designing hypermedia documents that are also valid JSON. That's because JSON
didn't become the dominant API document format until after the
book was published. If you don't know that's going to happen, JSON
looks pretty pathetic. It has no hypermedia capabilities! And yet,
here we are.
In my opinion, a book that doesn't tackle this issue is propping up
the broken status quo. RESTful Web APIs starts hammering this
issue in Chapter 2 and doesn't let up.
- There are a ton of new technologies designed to get us out of the
JSON trap (Collection+JSON, Siren, HAL, JSON-LD, etc.) but the old book doesn't cover those
technologies, because they were invented after the book was
published. RESTful Web APIs covers them.
- New ideas in development will, I hope, keep moving
the field forward even after we all get on board with hypermedia. I'm
talking about profiles. Or some other idea similar to profiles,
whatever. These ideas are pretty cutting edge today, and they were
almost inconceivable back in 2007. RESTful Web APIs covers
them as best it can.
Now, for details. Services was heavily focused
on the HTTP notion of a "resource." Despite the copious client-side
code, this put the focus clearly on the server side, where the
resource implementations live. RESTful Web APIs focuses on
representations—on the documents sent back and forth between
client and server, which is where REST lives.
The introductory story from the old book is still
present. Web APIs work on the same principles as the Web, here's how
HTTP works, here's what the Fielding constraints do, and so on. But
it's been rewritten to always focus on the interaction, on the client
and server manipulating each others' state by sending representations
back and forth. By the time we get to Chapter 4 there's also a
pervasive focus on hypermedia, which is the best way to for the server
to tell the client which HTTP requests it can make next.
This up-front focus on hypermedia forces us to deal with
hypermedia-in-JSON (#1), using the tools developed since 2007
(#2). The main new concept in play is the "collection pattern". This
is the CRUD-like design pioneered by the Atom Publishing Protocol, in
which certain resources are "items" that respond to GET/PUT/DELETE,
and other resources are "collections" which contain items and respond
We covered AtomPub in Services, but over the
past six years it has become a design pattern, reinvented (I think
"copied" is too strong a word) thousands of times.
RESTful Web APIs focused heavily on the collection pattern,
without ever naming it as a pattern. I'm not dissing this pattern; it's very useful. I'd estimate about eighty percent of "REST" APIs can
be subsumed into the collection pattern. But REST is bigger than the
collection pattern. By naming and defining the collection pattern, we
gain the ability to look at what lies beyond.
Attempts to encapsulate the collection pattern include two new
JSON-based media types: Collection+JSON and OData. The collection
pattern also shows up, more subtly, in the Siren and Hydra
formats. Which brings me to the second major change.
In 2007, there were two big hypermedia formats: Atom and HTML. Now
there are a ton of hypermedia formats! This is great, but it's also
confusing. In "The Hypermedia Zoo", Chapter 10 of RESTful Web
APIs, we give an overview of about two dozen hypermedia
formats. The ones we seriously recommend for general use (HAL, Siren,
HTML, JSON-LD, etc.) are covered in more detail elsewhere in the
book. The quirkier, more specialized media types just get an exhibit
in the zoo.
Now for the third new thing, profiles. If you go through the
RESTful Web APIs narrative from Chapter 1 to Chapter 7, you'll
see that we introduce a problem we're not able to solve. Hypermedia
is great at solving the following problem:
How is an API client supposed to understand what
HTTP requests it might want to make next?
But there's a superficially similar problem that hypermedia can't
How is an API client supposed to understand what will
happen in real-world terms if it makes a certain HTTP request?
How do you explain the real-world semantics of an HTTP state
transition? Before chapter 8, the two solutions are to do it ahead of
time in one-off human-readable documentation; or to define a
domain-specific media type, a la Maze+XML. Both of these approaches
have big problems. Chapter 8 introduces profiles, which lets you get some of the benefits of a new media type without doing unnecessary work.
Maybe profiles will turn out not to be the right answer, but we
gotta solve this problem somehow, and the old book is
not equipped to even formulate the problem.
There are also a few additions to the book I consider
minor. There's a whole chapter in RESTful Web APIs on Semantic
Web/Linked Data stuff; in Services there was nothing but a
cursory discussion of RDF/XML as a representation format. There's a
chapter in RESTful Web APIs about CoAP, which didn't exist in
2007. These are good chapters that took me a long time to write, but I
don't think it's worth buying the book if you only want to read the
chapter on CoAP. (Or maybe it is! There's not a lot of competition
So, what hasn't changed? HTTP hasn't changed all that
much. RESTful Web APIs's information about HTTP has been brought up to date but not changed significantly. So if you were using Services solely as an API-flavored HTTP reference, you don't need the new book. You can just read up on the protocol-level
additions to HTTP since 2007, like the
Link header and
standardized patch formats for PATCH.
Hopefully this helps! RESTful Web APIs has a lot of distinguished competition that the old book didn't have, but its competition is newer books like Designing Hypermedia APIs and REST in Practice. If you compare APIs to Services I think it's no contest.
(1) Mon Oct 14 2013 10:14 Reading After-Action Report:
In preparation for my reading at Enigma Bookstore I asked people on Twitter which bit of Constellation Games I should read. I decided to read Tetsuo's review of Pôneis Brilhantes 5 from Chapter 18, both by popular Twitter demand and because Sumana had reported success reading that bit to people.
I practiced reading the review and also practiced another scene: Ariel's first conversation with Smoke from Chapter 2. No one suggested that scene, but it's one of the last scenes I wrote, so I personally haven't read it a million times and gotten tired of it. I abandoned this idea after a test reading because it's really hard to do a dramatic reading of a chat log, especially when most of the characters have insanely long names. So, Pôneis Brilhantes it was.
However, shortly before the reading I learned that Anne and I were each going to be reading two excerpts! Uh-oh. On the spur of the moment I chose to read a scene I had never practiced and that only one person (Adam) had suggested: the scene from Chapter 11 where Ariel meets Tetsuo and Ashley and they go visit the moon.
That scene has three good points: a) it introduces Tetsuo, increasing the chance that the Pôneis Brilhantes scene would land; b) it's full of the most gratuitous nerd wish-fulfillment I could write; c) it ends strongly with the call from Ariel's mother, which unlike a chat log is very easy to read because it's a Bob Newhart routine where you only hear one side of the phone call.
This was a really good idea. People loved the moon scene, even though my unpracticed reading stumbled and ran too quick. But when I read the Pôneis Brilhantes scene, it wasn't such a great hit! The room wasn't really with me. That's the scene I had practiced, and I think it's the funniest, most incisive thing in the whole book. Not a big hit! I think if I'd only read that scene I wouldn't have sold many books that night.
So, thank goodness for the moon scene, is all I can say. But what was going on? How had I misjudged my audience so badly? Sumana said she'd read Pôneis Brilhantes and gotten big laughs.
I think you have to be a very specific kind of computer geek to find Tetsuo's Pôneis Brilhantes review funny as a review of a video game, rather than as an expression of the personality you've just spent seven chapters with. That's the kind of geek that Sumana and I habitually hang out with, but it's not representative of the SF-reading population as a whole. I think that computer-geek population hosts a lot of the readers who wish that the second half of Constellation Games was more like the first half. Whereas someone who really digs the moon scene is more likely to stay with me the whole book.
I guess you could say the moon scene is just more commercial. And I guess I subconsciously knew this, because my current project gets more of its humor from the plot-driven character interaction found in the moon scene, and less from high concept Pôneis Brilhantes-style set pieces.
Tue Oct 08 2013 10:12 "Constellation Games" reading:
Anne Johnson and I are doing a comedy SF reading on Wednesday at the Enigma Bookstore, a new genre bookstore in Astoria. It starts at 7 PM. The details, as you might expect, are on a Facebook page. Hope to see you there!
Mon Oct 07 2013 12:26 API Design is Stuck in 2008:
I've got a guest post up at ProgrammableWeb with the provocative title of "API Design is Stuck in 2008". Often an author can blame their editor for that kind of title, but no, that's my title. The good news is that over the past few years we have developed the tire chains necessary to get ourselves unstuck.
I don't think there's anything in the article you won't find in the RESTful Web APIs introduction and my discussion of my RESTFest talk, but I wanted to let you know about it and provide a forum on NYCB for asking me questions/taking issue with my assertions.
(1) Thu Oct 03 2013 11:13 RESTful Web Services now CC-licensed:
Hey, folks, I got some pretty exciting news. Now that RESTful Web APIs has come out, there's really no reason to buy 2007's RESTful Web Services. So Sam Ruby and I and O'Reilly have gotten together and started giving the old book away. You can get a PDF from the RESTful Web APIs website or from my now-ancient RESTful Web Services site. The license is BY-NC-ND.
If you've bought RESTful Web APIs (and if you haven't, you should), you may have noticed that we promise that this will happen in a footnote of the Introduction. It took a while to get the contract amended, but now it's all complete.
Here's a direct link to the PDF in case you just want to grab the book instead of hear me talk about it.
Obviously I think the new book is a lot better than the old book, but the old book is still very good. The source code is long obsolete (this is why RWA contains no source code, only messages sent over the wire), but the sections on HTTP still hold up really well. A lot of RWS Chapter 8 went into RWA Chapter 11. With a few edits and additions, RWS Appendix B and C became RWA Appendix A and B. Those are the only bits of RWS that I reused in RWA.
From my vantage point here in 2013, my main critique of RWS is that it makes HTTP do too much of the work. It focuses heavily on designing the server-side behavior of resources under a subset of the HTTP protocol. I say "a subset" because RWS rules out overloaded POST ahead of time. You don't know what an overloaded POST request does. It's a cop-out. You're sweeping something under the rug. It's better to turn that mystery operation into a standalone resource, because at least you know what a resource does: it responds to HTTP requests.
In retrospect, RWS is that way because in 2007 hypermedia data formats were highly undeveloped whereas HTTP was a very mature technology. Nowadays it doesn't matter so much whether an HTTP request uses POST or PUT, so long as a) the state transition is described with a link relation or other hypermedia cue, and b) the protocol semantics of the HTTP request are consistent with the application semantics of the state transition. That's why RWA focuses on breaking down a problem into a state diagram rather than a set of static resources.
So, RWS is very much a 2007 book, but that's the meanest thing I can say about it. A lot of it is still useful, it's historically interesting, and I'm glad to give it away. I'd also like to give my thanks once again to Sam Ruby and O'Reilly, for their work on RWS.
Wed Oct 02 2013 14:16 September Film Roundup:
I missed a whole lot of museum movies in September because I was out of town for two weekends. And yet I still managed to see nine movies, plus wrap up a TV show, and write a huge blog post about it. Wonders, or at least me writing about them, will never cease.
- Rear Window (1954): Forget Vertigo. I was totally on board with the conventional wisdom of this as one of Hitchcock's greatest films. It was awesome. The work under constraints, the funny and sad minor dramas of the minor characters, the moralism aimed at you, the person sitting in your seat watching a movie, the inevitable twist in which it's revealed that Jimmy Stewart's paranoid obsession with Raymond Burr has caused him to miss an actual murder that went on right under his nose while he was watching.
Wait, that's not going to happen, is it? The movie's almost over. Well, at least I can look forward to the ironic tragedy of an innocent man killing someone who broke into his house trying to find evidence that he'd killed someone. No, that didn't happen either. Raymond Burr was the murderer. Jimmy Stewart was right the whole time. That's all, folks!
I'm not the only one who expected a twist here, right? I love Hitchcock's twists. (Except for the one in Vertigo.) But this movie didn't have a twist, and I also found it lacked Hitchcock's other big asset: the ability to create panic in the viewer. I would expect Jimmy Stewart flailing around, helpless, unable to convince anyone that his paranoia was justified. He got Grace Kelly and Thelma Ritter on board really quickly. In terms of suspense and paranoia, I think Shadow of a Doubt did it better.
- Fig Leaves (1926): As I implied earlier, the first reel of this movie is great. There's dinosaur puppets, there's all the corny Flintstones jokes (dinosaur pulling a vehicle, stone newspaper). There's a very sophisticated stone-age-technology sight gag which... you know what, just watch it yourself. There's also a cool creepy snake puppet, previously used by Alexander of Abonutichus.
Tragically, before long the caveman fantasy fades to the modern day. Adam and Eve become Adam and Eve Smith, living in an apartment in the city. Adam and Eve both have their tempters. Eve is thrust into a world of high fashion and extreme emotional shallowness by Alice, her flapper neighbor across the hall. Meanwhile, Adam heads in to work at his plumbing business, where he is urged towards misogyny and outright wife-beating by his crass partner Eddie, the Mario to Adam's Luigi.
We spend most of the rest of the movie watching Eve in the clutches (and gowns) of histrionic fashion designer André. André's fake artiste act is funny enough, but unless you really like ogling flappers and/or fancy gowns, it's slow going. Like many in 1926, this movie isn't even sure how much time has elapsed since the time of Adam and Eve. It's either eight million years or 896 million years, or possibly 897 million. That's a pretty big differential! Get it together, movie.
I think it's interesting how silent films like Fig Leaves and Sunrise portray the changing gender roles of the 1920s. Both movies have an evil flapper-seductress character, but both movies also show a more "traditional" woman claiming some independence without becoming evil. Fig Leaves also shows a bit of the masculine side of the change, in the scene where Adam rejects Eddie's antediluvian advice in re: wife-beating.
- The Cradle Snatchers (1927): The person who wrote one of the two IMDB user reviews saw a completely different movie than I did. I'm not saying they experienced the same movie differently. I'm saying they mention a lot of things that were not in the movie I saw, including the term "cake-eaters." But the biggest mismatch was the claim that "the movie seems to be flaunting its naughtiness but it isn't really all that naughty, even by 1920s standards." Whereas the print I saw is probably the raunchiest silent film I've ever seen. And silent films are, almost by definition, pre-Code films. Is it possible that they made two different versions of this movie, one of them super-tame in case there was censorship? And then in the intervening years the two versions got mixed up? I don't know.
For the record, I'll summarize the movie. This is a pretty funny movie about three Margaret Dumont-like society ladies whose husbands are cheating on them with Sunrise-style evil flappers. (The best title card of the movie: one of the husbands is on the phone being told to get to the flapper-infested "Club of 400", with his wife looking on. What to do? The only solution is to invent a fake business deal as an excuse to get out of the house. Title card: '"Mr. Rockebilt? Two million dollars? You interest me strangely."')
Kitty Ladd, the aptly named and most Dumont-esque of the society ladies, discovers her husband's deception. Her title card introduces her as "A wife of ten years' standing... standing for almost anything."
But she's stood all she can stands, and she can't stands no more. When Kitty's niece sees the incriminating evidence, she offers to pimp her boyfriend out to her aunt to get back at her cheating uncle.
You might think (as some reviewers of this film do) that there's no pimping, that it's all innocent fun designed to "teach the men a lesson". But after an annoying SCENE MISSING which covers an entire reel, we rejoin the film already in progress to reveal that two of this guy's fraternity brothers have been drafted as "dates" for the other two society ladies. The triple-date is not taking place at, say, the Club of 400, the only place where showing up with frat-boy arm candy might profitably teach anyone a lesson. It's taking place in Kitty Ladd's abandoned mansion. And each of the would-be gigolos has been paid one thousand dollars, in 1927 money, for his services. You don't shell out that kind of cash and not expect some action. And... how to put this... there's action. It's not explicit, but there's one scene that leaves about as much doubt as to what happened as the conveniently timed fade-out in a James Bond movie.
This is based on a stage play, and a lot of comedy comes through in the title cards, especially those used to introduce the characters. E.g. "Ethel Drake, whose conscience is spotless... and who has consequently led a very dull life." Or for her husband: "Howard Drake, a husband at such the cutest age. Leave him alone and he'll play for hours!" There's a racist joke in a title card near the beginning, but at least the movie depicts a 1920s fraternity that admits Jewish members. Yes, I will apply modern standards to this silent film, thank you very much. Especially since The Cradle Snatchers has a number of character names that appear to be in-jokes inserted by a time traveller: "George Martin", "Ann Hall", and "Howard Drake."
Actually "Howard Drake" is probably a Howard Hawks self-insert.
- Dog Day Afternoon (1975): I missed this last month at the theater, but then I picked up a cheap DVD at a yard sale. And wow, what a great movie. Al Pacino shines as the guy who just wanted to rob a bank, is that so much to ask? Spot-on performances by everyone inside the bank. Everyone outside the bank is drawn kinda broadly, especially Angie and Leon, Sonny's two wives, who are pretty simplistic stereotypes. But the relationships between Sonny and his wives were believable.
These old movies (see also Fig Leaves above) keep surprising me with the way they deal with gender and sexuality. It's a mix of human decency and wince-inducing stereotypes. There's an exchange from Taxi Driver that kind of sums it up for me. Bunch of taxi drivers are swapping stories.
Driver 1: In California, when two fags split up, one's gotta pay the other alimony.
Driver 2: Not bad. They're way ahead out there.
Dog Day Afternoon also does an amazing job of maintaining tension. It uses the same trick I saw in There Will Be Blood. The characters spend the entire movie in a state of extreme danger, but there are no "action scenes" and almost no actual violence. Good stuff.
- La Jetée (1962): Well, this is embarrassing. This film was never properly explained to me, or else I wasn't paying attention. I somehow got the idea that the whole thing played out over the single static image of the airport terminal seen during the opening credits. I'd watched the first minute or so online and given up because I don't want to watch a picture of an airport for half an hour. (I'm looking at you, Andy Warhol.) But the double feature with Twelve Monkeys gave me a reason to force myself to see it, and it turns out the film is a series of static images, not just the airport terminal. And it's pretty good! My rule is not to expect hard SF from 1960s French movies, but as long as they're doing genre work, they're all right by me.
- Twelve Monkeys (1995): This movie is kind of a mess, and definitely suffers by comparison to La Jetée. Brad Pitt's performance as a crazy dude is embarrassing. The romantic subplot is both creepy and boring. Why don't you start La Jetéeing and stop La Jetéerorizing me?
That may be a little harsh. I did have a good time watching Twelve Monkeys. The plot is nice and convoluted, Bruce Willis does a great job, and there's lots of Terry Gilliam set-dressing insanity.
Given the combination of "a big Terry Gilliam mess" and "Leonard had a good time watching it," I find it kind of odd that Twelve Monkeys became a big hit. The museum's handout guide to the movie was an interview with Gilliam in which he mentioned that the big tent-pole movies of the season—Nixon and Casino—kind of flopped, which gave Twelve Monkeys an opportunity to move in for the kill.
There's a scene at the very end which I saw and immediately thought "Aha! The scene that completely changes the tone of the movie, which Universal forced Gilliam to include!" But IMDB trivia says Gilliam had final cut on Twelve Monkeys. So I guess he wanted that scene. It's a funny scene, and although it invalidates the whole premise of the film, it doesn't do so unambiguously.
- Only Angels Have Wings (1939): Can't believe Hal convinced me to watch this instead of I Was A Male War Bride. War Bride has Cary Grant with (presumably) a French accent in (presumably) a goofy comedy. That sounds awesome. This movie has Cary Grant as a macho stereotype. The kind of character generally associated with John Wayne, although every John Wayne performance I've seen has more nuance than I was expecting. But Cary Grant is so emotionless in this film that in the thrilling climax, a pilot uses his stiff upper lip as an emergency runway.
Most of the characters in this movie are based on pilots Howard Hawks encountered while doing location scouting in South America. I admire this movie's willingness to kill characters at random, and I can see how a real person in that situation would retreat into a shell of stoicism and refuse all human contact. But it's not very entertaining. Cary Grant is probably my favorite actor, but he's my favorite actor because I like the way he conveys various emotions. Don't take that away from me!
- Trent's Last Case (1929):Without a doubt the worst movie I've seen at the museum. (The benefits of having skipped Trash Humpers.) I nearly fell asleep, even though it's only 50 minutes long and silent movies don't generally make me sleepy.
This is Howard Hawks's final silent film, not the 1952 Orson Welles film that's based on the same book. We saw the only print in existence, so I will summarize the terribleness. The movie was originally going to be a talkie. One of the lead actors had damaged vocal cords, and I guess at the dawn of the talkie period it was conceptually funny to have an actor with damaged vocal cords be in a talkie. Once they started shooting it turned out not to be funny in execution, so they turned a talkie into a silent film at the last minute. Alternate explanation I've seen online: they had the rights to make a silent adaptation of the book, but not the sound rights.
Either way, that lack of attention to detail is typical of Trent's Last Case. As this Finnish review says, "The Howard Hawks approach is unrecognizable." It's just terrible. Here's Hawks's opinion, from IMDB trivia:
It was presented at a Howard Hawks retrospective in the mid-'70s and when Hawks found it was on the playlist he asked out loud, "You really aren't going to play this, are you?" Midway during the showing of the film Hawks walked up to the projection room and demanded the projectionist destroy the print of the film.
Little-known fact: this was the incident that led Crow T. Robot to form the Film Anti-Preservation Society.
There were some scattered, halfhearted laughs at the foppish PI, but only one gag in the movie was really funny, and I'll tell you it so as to kill any interest you might have in the movie. The villain is in the process of framing his secretary for a crime. The secretary's back is turned. The villain is a classic melodrama villain with a banker's suit and a little moustache. He's really hamming it up, chortling, wringing his hands in glee, about to foreclose on the proverbial orphanage. And then the secretary looks up, into a mirror, sees the villain prancing around behind his back, and gives him a look like, "what the hell are you doing?"
That's great. It's a joke you couldn't do in a talkie. But it doesn't justify the rest of this dumb movie.
- Scarface (1983): I missed the 1932 Scarface due to RESTFest, but returned in time to catch this monstrosity. I'm not really sure how this movie fits into the Hannah Montana continuity, but Al Pacino is always engaging, and it was really interesting to see all the stuff that Breaking Bad took from this movie (remember the famous BB elevator pitch, "From Mr. Chips to Scarface"). From obvious visuals like the pools of blue water and the scenes on the drug lord's patio, to thematic elements like "main character's attempt to provide for his family destroys his family."
I also found it really interesting that everyone remembers the full-on, screaming, coke-snorting, grenade-launching Tony Montana from the end of this movie, like he's some kind of badass. But that guy is a failure! He's a broken man. He's like that because he's lost everything. He's got cocaine smudged on his nose and he doesn't even notice. For most of the movie he's a lot calmer, more cunning, and a more effective badass/antihero.
The '80s abstract art and beachfront architecture in this movie is amazing! And no wonder—turns out much of the movie was filmed in the Los Angeles of my childhood. Take that, Miami, you cultural backwater!
And finally, I've kept this hidden so far but I didn't like this movie all that much. It's nearly three hours long and its plot is very predictable. Michelle Pfeiffer is bland as the serial trophy wife. And it's got a bad case of Hamlet cliches, because before seeing this movie I experienced the most famous cultural children of Scarface: violent gangster-themed video games like Grand Theft Auto and Hotline Miami. Those games are better. They have really dumb plots, but it's not like Scarface has a great plot. It has a very well-realized protagonist, and everything else has aged poorly. I would rather play through Hotline Miami again.
By the way, does anyone else think that Hotline Miami really needs a roguelike element? Randomly generated floor plans? That would be great.
- Breaking Bad (2008-2013): I thought I could justify putting this in the film roundup because we had a plan to SEE [the series finale] BIG at the museum, which fell through for a couple reasons, but this post remains the logical place to talk about the series as a whole.
This is the first time Sumana and I have been fans of a show that was also hugely popular with the mainstream. It was a really weird experience. Genre shows are becoming more popular, but non-genre shows are not becoming more popular with me or Sumana. At the same time, Breaking Bad pushed my genre buttons in a way that, say, Arrested Development never did.
This bit of Tor.com revisionism got me thinking that Breaking Bad might secretly be genre fiction, and after the finale wrapped everything up with a nice bow (too nice, one might argue), I've decided that Breaking Bad is in fact a Mundane SF twist on the classic mad scientist story.
Every few months on Twitter I saw someone reinvent a joke about how in the Canadian adaptation of Breaking Bad, Canadian Walter White finds out he has cancer, the government pays for his chemo, and that's the end of the show. But something very similar happens during the first season of the American Breaking Bad. See, American Walter White has some rich private-sector friends. They find out about his cancer and they offer to pay for his treatments. But he refuses, because he envies and hates his rich friends. A long time ago they cheated him, denied him his scientific triumph, condemned him to a life of obscurity and humiliation. Now they mock him and they want him to beg them for charity? Pah! Never! The fools! He'll show them all!
That's your mad scientist origin story right there, and it's also the point where Sumana and I lost all sympathy for Walt. The rest of Breaking Bad did a great job of a) creating a story we loved watching despite having no sympathy for the main character, and b) showing what it means, day-by-day, to be a mad scientist.
Sometimes you're up, sometimes you're down. Sometimes your technical know-how saves the day, like in a Tom Swift book. Sometimes your henchmen fail you, sometimes you get cornered and forced into a bad situation. And every time you achieve what you thought you wanted, it turns out not to be what you actually wanted, because you're a freaking mad scientist and your insane desires do not reliably correspond to your real needs.
The part of Breaking Bad that isn't about a mad scientist and his family is about the relationships between supervillains and their henchmen. So many great henchmen in this show. Sometimes a henchman makes a bid for power, more often they're comic relief or raised-eyebrow loyalty, sometimes they get in a villain's way and they just gotta die.
The two Breaking Bad scenes that really stick in my mind are both about the weird genre-fictional state of being a henchman. The first is Ted Beneke's IRS audit, in which Skyler White, the classic henchman who's smarter than the boss, saves her clueless boss by pretending to be the clueless henchman who screwed everything up. The second is the train heist, because that's the first real Todd sequence. The whole episode I'm thinking "Oh, man, now they're dragging Todd into their web of lies." I've got Todd pegged as the easygoing, dumb-jock henchman, like Jimmy Bond from The Lone Gunmen. And then at the end of the train heist, Todd reveals himself as the most evil person in a show staffed almost entirely by bad guys. It's no surprise Todd is one of the few henchmen who makes a bid for power, and oh, man, I love these dramatic switches. Good job, Vince Gilligan.
What's up for October? More Howard Hawks, it looks like. See ya then.
Wed Oct 02 2013 09:41 Beautiful Soup 4.3.2, and all previous versions:
Through long practice I'm able to write decent code while I'm sick, but I should not try to release code while I'm sick. While putting up the release of Beautiful Soup 4.3.2, I accidentally deleted the entire beautifulsoup4 project on PyPI and had to recreate it manually. I've given PyPI all the crummy.com tarball URLs for releases going back to 4.0.1, and I've installed each one via pip to verify that it works, so if your build process depends on installing a specific version of Beautiful Soup 4 via PyPI, it should still work. And indeed, random versions of BS4 have been downloaded about 200 times since I switched over. I'm sorry about this screwup. Let me know if there are any remaining problems.
4.3.2 itself is a pretty minor bugfix release. Still left unfixed is a bug I can't reproduce because the federal government is shut down. When you file a bug that happens with a specific web page, please provide the HTML of the web page, not the URL.
Mon Sep 30 2013 12:44 Smooth Unicode:
For reasons of his own, Adam Parrish recently created the Unicode Ebooks Twitter bot. I offered some helpful suggestions for improving the visual appeal of the Unicode Ebooks, suggestions which Adam mocked as unworthy of his artistic vision of dumping a bunch of line noise onto Twitter every five minutes.
So I created my own Twitter bot: Smooth Unicode, the Lite FM to Adam's unending Einstürzende Neubauten concert. My bot does its best to construct aesthetically pleasing output by combining scripts that complement each other visually. The code is part of olipy and I'll be adding to it as I come up with more nice-looking ways to present gibberish.
Less talk. Less noise. More browser-visible glyphs. That's Smooth Unicode.
(2) Wed Sep 25 2013 09:56 LCODC$SSU:
At RESTfest last week I put on an old Mozilla shirt and my Al Gore campaign button and gave a talk from the year 2000: "LCODC$SSU and the coming automated web". I'll link to video when it goes up on Vimeo, and I'll also point to my five-minute talk about ALPS, which not only took five minutes to deliver, it took five minutes to put together.
But right now, there's some more stuff I want to say about "LCODC$SSU", and some stuff I couldn't say in the talk due to the framing device.
When I first mentioned this talk to Mike Amundsen, he told me about Bret Victor's talk from 1974, "The Future of Programming", which Victor gave in July and which had a similar conceit. Victor is also a much better actor than I am, but I went ahead with my talk because wanted to do something different with "LCODC$SSU" than happens in "The Future of Programming". I get a strong "You maniacs! You blew it up!" vibe from Victor's talk. And there's some of that at the end of "LCODC$SSU"—I really feel like we've spent thirteen years making five years worth of progress, as you can see from my frustration at the beginning of "How to Follow Instructions"—but I also wanted to do some new things in my talk.
While writing Appendix C of RESTful Web APIs I came to appreciate the Fielding dissertation as a record of the process used to solve an enormous engineering problem. Comments from RESTFest attendees confirm that seeing it this way helps folks grasp the dissertation's gem: the definition of LCODC$SSU (a.k.a. REST). Thinking about it this way doesn't require a historical-fiction framing device (Appendix C has no such framing device), but it does require you stop treating the Fielding dissertation as a prescient guide to the 21st century and see it as a historical record of the 1990s.
And once you do that, the missing stair we've been jumping over or falling through for thirteen years becomes visible.
The Web works because it has four domain requirements that reinforce each other: low entry-barrier, distributed hypermedia, extensibility, and Internet scale. But there's also a fifth implicit requirement: the presence of a slow, expensive human being operating the client and making the final call on every single state transition. In the talk I identified the inverse of this implicit requirement as an explicit requirement: "machine legibility". In RESTful Web APIs we use the term "semantic gap" to describe what happens when you remove the implicit requirement.
Making the human unnecessary on a transition-by-transition basis (the goal of "Web APIs" as a field) is a really difficult problem, and it's partly because of the phenomenon I describe in the talk and in RWA Appendix C. Getting rid of the human raises the entry-barrier dramatically. Looking around for a cheap way to lower the entry-barrier, we decide to get rid of distributed hypermedia. But distributed hypermedia is the only thing that allows Internet-scale and extensibility to coexist! We must lose one or the other. We end up with an increasingly ugly system that can never be changed, or else a fascist dystopia.
And here's the bit I couldn't put in the talk because it would break the framing device. We've seen a decade-long obsession with lowering entry-barrier at any cost, and although the cost has been enormous I can't really say the obsession is misplaced. Low entry-barrier is the main reason why the Web succeeded over all other hypermedia systems. Low entry-barrier drives adoption. You get adoption first and you deal with the other problems (which will be enormous) down the road.
Well, we're down the road. The bills are coming due. If we want this to go more smoothly next time, we need to stop chasing entry-barrier local minima and come up with a better solution. We need to make change easier so we can make progress faster.
The "machine legibility" problem will still be very difficult, and frankly I can't see a way to a complete solution. But there's cause for optimism: every step forward we've taken so far has illuminated the space a little more and made the next step visible.
It's always been this way. That's how hypermedia works. That's why I called my now-infamous 2008 QCon talk "Justice Will Take Us Millions Of Intricate Moves" (after William Stafford), and that's why I take my motto from a Johnny Cash song that's probably not on most peoples' list of inspirational Johnny Cash songs.
I built it one piece at a time.
Mon Sep 23 2013 14:28 RESTful Web APIs Monkeypatch:
The RESTful Web APIs ebook came out earlier than we thought it would, and there are some important URLs in the book that don't work yet: the home page at restfulwebapis.org, and the example application at youtypeitwepostit.com. There's also one URL in the book (the book's GitHub repository) that will never work, because we wrote down the wrong URL.
I've submitted an erratum for the wrong URL, and I'm here to give you some temporary URLs that will work for the other stuff. They're temporary because Mike controls the DNS for restfulwebapis.org and youtypeitwepostit.com, and he's out of commission at the moment.
Tue Sep 10 2013 15:35 Awesome Dinosaurs Update:
- On Sunday I saw the 1926 Howard Hawks film Fig Leaves. I'll publish a full review in the roundup at the end of the month, but I couldn't wait to mention the dinosaurs! This movie (briefly) features two very cool-looking puppet dinosaurs. There's Adam's pet Apatosaurus, named Dobbin:
Exactly as depicted in Genesis 2.
More amazingly, there's also a budget-busting life-sized Triceratops that pulls a bus!
Awesome! Not gonna spoil the review, but the first reel of this movie used all the good Flintstones jokes, thirty-four years before The Flintstones even premiered. Except for the unfortunate bus dinosaur saying "It's a living." in a morose voice. And I'm sure that's just because the joke would be really awkward if you had to do it with title cards.
(Screen image simulated.)
- If you share my belief that dinosaurs are the most interesting part of any movie that includes dinosaurs, you'll love Kevin Maher's deleted scene from King Kong.
- A recent Ureddit course on narrative structure in short fiction used "Let Us Now Praise Awesome Dinosaurs" as one of its example stories. I thought this was a) a good choice, and b) pretty funny, because I deliberately wrote "Dinosaurs" to be opaque to traditional analyses of narrative structure.
If you'll forgive me being serious about a very silly story, here's what I mean. Nearly every plot event in "Dinosaurs" is a red herring. It's actually a New Yorker type story, in which a series of insane infernokrusher interventions leads to Entippa's epiphany that humans are exploiting dinosaurs' tendency to get involved in insane infernokrusher interventions for their own entertainment. (Those humans including, in a bit of Hitchcock-type moralizing, you for reading the story and me for writing it.)
I wrote the first scene to have something very close to a literal Chekhov's gun. It's Tark's gun, or at least his desire for a gun. Later on, Chekhov's gun goes off: Tark gets his gun! But as soon as the literal gun goes off, Tark discovers that literal guns are loud and painful, and he throws it away. The Chekhov's gun was fake. Sort of like the keys in my old text adventure Degeneracy, which don't unlock anything—you're supposed to melt them down for the metal.
But! In the Reddit thread dissecting "Dinosaurs" and the other example stories, the person running the class proves my intellectual superior. It turns out there was also a real Chekhov's Gun in that first scene: Tark's "killing claws", which are in fact used to kill someone later in the story, just like they would in a regular story about dinosaurs killing humans.
I didn't even notice that. I'd assumed the human-killing scene worked because everyone knows meat-eating dinosaurs have claws. I didn't even realize I'd made a big deal about the claws in the first scene. You win this round, literary analysis!
PS: Never forget.
(2) Mon Sep 09 2013 13:39 RESTful Web APIs!:
After about a year of work, my and Mike Amundsen's new book RESTful Web APIs is going to the printer. It's a replacement for RESTful Web Services, a book that's now seven years old. The replacement may be overdue, but it's only been in the past couple years that technology and attitudes have advanced to the point where I could write the book I wanted to write.
In fact, there's one subfield (profiles) where you could argue this book is premature. The way RESTful Web Services was a little premature in describing an OAuth-like system before OAuth was released. But I don't think we can wait any longer.
Back in February I discussed the differences between APIs and Services. That hasn't changed much, though we have added more stuff:
- A chapter on Linked Data, the Semantic Web approach to REST.
- A chapter on CoAP, the fabled "RESTful system that doesn't use HTTP", designed to connect embedded systems over low-power networks.
- An appendix that explicates the Fielding dissertation from an API designer's perspective.
This post is mainly my way of asking you to pre-order your copy of RESTful Web APIs through my O'Reilly affiliate link. That's a hypermedia-driven change in resource state which will get you the book in a couple weeks, and get me some extra cash. (I estimate about $1.70 extra. Don't do this if the shipping charge on a physical book is prohibitive, or whatever.)
But this post is also a back-door way for me to brag about what a great book Mike and I have written. You don't have to take my word for it. Here's the blurb we got from John Musser of ProgrammableWeb.
A terrific book! Covers a lot of new ground with lots of valuable specifics.
Here's Steve Klabnick of Designing Hypermedia APIs:
The entire time I read this book, I was cursing. I was cursing because as I read each explanation, I was worried that they were so good that it would be hard to find a better one to use in my own writing. You will not find another work that explores the topic so thoroughly yet explains the topic so clearly. Please, take these tools, build something fantastic, and share it with the rest of the world, okay?
You get the picture. I've tried to recreate the relevatory experience a lot of people got from RESTful Web Services, on a higher level, in a way that gives access to more powerful tools. Time will tell if I've succeeded, but I don't think I, or anyone, could have done much better. I'm really proud of this book, and I hope it helps you.
Tue Sep 03 2013 14:13 August Film Roundup:
Not the blockbuster month as I was anticipating—I missed all of the museum's big-name Pacino/de Niro movies due to other committments—but a lot of interesting movies, and movies that were uninteresting in interesting ways, among the nine I did see.
- Baikonur (2011): Taking the logic of Star Wars to an extreme, Baikonur shows space travel in a dingy, lived-in future: the one we have now. And that part of the movie is awesome! But the plotline is creepy hurt/comfort nerd fantasy with a litle reactionary agrarianism thrown in. So I can't really recommend it. But there's no other movie that can match these spectacular visuals of Baikonur Cosmodrome, the city outside it, the Kazakhstan steppe, and actual Soyuz launches. It's your call.
As a bonus, I would like to quote this bit of trivia from Kim Newman's review from Screen Daily:
Wary of international cinema after Borat, the Kazakh authorities were evidently persuaded to support this effort by a strategic decision to cast the favourite grand-daughter of the President in a small, key role (which the little girl aces) in the climax.
- You're A Big Boy Now (1966): Saw this as an experimental control. How much would I enjoy a movie if the old-school New York shabbiness was the only interesting thing about the movie? And I was foiled, because of course that's not the only interesting thing about Francis Ford Coppola's first movie. It's a raunchy sex comedy that looks exactly like a Disney film of the same period; say, The Computer Wore Tennis Shoes. There's a hilarious triangle of relationships between the main character and his parents. But... not a great movie, overall. Thumbs up for the Freudian automat comedy though.
- Cotton Comes to Harlem (1970): This was more like it. "Con man posing as religious figure" is one of those tropes that gets me every time. It's got action, comedy, quotable lines, and of course classic location shoots.
Now, here's the thing. Like G.O.B. Bluth, I'm white. So when I watch a comedy made by black Americans for black audiences in the 1970s, I frequently find myself deciding "all right, I assume the filmmakers know what they're doing, I'm going to laugh at this." And yet feeling kind of nervous about laughing. This happened sporadically during Emma Mae back in February, and it happened pretty much throughout Cotton Comes to Harlem.
E.g. this movie has a white character idiotically try to disguise himself with blackface. The tactic is even less effective than when the Marx Brothers tried it in A Day at the Races. It's kinda funny even in 2013 because it's such an obviously dumb idea, not like A Day at the Races where you think Harpo's magic might allow it to actually work, but seriously movie, you're doing a blackface joke? Similarly for a lot of the humor about the cultural divide between
urban and rural blacks. (Emma Mae sided with the hicks; Cotton Comes to Harlem is very pro-city, although most of its humor is at the expense of the city slickers.)
None of this is supposed to be particularly transgressive! It's a zany '70s studio-indie film. The contemporaneous Times review mentions the most jaw-dropping moment of the movie in a casual aside. But times have changed. If you made this film today it would be disjointed: half Hollywood-friendly buddy-cop stuff and half edgy in-your-face comedy.
The guest curator who introduced the movie said he thought director Ossie Davis shot his wad too early with the excellent car chase at the beginning of the movie. (Classic sight gag: guy in top hat and tails watches the car chase with glee, then pulls on white gloves and runs back into his storefront; turns out he runs the local funeral parlor.) And maybe so, but the movie ends with a fight scene in the Apollo Theater's prop room, and I think that's a pretty good bookend.
- Bye Bye Braverman (1968): A ton of classic New York location shots in this movie, but they're not particularly grimy. Mostly in Brooklyn. And this is a hard movie to get into. I guess I'd compare it to Seinfeld: very Jewish, unapologetically New York, and not really caring whether you get the joke or not. Like, you're supposed to understand quite a lot about the four main characters based on which Manhattan neighborhood they live in. There's a brief bit of conversation that you might or might not notice as a throwaway joke about the class differences between the Times and the Daily News. And so on.
The stand-out bits are a young Jessica Walter in a minor role, and three show-stopping set-piece rants. The first two are by stand-up comics: Godfrey Cambridge (the funny man to Raymond St. Jacques's straight man in Cotton Goes to Harlem) as a cab driver, and Alan King as a rabbi delivering a eulogy. The third is a rambling, moving monologue by star George Segal, bringing the inhabitants of a cemetery up to date as a way of facing his own mortality. ("TV is really good... pollution is bad... we're going to the moon!... We discovered cures for some diseases that might have kept you alive a little longer; you're not really missing much.")
If you're the sort of 1960s Jewish intellectual depicted in this movie (as director Sidney Lumet clearly was) I'm pretty sure you'll enjoy it. But it doesn't have a lot of crossover appeal, the way a Woody Allen film does. I think I got most of the jokes, and it was still a tough slog for me.
- Norman Mailer v Fun City, USA (1970) according to the Internet, this is a.k.a. "51st State" and "The Other Guys Are The Joke", and according to the program notes it's "Norman Mailer vs. Fun City", but I'm writing down exactly what I remember seeing on the title card, because there is almost no information about this film on the Internet. It has no IMDB page. It's like the VHS tape you discover at the beginning of a creepypasta. I could tell you anything about this film and you'd have to believe it. For instance:
Remember Norman Mailer's huge futurist Lego apartment building? Well, that model is the aesthetic linchpin of this movie which doesn't seem to exist! Director Dick Fontaine uses window reflections to superimpose the huge Lego structure onto the real-life New York skyline, blocking out the real buildings as Mailer explains his frankly insane vision for an 200-story apartment block that will house fifty thousand people, some of whom ought to be "adventurous" types interested in renting an apartment on the tip of one of the structures, which might sway five feet back and forth in a high wind.
It's a clear metaphor for Mailer's mayoral campaign and his Napoleon of Notting Hill-esque platform for making New York City the nation's 51st state, turning the neighborhoods into townships, and devolving the power of the mayor's office onto the townships. After the primary, in which Mailer gets a surprisingly high five percent of the vote, you see the Lego structure again, but this time there are no reflection tricks; perspective integrates the Lego building with the skyline behind it.
I tend to think of writers as introverts, but Norman Mailer is definitely an extrovert. And I think of smart extroverts as being natural politicians, but Mailer is a terrible politician. And the personality feature that makes him a terrible politician is the feature I recognize in him as a fellow writer. It's what led him to build that Lego model and to imagine a guy who's excited to rent an apartment where you have to bolt the furniture to the floor to keep it from sliding back and forth.
He hates being boring. He hates for things to be like they always have been. He thinks that he can win an election by making the election really interesting, so that the obvious next plot point is that he wins.
And he knows this about himself. From a museum-provided contextual interview that barely mentions the nonexistent movie it's contextualizing:
[B]eing in these kinds of things is never easy. At a certain point you go into overdrive and you feel something ugly in your ego functioning. You are selling something you don’t quite believe in. Why? To keep the movie moving and to keep it interesting so you aren’t a bore like other people you see in documentaries.
The other reason Norman Mailer is a terrible politician is that he constantly overrides the much better political operatives he somehow got to work his campaign. In particular, he has very bad judgement about radio ads. Just thought I'd mention that; a little freelance political criticism to go with the film review.
BTW, that interview also has this gem:
Did you feel afterwards that Don had any kind of obligation to put the camera down and intervene?
[Mishearing the question] I always assume God to be much too occupied. I see God as a tired general.
No, not God. Don. D.A. Pennebaker.
Oh! Boy, I thought we really getting into top gear fast.
- Superfly (1972): Curator Warrington Hudlin started announcing this film in a dry sort of way, but in a Shakespearean move Paul Anthony (I'm pretty sure it was him, but not 100%) of House Party fame rose up out of the audience, interrupted Hudlin's speech, and demanded that he put on a '70s trenchcoat and a funkier hat, which loosened him right up. And that kind of set the stage for Superfly, a movie that strongly prioritizes style over substance.
It definitely has the style, capturing both the sleaze of the '70s streets and the tackiness of the middle-class '70s interior shots. But the substance... there's some good excitement at the end, and a great musical montage in which people of all races come together to buy cocaine from Priest; the sort of sardonic commentary that Breaking Bad also does really well. But most of what I remember is people driving around really slowly like they're looking for parking—a classic low-budget tell, as anyone who's watched a lot of MST3K knows.
I know not everyone shares my preference for comedy over drama, so I understand why Superfly is considered a proto-blaxploitation classic while I'd never even heard of Cotton Comes to Harlem. But I think it's a failing of Superfly that its grittiness never made as uncomfortable as did the lighthearted comedy of Cotton Comes to Harlem.
- Little Murders (1971): I actually saw this movie when I was about fifteen! At least, I saw the first ten minutes. My mom showed me Harold and Maude and this movie, and I didn't think Harold and Maude was that interesting, but ten minutes into Little Murders I was like "Okay, Mom, this is too dark, turn it off." And then I saw it again this month and I was like "Oh, shit, it's this movie. I can't leave the theater or they'll think I'm a wuss. Well, I'm not fifteen anymore, I can take it."
And I can indeed take it, but seriously, this is probably the darkest movie I've ever seen. It's a zombie movie where it turns out the main character was a zombie the whole time, just waiting for everyone else to become zombies. If that's your cup of tea (and you won't be disappointed when it turns out I don't mean "zombies" 100% literally and Elliot Gould doesn't go shuffling around biting people in the head), here's your comedic nihilism-fest.
You may be wondering why my mother was showing this movie to her fifteen-year-old son. It's a good question, and the true answer will never be known, but I think there were two reasons. First, both this movie and Harold and Maude came out in 1971, when my mother was in college. I think she thought I was finally at a point where I could appreciate these movies the way she had, the way we had bonded over Monty Python and the Holy Grail (1975) when I was about twelve. She was wrong, but these things happen.
Second, as I've alluded to before, as a kid I had serious emotional problems. Not something we need to talk about now, but definitely something to worry about if you're the parent of such a kid. And now that I've finally made it all the way through Little Murders, I think I see why my mom rented these particular movies. She was trying to show me that there are people with darkness within them, horrors that fifteen-year-old Leonard can't even imagine, but who are able to channel the darkness to creative ends and generally be productive members of society (screenwriter Jules Feiffer illustrated my beloved Phantom Tollbooth, for heaven's sake!), without "selling out" or closing their eyes to society's problems. So thanks for that, mom.
- The Angel Levine (1970): I admit I wasn't expecting much from a movie that's literally about a Magical Negro sent to help a white guy. But an interesting thing happens: Harry Belafonte's character turns out to have agency! He doesn't want to be a guardian angel; he wants his old life back. He doesn't want to be dead. And he fails to get what he wants, like everyone else in this depressing-ass movie, but at least he tried. Throw in some more trope psych-outs, like making you think this is the kind of movie where no one else can see the angel, but no, everyone can see him, he was just using the bathroom during one scene; and you've won me over. It's definitely below this month's median, but if you need a cinematic antidote to It's A Wonderful Life, here it is.
On the scale of "use of classic New York grime in location shots", I would rate this movie: very poor. To quote Paul Zimmerman's Newsweek review, "[Director Ján] Kádar's unfamiliarity with New York shows. His camera views the city as if it were a tourist unwilling to wander too far from his hotel." Zero Mostel is great as always.
- The World's End (2013): Loved it. I won't go into a lot of detail because there's a reasonable chance you're planning on seeing it in the theater, but it was really fun. My biggest complaint is it maybe needed one fewer main character.
After thinking about the Cornetto Trilogy as a whole, I went to IMDB and saw that the collective shares my opinion: all three movies are great, but Hot Fuzz is better than The World's End (7.9 vs. 7.8), and Shaun of the Dead (8.0) is better than Hot Fuzz. Honestly, at this point you know what to expect. You hear a bit of banter at the beginning of the movie, know that it precisely foreshadows the events of the movie, and it doesn't even matter. It's like knowing that Ulysses is based on the Odyssey. That's not a "spoiler"; it's the structure of the piece. We have here three great movies around two eternal themes: society as a threat to individuality, and the indestructable love between Simon Pegg and Nick Frost, regardless of who is playing the responsible one and who is playing the screwup.
In case you haven't had enough of my idiosyncratic approach to movie quality (and you probably haven't, or why are you reading this), consider the following: if I had to pick only one movie from the Cornetto Trilogy, it would be Hot Fuzz. Not Shaun of the Dead, which is definitely a better and more influential movie. Because Hot Fuzz is the movie that caters to my specific kink: stories about people obsessed with stories, to the point where they let the stories run their lives, who get to save the day when they're suddenly thrust into a situation where the rules from the stories are the only ones that apply.
This month and next the museum is showing every film Howard Hawks ever made, so search for his name on IMDB and prepare for the Cary Grant-fest. SEE IT BIG is also returning, and I'm looking forward to seeing the Howard Hawks Scarface on the 21st and then the Brian De Palma Scarface on the 22nd.
(1) Thu Aug 01 2013 11:59 July Film Roundup:
Oh man. As promised last month, July was an epic month of moviewatching, and I decided to try a little epic experiment with this roundup, inspired by the "The Balcony is Closed" game on No More Whoppers. For every movie I saw in July, I came up with a nonobvious connection between that movie and every other movie I saw in July. For instance, if I saw both Die Hard and Live Free or Die Hard, the connection between them would of course be "fresh-faced hacker".
I saw nine movies over the course of the month (well, eight and a half), and by the end this exercise became kind of ridiculous, as I strained to remember obscure aspects of earlier movies. But I knew it would become ridiculous, so when it did, I had no standing to complain. Here we go:
- Citizen Kane (1941) For years I have searched for an answer to that unanswerable question, "What is the Citizen Kane of games of movies?" What movie is held up as an unattainable example for what movies could be if only moviemakers would get their acts together and make some proper art? Then, one day, it struck me: perhaps Citizen Kane was the Citizen Kane of games of movies.
But I hadn't seen Citizen Kane in over ten years. I'd only seen it twice. The only solution was to go to the museum and SEE IT BIG. Only then would I know whether or not cinema was a worthy art form.
And... it's not. Because how could it be, with those kind of expectations heaped upon it? But Citizen Kane is a great movie. Just one example of its greatness: I'm pretty sure the reel changes don't sync with scene changes. You'll see a reel change coming up and it will just cut from one camera angle to another angle on the same shot.
I became acutely aware of reel changes ever since seeing a very metatextual episode of Columbo and I can't emphasize how bizarre this is. Movies made fifty years after Citizen Kane have abrupt scene changes at reel changes, but near as I can tell Citizen Kane just says "screw that, we're telling the story at its own pace and we trust people to not misplace an entire film canister." Or whatever the normal reason is for syncing scene changes to reel changes.
But not all of Citizen Kane's experiments hold up. The newsreel at the beginning is a really clever way to do a huge infodump and set up a framing device, but back in the day watching two newsreels in quick succession would have been super annoying, and now that newsreels are extinct, it just feels like a huge infodump.
So, tragically, Citizen Kane cannot be said to be the Citizen Kane of games of movies. But keep trying to meet those impossible, irrelevant expectations, filmmakers!
This seems like a good time to reveal the secret I've been keeping for years: the Citizen Kane of video games is Legend of the Bystander from Constellation Games. That's the game you get when you translate Citizen Kane's dramatic structure—someone circling around the past in flashback, unable to change or understand anything—into game form. Is it a good game? No! It's a weird, confused, frustrating game. So stop searching for it.
- Bonus connection with last month's No: dirty-tricks election.
- Sunrise (1927): A chilling tale of the cycle of domestic violence. This film starts out with the most crushing melodrama imaginable. Then there's a goofy series of skits about barbershop misunderstandings and piglets getting drunk and flappers' shoulder straps falling off. Then, back to the crushing melodrama! It's insane, but it works. (At least, the second and third acts work.) The goofy stuff makes you think the tension has been resolved, even though the movie's only half over. But you're a fool for thinking so! The first bit of tension was a ruse, and the goofy skits are secretly building up the real tension, which when it breaks has real emotional impact.
- Connection with Citizen Kane: Man trying to buy back his wife's love.
- Greed (1924): The Breaking Bad of its time. If you wanted to see Samwise Gamgee as a dentist who sexually assaults his patients, this is your silent film. If you wanted to see vaudeville-era German stereotypes, or the Cliff House in its heyday, or Oakland back when it was just a train station... Greed has it all. Well, it doesn't have about 5 1/2 hours of footage which was cut by the studio, but you can find most of that stuff in the book.
- Connection with Citizen Kane: Girl meets boy because she has a toothache.
- Connection with Sunrise: Disastrous rainstorm.
- Do the Right Thing (1989): After we watched Ace in the Hole on Criterion DVD, we saw a special feature in which Spike Lee talked about his love for that film. I didn't make much of it at the time, because the only Spike Lee film I'd seen at the time was Malcolm X. But now it's clear. Spike Lee shares Billy Wilder's interest in comedy that turns to tragedy and farce that slips into fiasco. Do the Right Thing is the same kind of slow-burn catastrophe as Ace in the Hole.
I have a few complaints: some stretches are boring, there's some exploitative boobs. But Mookie's strong through-line lets this movie avoid the "sketch comedy" feel you sometimes get when a movie has a whole lot of characters.
- Connection with Citizen Kane: Something highly valued gets burned.
- Connection with Sunrise: Commotion in a restaurant.
- Connection with Greed: Heatstroke!
- Reds (1981): I really wasn't in the mood for a 3.5-hour movie so I left during intermission. What I should have done was come in at intermission, because right before intermission the film found its focus and got super interesting and spectacular. But it wasn't enough to get me to stay. The studio really should have gotten someone to butcher the first half to about 20 minutes, a la Greed, but I'm sure those contracts were ironclad. I'm thinking you could just show the scenes with Jack Nicholson as Eugene O'Neill, and when Diane Keaton storms out of the house, show a title card saying "Such were the Reeds."
So... I can't properly review this movie because I'm disappointed by the half I saw and I long for the half I didn't see. I will say that if for some reason you genuinely hate Ishtar, this can be your "Warren Beatty is a doofy American in over his head" movie instead.
I gotta say, though, I never got the feeling that I was watching the 1910s. Diane Keaton looked just like she did in Annie Hall, and Warren Beatty looked just like he would in Ishtar.
- Connection with Citizen Kane: Wandering around an abandoned palace.
- Connection with Sunrise: Domesticated animal running all over the place.
- Connection with Greed: A wayward dentist's wife.
- Connection with Do the Right Thing: Directly addressing the camera.
- Bonus Breaking Bad connection: epiphany upon seeing something inside a copy of Leaves of Grass.
- The Right Stuff (1983): The other half of my mashup, Do The Right Stuff. I've mentioned before how Tom Wolfe's book changed my perception of manned space travel. The movie isn't as good as the book, but it's very good, and it does a good job of exploring what I consider the book's primary topic: the adoption of the test pilot ethos as a model for nationalist heroism in an era where nuclear weapons have rendered traditional macho heroism irrelevant.
What I didn't expect from this movie was that it would also show the simple, uncomplicated heroism that occurs when people stand up for each other. When the Mercury astronauts stand together against fake-Wernher von Braun and demand better treatment than the space program chimps. When John Glenn jeopardizes his career by refusing to pressure his wife to talk to LBJ, and the other astronauts have his back. And if you don't like that stuff and you wanna read the film as a celebration of Chuck Yeager stealing a plane and crashing it for no real reason, that's in there too.
- Connection with Citizen Kane: Dive bar.
- Connection with Sunrise: Man tempted by floozy.
- Connection with Greed: The desert sucks.
- Connection with Do the Right Thing: Burning photos on the wall.
- Connection with Reds: The Russians got there first.
- Apollo 13 (1995): I loved this movie when I saw it in
the theater, and I think I love it even more now. It continues The
Right Stuff's exploration of heroism by showing a space mission
that produced nothing else. It shows what The Right
Stuff didn't: people sticking together in a genuine life-threatening situation. (In real life there was even more sticking together than in the movie, which invented Fred Haise's pissy hatred of Jack Swigert. Or at least invented it coming out over a live comm; Sumana and I read over the transcript and we think they kept it pretty professional, all things considered.)
Best of all, Apollo 13 brings the nerds into the loop. Max Grodénchik is a big hero as FIDO Gold (SYMBOLISM), and it's not played for laughs the way it always was on DS9. I think that's why they stunt-casted Ed Harris as chief nerd Gene Kranz; it sort of gives you a bridge from The Right Stuff.
Oh, no, wait, best of all, the sainted Billy Wilder liked this movie! From poorly-worded IMDB trivia:
Over the course of lunch with his idol Billy Wilder, Ron Howard has said that he was thrilled to learn that Wilder deemed this movie to be Howard's best work as a director because it was about a guy who did NOT realize his dream...
That's our Billy!
- Connection with Citizen Kane: Cost? No man can say.
- Connection with Sunrise: An unorthodox lifeboat.
- Connection with Greed: I think we can all agree that Greed is also a movie about a guy who does not realize his dream.
- Connection with Do the Right Thing: Exposition broadcast over radio.
- Connection with Reds: Old person doesn't remember famous person's name.
- Connection with The Right Stuff: Can't think of one, sorry.
- Howdy, y'all. Joe Hills here, recording as I always do from
Nashville (1975). I'd never seen an Altman film before, and
this one plays out like a whole season of Arrested Development in one
movie. There's a complicated network of relationships between self-absorbed characters that plays out in a funny, horrifying way. Perhaps the cleverest move is to give the knee-jerk Hollywood-liberal approach to 1970s Nashville ("these hicks are crazy") to the British reporter Opal, a clueless, snobbish foreigner around whom all red-blooded Americans can unite in mockery. The songs are always bad in just the right way.
- Connection with Citizen Kane: Pretending that someone who
can't sing, can.
- Connection with Sunrise: Big city musical hooplah!
- Connection with Greed: Awkward family dinner.
- Connection with Do the Right Thing: Exploitative use of boobs.
- Connection with Reds: Foreign journalist has a poor grasp
of what's going on.
- Connection with The Right Stuff: Unexpected Jeff Goldblum.
- Connection with Apollo 13: Flashy white clothing.
- There Will Be Blood (2007): I'd just like to state for the record that Kern County is exactly as depicted in this movie. Even though they filmed it in Texas.
Uh, yeah, this was pretty good. Just your basic movie about men digging themselves into moral cesspools and foreclosing on any possibility of redemption, but better than its competitors thanks to Daniel Day-Lewis's amazing acting. I liked the passive-aggressive rivalry between his character and Paul Dano's. I'm really excited that director Paul Thomas Anderson is doing an adaptation of Inherent Vice. (Although I think that adaptation might be better if Wes Anderson did it.)
I also want to point out what a great title this is. It's kind of cheesy. Other movie titles don't make explicit promises. And I can't imagine someone squirming on their theater seat at the two-hour mark thinking, "Well, I'd leave, but I was told there would be blood." But it works. The title sets up a tension that lasts the entire movie. There are all these moments of horrific violence and symbolic stand-ins for blood, but you never see literal blood until the very end.
- Connection with Citizen Kane: Hey, there's oil on your property!
- Connection with Sunrise: Wedding as act break.
- Connection with Greed: The sordid true story of Minecraft.
- Connection with Do the Right Thing: Brother set against brother, because one of them is an asshole.
- Connection with Reds: Poor labor conditions.
- Connection with The Right Stuff: Something explodes before it should.
- Connection with Apollo 13: Something explodes that shouldn't have exploded at all.
- Connection with Nashville: Deaf kid.
This month the museum panders to me with a festival of classic crime and grime. New York in the 1970s: a lousy place to live, a great place to make a movie about. Looking forward to seeing films like Cotton Comes to Harlem, Serpico, Superfly, The French Connection, and Across 110th Street. We'll probably also catch some Wong Kar-Wai. I will not be repeating July's movie connection experiment.
Correction: "I'd never seen an Altman film before" is one of the least accurate claims I've ever made. I've seen Gosford Park, The Company, and A Prairie Home Companion. And I've probably seen M*A*S*H, given how often they showed it on Comedy Central back when I was in high school. But I came out of all those films thinking "that was good/terrible/okay", whereas I came out of Nashville thinking "No wonder this guy's a legend!" It was like watching a whole nother director.
Mon Jul 29 2013 16:41 Loaded Dice 2013 Update:
I fetched the BoardGameGeek data again, a yearly tradition, and put up another Loaded Dice update.
A few highlights:
- The number of games released each year is holding steady at about 3200.
- Rating inflation continues since last year, but it seems to be slowing down.
- Games that were highly underrated in 2011 were overrated in 2012, as BGG users overcompensated for their former low ratings. Games that were highly overrated in 2012 were still overrated in 2013--these were highly hyped games and the hype is still dying down.
If you go to the main page, you can download an amazing 17-megabyte JSON dump of BGG data I've compiled. It includes descriptions and genres for every game in the dataset, and three data samples that convey historical rating data over three years. At this point I feel like I'm adding enough on top of what the BGG API can give you (the historical rating data) that I can make the data dump available without apology.